2023-12-08T08:24:53.431 INFO:root:teuthology version: 0.0.1.dev214+gf2de108 2023-12-08T08:24:53.431 DEBUG:teuthology.run:Teuthology command: teuthology --name vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} --archive /home/teuthworker/archive/vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi/7483789 --verbose --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi/7483789/orig.config.yaml 2023-12-08T08:24:53.473 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-08T08:24:53.546 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi/7483789 branch: wip-vshankar-testing-20231206.125818 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} email: vshankar@redhat.com first_in_suite: false job_id: '7483789' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20231206.125818 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: a2cd52c00149b5caff897ea7f79d0529b62620da ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a2cd52c00149b5caff897ea7f79d0529b62620da thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20231206.125818 sha1: a2cd52c00149b5caff897ea7f79d0529b62620da owner: scheduled_vshankar@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 4099 sha1: a2cd52c00149b5caff897ea7f79d0529b62620da sleep_before_teardown: 0 subset: 359/1024 suite: fs suite_branch: wip-vshankar-testing-20231206.125818 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a2cd52c00149b5caff897ea7f79d0529b62620da/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a2cd52c00149b5caff897ea7f79d0529b62620da syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF/oq4PR8P0nJ1+Zg8yIxbCvd5uplT1Kp/CNd9kDgv628WHtoI5X3/30LU03pgRbYeqM0LZ1UwK54Ov5z9suKKw= smithi112.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGGxxAwWuNsIuQ1f/M3QMGpbiwr2qHjOiVWTolZZ5TlFO+WC81Y8dSPRPc2VYZtZkaCNTNp/UNxvCK+rcOPWngw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: f2de108beb0a6dde7f3e6c96039071828a1f6269 timestamp: 2023-12-08_07:46:48 tube: smithi use_shaman: true user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-08T08:24:53.546 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a2cd52c00149b5caff897ea7f79d0529b62620da/qa; will attempt to use it 2023-12-08T08:24:53.547 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a2cd52c00149b5caff897ea7f79d0529b62620da/qa/tasks 2023-12-08T08:24:53.547 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-08T08:24:53.548 INFO:teuthology.task.internal:Checking packages... 2023-12-08T08:24:53.571 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a2cd52c00149b5caff897ea7f79d0529b62620da' 2023-12-08T08:24:53.571 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-08T08:24:53.571 INFO:teuthology.packaging:ref: None 2023-12-08T08:24:53.571 INFO:teuthology.packaging:tag: None 2023-12-08T08:24:53.571 INFO:teuthology.packaging:branch: wip-vshankar-testing-20231206.125818 2023-12-08T08:24:53.571 INFO:teuthology.packaging:sha1: a2cd52c00149b5caff897ea7f79d0529b62620da 2023-12-08T08:24:53.571 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20231206.125818 2023-12-08T08:24:53.716 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-46.ga2cd52c0 2023-12-08T08:24:53.716 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-08T08:24:53.728 INFO:teuthology.task.internal:no buildpackages task found 2023-12-08T08:24:53.728 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-08T08:24:53.757 INFO:teuthology.task.internal:Saving configuration 2023-12-08T08:24:53.773 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-08T08:24:53.785 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-08T08:24:53.811 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi/7483789', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-08 08:19:51.904148', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF/oq4PR8P0nJ1+Zg8yIxbCvd5uplT1Kp/CNd9kDgv628WHtoI5X3/30LU03pgRbYeqM0LZ1UwK54Ov5z9suKKw='} 2023-12-08T08:24:53.836 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi112.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi/7483789', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-08 08:19:51.905975', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGGxxAwWuNsIuQ1f/M3QMGpbiwr2qHjOiVWTolZZ5TlFO+WC81Y8dSPRPc2VYZtZkaCNTNp/UNxvCK+rcOPWngw='} 2023-12-08T08:24:53.836 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-08T08:24:53.848 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-12-08T08:24:53.848 INFO:teuthology.task.internal:roles: ubuntu@smithi112.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-12-08T08:24:53.848 INFO:teuthology.run_tasks:Running task console_log... 2023-12-08T08:24:53.925 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f10f7010a60>, signals=[15]) 2023-12-08T08:24:53.925 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-08T08:24:53.936 INFO:teuthology.task.internal:Opening connections... 2023-12-08T08:24:53.936 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2023-12-08T08:24:53.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T08:24:54.037 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi112.front.sepia.ceph.com 2023-12-08T08:24:54.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T08:24:54.124 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-08T08:24:54.137 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2023-12-08T08:24:54.155 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2023-12-08T08:24:54.156 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-12-08T08:24:54.213 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2023-12-08T08:24:54.213 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2023-12-08T08:24:54.213 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2023-12-08T08:24:54.213 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2023-12-08T08:24:54.213 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2023-12-08T08:24:54.213 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2023-12-08T08:24:54.213 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-08T08:24:54.214 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2023-12-08T08:24:54.214 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2023-12-08T08:24:54.214 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-08T08:24:54.214 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2023-12-08T08:24:54.214 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-08T08:24:54.214 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-08T08:24:54.214 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-08T08:24:54.214 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-12-08T08:24:54.239 DEBUG:teuthology.orchestra.run.smithi112:> uname -m 2023-12-08T08:24:54.256 INFO:teuthology.orchestra.run.smithi112.stdout:x86_64 2023-12-08T08:24:54.257 DEBUG:teuthology.orchestra.run.smithi112:> cat /etc/os-release 2023-12-08T08:24:54.316 INFO:teuthology.orchestra.run.smithi112.stdout:NAME="CentOS Stream" 2023-12-08T08:24:54.317 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION="9" 2023-12-08T08:24:54.317 INFO:teuthology.orchestra.run.smithi112.stdout:ID="centos" 2023-12-08T08:24:54.317 INFO:teuthology.orchestra.run.smithi112.stdout:ID_LIKE="rhel fedora" 2023-12-08T08:24:54.317 INFO:teuthology.orchestra.run.smithi112.stdout:VERSION_ID="9" 2023-12-08T08:24:54.317 INFO:teuthology.orchestra.run.smithi112.stdout:PLATFORM_ID="platform:el9" 2023-12-08T08:24:54.317 INFO:teuthology.orchestra.run.smithi112.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-08T08:24:54.317 INFO:teuthology.orchestra.run.smithi112.stdout:ANSI_COLOR="0;31" 2023-12-08T08:24:54.317 INFO:teuthology.orchestra.run.smithi112.stdout:LOGO="fedora-logo-icon" 2023-12-08T08:24:54.317 INFO:teuthology.orchestra.run.smithi112.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-08T08:24:54.317 INFO:teuthology.orchestra.run.smithi112.stdout:HOME_URL="https://centos.org/" 2023-12-08T08:24:54.317 INFO:teuthology.orchestra.run.smithi112.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-08T08:24:54.317 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-08T08:24:54.318 INFO:teuthology.orchestra.run.smithi112.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-08T08:24:54.318 INFO:teuthology.lock.ops:Updating smithi112.front.sepia.ceph.com on lock server 2023-12-08T08:24:54.339 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-08T08:24:54.354 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-08T08:24:54.389 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-08T08:24:54.389 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2023-12-08T08:24:54.391 DEBUG:teuthology.orchestra.run.smithi112:> test '!' -e /home/ubuntu/cephtest 2023-12-08T08:24:54.407 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-08T08:24:54.416 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-08T08:24:54.416 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2023-12-08T08:24:54.449 DEBUG:teuthology.orchestra.run.smithi112:> test -z $(ls -A /var/lib/ceph) 2023-12-08T08:24:54.482 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-08T08:24:54.540 INFO:teuthology.run_tasks:Running task kernel... 2023-12-08T08:24:54.560 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-12-08T08:24:54.561 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-12-08T08:24:54.561 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-08T08:24:54.562 DEBUG:teuthology.orchestra.run.smithi112:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-08T08:24:54.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T08:24:54.577 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2023-12-08T08:24:54.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T08:24:54.578 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2023-12-08T08:24:54.636 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-361.el9.x86_64 2023-12-08T08:24:54.636 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.14.0-361.el9.x86_64 2023-12-08T08:24:54.637 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2023-12-08T08:24:54.638 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-361.el9.x86_64 2023-12-08T08:24:54.638 INFO:teuthology.task.kernel:Running kernel on smithi112: 5.14.0-361.el9.x86_64 2023-12-08T08:24:54.638 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum install -y kernel 2023-12-08T08:24:57.281 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS-9 - AppStream 7.1 MB/s | 6.4 MB 00:00 2023-12-08T08:24:57.431 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - AppStream 5.3 MB/s | 6.4 MB 00:01 2023-12-08T08:24:58.806 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - BaseOS 7.5 MB/s | 2.0 MB 00:00 2023-12-08T08:24:58.914 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS-9 - BaseOS 3.5 MB/s | 2.0 MB 00:00 2023-12-08T08:25:00.055 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 7.9 MB 00:00 2023-12-08T08:25:01.765 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 3.0 MB/s | 7.9 MB 00:02 2023-12-08T08:25:03.214 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 8.8 MB/s | 18 MB 00:02 2023-12-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 18 MB 00:03 2023-12-08T08:25:07.590 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 5.8 MB 00:01 2023-12-08T08:25:11.334 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 5.8 MB 00:01 2023-12-08T08:25:28.380 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 956 B/s | 15 kB 00:15 2023-12-08T08:25:29.792 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 16 MB/s | 20 MB 00:01 2023-12-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2023-12-08T08:25:36.996 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-12-08T08:25:36.996 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-12-08T08:25:37.086 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-12-08T08:25:37.095 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-12-08T08:25:37.096 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2023-12-08T08:25:37.096 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-12-08T08:25:37.096 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-12-08T08:25:37.096 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-390.el9 baseos 5.6 M 2023-12-08T08:25:37.096 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-12-08T08:25:37.096 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core x86_64 5.14.0-390.el9 baseos 20 M 2023-12-08T08:25:37.096 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules x86_64 5.14.0-390.el9 baseos 38 M 2023-12-08T08:25:37.096 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core x86_64 5.14.0-390.el9 baseos 32 M 2023-12-08T08:25:37.096 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:25:37.096 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-12-08T08:25:37.096 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-12-08T08:25:37.096 INFO:teuthology.orchestra.run.smithi016.stdout:Install 4 Packages 2023-12-08T08:25:37.096 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:25:37.097 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 96 M 2023-12-08T08:25:37.097 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 121 M 2023-12-08T08:25:37.097 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-12-08T08:25:38.341 INFO:teuthology.orchestra.run.smithi016.stdout:(1/4): kernel-5.14.0-390.el9.x86_64.rpm 5.8 MB/s | 5.6 MB 00:00 2023-12-08T08:25:39.066 INFO:teuthology.orchestra.run.smithi016.stdout:(2/4): kernel-core-5.14.0-390.el9.x86_64.rpm 12 MB/s | 20 MB 00:01 2023-12-08T08:25:39.933 INFO:teuthology.orchestra.run.smithi016.stdout:(3/4): kernel-modules-5.14.0-390.el9.x86_64.rpm 15 MB/s | 38 MB 00:02 2023-12-08T08:25:40.767 INFO:teuthology.orchestra.run.smithi016.stdout:(4/4): kernel-modules-core-5.14.0-390.el9.x86_6 13 MB/s | 32 MB 00:02 2023-12-08T08:25:40.767 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-12-08T08:25:40.767 INFO:teuthology.orchestra.run.smithi016.stdout:Total 26 MB/s | 96 MB 00:03 2023-12-08T08:25:40.821 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 473 B/s | 15 kB 00:32 2023-12-08T08:25:41.679 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-12-08T08:25:42.104 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-12-08T08:25:42.104 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-12-08T08:25:43.462 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-12-08T08:25:43.463 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-12-08T08:25:46.260 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-12-08T08:25:46.963 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-core-5.14.0-390.el9.x86_64 1/4 2023-12-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-core-5.14.0-390.el9.x86_64 2/4 2023-12-08T08:25:49.973 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-390.el9.x86_64 2/4 2023-12-08T08:25:50.175 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-5.14.0-390.el9.x86_64 3/4 2023-12-08T08:25:50.291 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 2.1 MB/s | 20 MB 00:09 2023-12-08T08:25:55.065 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2023-12-08T08:25:55.154 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-390.el9.x86_64 3/4 2023-12-08T08:25:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-5.14.0-390.el9.x86_64 4/4 2023-12-08T08:25:57.278 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2023-12-08T08:25:57.278 INFO:teuthology.orchestra.run.smithi112.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2023-12-08T08:25:57.367 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-12-08T08:25:57.379 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-12-08T08:25:57.379 INFO:teuthology.orchestra.run.smithi112.stdout: Package Architecture Version Repository Size 2023-12-08T08:25:57.379 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-12-08T08:25:57.379 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-12-08T08:25:57.379 INFO:teuthology.orchestra.run.smithi112.stdout: kernel x86_64 5.14.0-390.el9 baseos 5.6 M 2023-12-08T08:25:57.379 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2023-12-08T08:25:57.379 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core x86_64 5.14.0-390.el9 baseos 20 M 2023-12-08T08:25:57.379 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules x86_64 5.14.0-390.el9 baseos 38 M 2023-12-08T08:25:57.379 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core x86_64 5.14.0-390.el9 baseos 32 M 2023-12-08T08:25:57.379 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:25:57.379 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-12-08T08:25:57.379 INFO:teuthology.orchestra.run.smithi112.stdout:================================================================================ 2023-12-08T08:25:57.380 INFO:teuthology.orchestra.run.smithi112.stdout:Install 4 Packages 2023-12-08T08:25:57.380 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:25:57.380 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 96 M 2023-12-08T08:25:57.380 INFO:teuthology.orchestra.run.smithi112.stdout:Installed size: 121 M 2023-12-08T08:25:57.380 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-12-08T08:25:58.398 INFO:teuthology.orchestra.run.smithi112.stdout:(1/4): kernel-5.14.0-390.el9.x86_64.rpm 5.7 MB/s | 5.6 MB 00:00 2023-12-08T08:25:59.190 INFO:teuthology.orchestra.run.smithi112.stdout:(2/4): kernel-core-5.14.0-390.el9.x86_64.rpm 12 MB/s | 20 MB 00:01 2023-12-08T08:25:59.848 INFO:teuthology.orchestra.run.smithi112.stdout:(3/4): kernel-modules-core-5.14.0-390.el9.x86_6 22 MB/s | 32 MB 00:01 2023-12-08T08:26:00.279 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-core-5.14.0-390.el9.x86_64 4/4 2023-12-08T08:26:00.290 INFO:teuthology.orchestra.run.smithi112.stdout:(4/4): kernel-modules-5.14.0-390.el9.x86_64.rpm 13 MB/s | 38 MB 00:02 2023-12-08T08:26:00.290 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-12-08T08:26:00.290 INFO:teuthology.orchestra.run.smithi112.stdout:Total 33 MB/s | 96 MB 00:02 2023-12-08T08:26:01.145 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-12-08T08:26:01.522 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-12-08T08:26:01.523 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-12-08T08:26:02.856 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-12-08T08:26:02.856 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-12-08T08:26:05.760 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-12-08T08:26:06.391 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-core-5.14.0-390.el9.x86_64 1/4 2023-12-08T08:26:06.532 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-core-5.14.0-390.el9.x86_64 2/4 2023-12-08T08:26:09.253 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-390.el9.x86_64 2/4 2023-12-08T08:26:09.434 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-modules-5.14.0-390.el9.x86_64 3/4 2023-12-08T08:26:14.051 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-390.el9.x86_64 3/4 2023-12-08T08:26:14.253 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : kernel-5.14.0-390.el9.x86_64 4/4 2023-12-08T08:26:18.840 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-core-5.14.0-390.el9.x86_64 4/4 2023-12-08T08:26:30.744 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-390.el9.x86_64 4/4 2023-12-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-390.el9.x86_64 4/4 2023-12-08T08:26:31.185 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-5.14.0-390.el9.x86_64 4/4 2023-12-08T08:26:31.185 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-390.el9.x86_64 1/4 2023-12-08T08:26:31.185 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-core-5.14.0-390.el9.x86_64 2/4 2023-12-08T08:26:31.185 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-5.14.0-390.el9.x86_64 3/4 2023-12-08T08:26:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-core-5.14.0-390.el9.x86_64 4/4 2023-12-08T08:26:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:26:31.628 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-12-08T08:26:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-5.14.0-390.el9.x86_64 2023-12-08T08:26:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core-5.14.0-390.el9.x86_64 2023-12-08T08:26:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-5.14.0-390.el9.x86_64 2023-12-08T08:26:31.628 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core-5.14.0-390.el9.x86_64 2023-12-08T08:26:31.629 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:26:31.629 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-12-08T08:26:32.119 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2023-12-08T08:26:32.259 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-390.el9.x86_64 2023-12-08T08:26:32.259 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-390.el9.x86_64 2023-12-08T08:26:32.259 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-390.el9.x86_64 2023-12-08T08:26:32.259 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-08T08:26:32.259 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-08T08:26:32.260 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-12-08T08:26:32.260 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2023-12-08T08:26:32.260 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2023-12-08T08:26:32.382 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-390.el9.x86_64 2023-12-08T08:26:32.382 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-390.el9.x86_64 2023-12-08T08:26:32.383 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2023-12-08T08:26:32.420 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2023-12-08T08:26:32.420 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2023-12-08T08:26:32.420 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2023-12-08T08:26:32.420 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 68.el9 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8264254 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2023-12-08T08:26:32.421 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:26:32.422 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-08T08:26:32.422 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-08T08:26:32.422 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-08T08:26:32.422 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2023-12-08T08:26:32.422 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:26:32.422 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2023-12-08T08:26:32.423 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-08T08:26:32.423 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 5.14.0-390.el9.x86_64 2023-12-08T08:26:32.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-08T08:26:33.053 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2023-12-08T08:26:34.556 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-12-08T08:26:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:done 2023-12-08T08:26:34.581 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2023-12-08T08:26:34.596 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.7xWNddsUY4 2023-12-08T08:26:34.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7xWNddsUY4 2023-12-08T08:26:34.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.7xWNddsUY4 2023-12-08T08:26:34.821 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.7xWNddsUY4 is 6KB 2023-12-08T08:26:34.870 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.7xWNddsUY4 2023-12-08T08:26:34.885 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2023-12-08T08:26:34.952 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-12-08T08:26:34.952 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-12-08T08:26:34.953 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-12-08T08:26:34.953 INFO:teuthology.orchestra.run.smithi016.stdout:15949e3c88704300882482eddfd7baae-5.14.0-390.el9.x86_64.conf 2023-12-08T08:26:34.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-390.el9.x86_64 2023-12-08T08:26:35.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2023-12-08T08:26:35.199 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2023-12-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-core-5.14.0-390.el9.x86_64 4/4 2023-12-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-modules-5.14.0-390.el9.x86_64 4/4 2023-12-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: kernel-5.14.0-390.el9.x86_64 4/4 2023-12-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-5.14.0-390.el9.x86_64 1/4 2023-12-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-core-5.14.0-390.el9.x86_64 2/4 2023-12-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-5.14.0-390.el9.x86_64 3/4 2023-12-08T08:26:48.596 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : kernel-modules-core-5.14.0-390.el9.x86_64 4/4 2023-12-08T08:26:48.596 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:26:48.596 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-12-08T08:26:48.596 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-5.14.0-390.el9.x86_64 2023-12-08T08:26:48.596 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-core-5.14.0-390.el9.x86_64 2023-12-08T08:26:48.596 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-5.14.0-390.el9.x86_64 2023-12-08T08:26:48.596 INFO:teuthology.orchestra.run.smithi112.stdout: kernel-modules-core-5.14.0-390.el9.x86_64 2023-12-08T08:26:48.596 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:26:48.597 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-12-08T08:26:49.090 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2023-12-08T08:26:49.236 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-390.el9.x86_64 2023-12-08T08:26:49.236 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-390.el9.x86_64 2023-12-08T08:26:49.236 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-390.el9.x86_64 2023-12-08T08:26:49.236 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-08T08:26:49.236 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-08T08:26:49.236 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-12-08T08:26:49.236 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi112.front.sepia.ceph.com, path=None, version=distro) 2023-12-08T08:26:49.236 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q kernel | sort -rV | head -n 1 2023-12-08T08:26:49.354 INFO:teuthology.orchestra.run.smithi112.stdout:kernel-5.14.0-390.el9.x86_64 2023-12-08T08:26:49.354 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-390.el9.x86_64 2023-12-08T08:26:49.354 DEBUG:teuthology.orchestra.run.smithi112:> sudo rpm -qi grub2-tools 2023-12-08T08:26:49.388 INFO:teuthology.orchestra.run.smithi112.stdout:Name : grub2-tools 2023-12-08T08:26:49.388 INFO:teuthology.orchestra.run.smithi112.stdout:Epoch : 1 2023-12-08T08:26:49.388 INFO:teuthology.orchestra.run.smithi112.stdout:Version : 2.06 2023-12-08T08:26:49.388 INFO:teuthology.orchestra.run.smithi112.stdout:Release : 68.el9 2023-12-08T08:26:49.388 INFO:teuthology.orchestra.run.smithi112.stdout:Architecture: x86_64 2023-12-08T08:26:49.388 INFO:teuthology.orchestra.run.smithi112.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-12-08T08:26:49.388 INFO:teuthology.orchestra.run.smithi112.stdout:Group : Unspecified 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout:Size : 8264254 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout:License : GPLv3+ 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout:Packager : builder@centos.org 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout:Vendor : CentOS 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout:Summary : Support tools for GRUB. 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout:Description : 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-08T08:26:49.389 INFO:teuthology.orchestra.run.smithi112.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-08T08:26:49.390 INFO:teuthology.orchestra.run.smithi112.stdout:hardware devices. 2023-12-08T08:26:49.390 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:26:49.390 INFO:teuthology.orchestra.run.smithi112.stdout:This subpackage provides tools for support of all platforms. 2023-12-08T08:26:49.390 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-08T08:26:49.391 INFO:teuthology.task.kernel:Updating grub on smithi112 to boot 5.14.0-390.el9.x86_64 2023-12-08T08:26:49.391 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-08T08:26:49.984 INFO:teuthology.orchestra.run.smithi112.stderr:Generating grub configuration file ... 2023-12-08T08:26:51.364 INFO:teuthology.orchestra.run.smithi112.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-12-08T08:26:51.396 INFO:teuthology.orchestra.run.smithi112.stderr:done 2023-12-08T08:26:51.398 DEBUG:teuthology.orchestra.run.smithi112:> mktemp 2023-12-08T08:26:51.413 INFO:teuthology.orchestra.run.smithi112.stdout:/tmp/tmp.jyMhT8l8n0 2023-12-08T08:26:51.413 DEBUG:teuthology.orchestra.run.smithi112:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jyMhT8l8n0 2023-12-08T08:26:51.481 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0666 /tmp/tmp.jyMhT8l8n0 2023-12-08T08:26:51.621 DEBUG:teuthology.orchestra.remote:smithi112:/tmp/tmp.jyMhT8l8n0 is 6KB 2023-12-08T08:26:51.640 DEBUG:teuthology.orchestra.run.smithi112:> rm -fr /tmp/tmp.jyMhT8l8n0 2023-12-08T08:26:51.654 DEBUG:teuthology.orchestra.run.smithi112:> sudo /bin/ls /boot/loader/entries || true 2023-12-08T08:26:51.718 INFO:teuthology.orchestra.run.smithi112.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-12-08T08:26:51.718 INFO:teuthology.orchestra.run.smithi112.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-12-08T08:26:51.718 INFO:teuthology.orchestra.run.smithi112.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-12-08T08:26:51.718 INFO:teuthology.orchestra.run.smithi112.stdout:15949e3c88704300882482eddfd7baae-5.14.0-390.el9.x86_64.conf 2023-12-08T08:26:51.719 DEBUG:teuthology.orchestra.run.smithi112:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-390.el9.x86_64 2023-12-08T08:26:51.868 DEBUG:teuthology.orchestra.run.smithi112:> sudo shutdown -r now 2023-12-08T08:27:05.118 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-12-08T08:27:05.119 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-12-08T08:27:05.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T08:27:21.886 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-12-08T08:27:21.886 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-12-08T08:27:21.887 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T08:27:23.618 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-12-08T08:27:32.628 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-12-08T08:27:32.628 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T08:27:35.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-12-08T08:27:40.258 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-12-08T08:27:47.694 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-12-08T08:27:47.695 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T08:27:49.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-12-08T08:27:49.261 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T08:27:50.754 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.16 2023-12-08T08:27:52.322 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.112 2023-12-08T08:28:04.334 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-12-08T08:28:04.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T08:28:05.757 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-12-08T08:28:05.757 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T08:28:36.176 DEBUG:teuthology.orchestra.run.smithi112:> true 2023-12-08T08:28:36.525 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-12-08T08:28:36.525 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-390.el9.x86_64"... 2023-12-08T08:28:36.525 DEBUG:teuthology.orchestra.run.smithi112:> uname -r 2023-12-08T08:28:36.539 INFO:teuthology.orchestra.run.smithi112.stdout:5.14.0-390.el9.x86_64 2023-12-08T08:28:36.539 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-390.el9.x86_64 vs 5.14.0-390.el9.x86_64 2023-12-08T08:28:36.539 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-08T08:28:36.539 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-08T08:28:37.540 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-12-08T08:28:37.540 DEBUG:teuthology.orchestra.run.smithi112:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-08T08:28:37.624 INFO:teuthology.orchestra.run.smithi112.stdout:ttyS1 2023-12-08T08:28:37.671 DEBUG:teuthology.parallel:result is None 2023-12-08T08:28:37.687 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-12-08T08:28:38.110 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-12-08T08:28:38.111 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-390.el9.x86_64"... 2023-12-08T08:28:38.111 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2023-12-08T08:28:38.125 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-390.el9.x86_64 2023-12-08T08:28:38.125 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-390.el9.x86_64 vs 5.14.0-390.el9.x86_64 2023-12-08T08:28:38.125 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-08T08:28:38.125 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-08T08:28:39.126 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-12-08T08:28:39.126 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-08T08:28:39.194 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2023-12-08T08:28:39.231 DEBUG:teuthology.parallel:result is None 2023-12-08T08:28:39.231 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-08T08:28:39.239 INFO:teuthology.task.internal:Creating test directory... 2023-12-08T08:28:39.239 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-08T08:28:39.241 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-08T08:28:39.255 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-08T08:28:39.262 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-08T08:28:39.272 INFO:teuthology.task.internal:Creating archive directory... 2023-12-08T08:28:39.272 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-08T08:28:39.292 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-08T08:28:39.333 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-08T08:28:39.340 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-08T08:28:39.340 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-08T08:28:39.351 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-08T08:28:39.379 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-08T08:28:39.395 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-08T08:28:39.404 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-08T08:28:39.417 INFO:teuthology.orchestra.run.smithi112.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-08T08:28:39.418 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-08T08:28:39.431 INFO:teuthology.task.internal:Configuring sudo... 2023-12-08T08:28:39.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-08T08:28:39.435 DEBUG:teuthology.orchestra.run.smithi112:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-08T08:28:39.490 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-08T08:28:39.502 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-08T08:28:39.503 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-08T08:28:39.505 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-08T08:28:39.548 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-08T08:28:39.595 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-08T08:28:39.662 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-08T08:28:39.705 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-08T08:28:39.747 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:28:39.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-08T08:28:39.814 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-08T08:28:39.856 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-08T08:28:39.910 DEBUG:teuthology.orchestra.run.smithi112:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-08T08:28:39.994 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-08T08:28:40.040 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:28:40.040 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-08T08:28:40.110 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2023-12-08T08:28:40.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo service rsyslog restart 2023-12-08T08:28:40.178 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-08T08:28:40.201 INFO:teuthology.orchestra.run.smithi112.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-08T08:28:40.691 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-08T08:28:40.698 INFO:teuthology.task.internal:Starting timer... 2023-12-08T08:28:40.699 INFO:teuthology.run_tasks:Running task pcp... 2023-12-08T08:28:40.708 INFO:teuthology.run_tasks:Running task selinux... 2023-12-08T08:28:40.716 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2023-12-08T08:28:40.818 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2023-12-08T08:28:40.823 DEBUG:teuthology.orchestra.run.smithi112:> sudo service auditd rotate 2023-12-08T08:28:40.870 INFO:teuthology.orchestra.run.smithi112.stdout:Rotating logs: 2023-12-08T08:28:40.872 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-08T08:28:40.872 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2023-12-08T08:28:40.915 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2023-12-08T08:28:40.915 DEBUG:teuthology.orchestra.run.smithi112:> /usr/sbin/getenforce 2023-12-08T08:28:40.949 INFO:teuthology.orchestra.run.smithi112.stdout:Permissive 2023-12-08T08:28:40.949 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi112.front.sepia.ceph.com': 'permissive'} 2023-12-08T08:28:40.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-08T08:28:41.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T08:28:41.009 DEBUG:teuthology.orchestra.run.smithi112:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-08T08:28:41.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T08:28:41.037 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-08T08:28:41.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2023-12-08T08:28:41.093 DEBUG:teuthology.orchestra.run.smithi112:> sudo /usr/sbin/setenforce permissive 2023-12-08T08:28:41.126 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-08T08:28:41.136 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-12-08T08:28:41.141 INFO:teuthology.repo_utils:Fetching main from origin 2023-12-08T08:28:41.258 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-08T08:28:41.275 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-08T08:28:41.276 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi112.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-08T08:35:24.862 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi112.front.sepia.ceph.com')] 2023-12-08T08:35:24.863 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-12-08T08:35:24.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T08:35:24.965 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-12-08T08:35:25.034 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-12-08T08:35:25.034 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-12-08T08:35:25.035 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi112.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-08T08:35:25.118 DEBUG:teuthology.orchestra.run.smithi112:> true 2023-12-08T08:35:25.190 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi112.front.sepia.ceph.com' 2023-12-08T08:35:25.191 INFO:teuthology.run_tasks:Running task clock... 2023-12-08T08:35:25.202 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-08T08:35:25.202 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-08T08:35:25.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-08T08:35:25.205 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-08T08:35:25.205 DEBUG:teuthology.orchestra.run.smithi112:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-08T08:35:25.240 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-08T08:35:25.261 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-08T08:35:25.262 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-08T08:35:25.279 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-08T08:35:25.298 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2023-12-08T08:35:25.309 INFO:teuthology.orchestra.run.smithi112.stderr:sudo: ntpd: command not found 2023-12-08T08:35:25.315 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2023-12-08T08:35:25.322 INFO:teuthology.orchestra.run.smithi112.stdout:506 Cannot talk to daemon 2023-12-08T08:35:25.335 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-08T08:35:25.339 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-08T08:35:25.355 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-08T08:35:25.356 INFO:teuthology.orchestra.run.smithi112.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-08T08:35:25.398 INFO:teuthology.orchestra.run.smithi112.stderr:bash: line 1: ntpq: command not found 2023-12-08T08:35:25.401 INFO:teuthology.orchestra.run.smithi112.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-08T08:35:25.401 INFO:teuthology.orchestra.run.smithi112.stdout:=============================================================================== 2023-12-08T08:35:25.401 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-08T08:35:25.401 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-08T08:35:25.401 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-08T08:35:25.401 INFO:teuthology.orchestra.run.smithi112.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-08T08:35:25.402 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2023-12-08T08:35:25.406 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-08T08:35:25.406 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2023-12-08T08:35:25.406 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-08T08:35:25.406 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-08T08:35:25.406 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-08T08:35:25.406 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-08T08:35:25.407 INFO:teuthology.run_tasks:Running task install... 2023-12-08T08:35:25.415 DEBUG:teuthology.task.install:project ceph 2023-12-08T08:35:25.415 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a2cd52c00149b5caff897ea7f79d0529b62620da'}} 2023-12-08T08:35:25.415 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a2cd52c00149b5caff897ea7f79d0529b62620da'} 2023-12-08T08:35:25.415 INFO:teuthology.task.install:Using flavor: default 2023-12-08T08:35:25.421 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-08T08:35:25.421 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-12-08T08:35:25.422 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a2cd52c00149b5caff897ea7f79d0529b62620da', 'tag': None, 'wait_for_package': False} 2023-12-08T08:35:25.423 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a2cd52c00149b5caff897ea7f79d0529b62620da 2023-12-08T08:35:25.425 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a2cd52c00149b5caff897ea7f79d0529b62620da', 'tag': None, 'wait_for_package': False} 2023-12-08T08:35:25.425 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a2cd52c00149b5caff897ea7f79d0529b62620da 2023-12-08T08:35:25.571 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231206.125818/a2cd52c00149b5caff897ea7f79d0529b62620da/centos/9/flavors/default/ 2023-12-08T08:35:25.571 INFO:teuthology.task.install.rpm:Package version is 19.0.0-46.ga2cd52c0 2023-12-08T08:35:25.585 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231206.125818/a2cd52c00149b5caff897ea7f79d0529b62620da/centos/9/flavors/default/ 2023-12-08T08:35:25.585 INFO:teuthology.task.install.rpm:Package version is 19.0.0-46.ga2cd52c0 2023-12-08T08:35:25.694 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231206.125818/a2cd52c00149b5caff897ea7f79d0529b62620da/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231206.125818/a2cd52c00149b5caff897ea7f79d0529b62620da/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231206.125818/a2cd52c00149b5caff897ea7f79d0529b62620da/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-08T08:35:25.694 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:35:25.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-08T08:35:25.697 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231206.125818/a2cd52c00149b5caff897ea7f79d0529b62620da/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231206.125818/a2cd52c00149b5caff897ea7f79d0529b62620da/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231206.125818/a2cd52c00149b5caff897ea7f79d0529b62620da/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-08T08:35:25.697 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:35:25.697 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-08T08:35:25.724 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-08T08:35:25.725 DEBUG:teuthology.orchestra.run.smithi112:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a2cd52c00149b5caff897ea7f79d0529b62620da/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-08T08:35:25.729 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-08T08:35:25.730 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a2cd52c00149b5caff897ea7f79d0529b62620da/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-08T08:35:25.793 DEBUG:teuthology.orchestra.run.smithi112:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-08T08:35:25.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-08T08:35:25.873 DEBUG:teuthology.orchestra.run.smithi112:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-08T08:35:25.888 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-08T08:35:25.918 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2023-12-08T08:35:25.920 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2023-12-08T08:35:25.939 INFO:teuthology.orchestra.run.smithi112.stdout:check_obsoletes = 1 2023-12-08T08:35:25.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum clean all 2023-12-08T08:35:26.178 INFO:teuthology.orchestra.run.smithi016.stdout:73 files removed 2023-12-08T08:35:26.182 INFO:teuthology.orchestra.run.smithi112.stdout:73 files removed 2023-12-08T08:35:26.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-08T08:35:26.209 DEBUG:teuthology.orchestra.run.smithi112:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-08T08:35:26.873 INFO:teuthology.orchestra.run.smithi112.stdout:ceph packages for x86_64 224 kB/s | 78 kB 00:00 2023-12-08T08:35:26.976 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 180 kB/s | 78 kB 00:00 2023-12-08T08:35:27.249 INFO:teuthology.orchestra.run.smithi112.stdout:ceph noarch packages 29 kB/s | 9.5 kB 00:00 2023-12-08T08:35:27.334 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 31 kB/s | 9.5 kB 00:00 2023-12-08T08:35:27.600 INFO:teuthology.orchestra.run.smithi112.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2023-12-08T08:35:27.685 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2023-12-08T08:35:28.204 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - AppStream 13 MB/s | 6.4 MB 00:00 2023-12-08T08:35:28.210 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS-9 - AppStream 11 MB/s | 6.4 MB 00:00 2023-12-08T08:35:29.722 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.0 MB 00:00 2023-12-08T08:35:29.727 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS-9 - BaseOS 4.3 MB/s | 2.0 MB 00:00 2023-12-08T08:35:31.468 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 7.9 MB 00:01 2023-12-08T08:35:32.328 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 3.5 MB/s | 7.9 MB 00:02 2023-12-08T08:35:34.213 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 25 MB/s | 18 MB 00:00 2023-12-08T08:35:39.581 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 5.8 MB 00:01 2023-12-08T08:35:41.281 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 15 kB 00:00 2023-12-08T08:35:42.985 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2023-12-08T08:35:45.586 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 18 MB 00:13 2023-12-08T08:35:48.000 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 36 kB/s | 1.7 kB 00:00 2023-12-08T08:35:49.536 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 5.8 MB 00:00 2023-12-08T08:35:50.636 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-08T08:35:50.637 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-08T08:35:50.649 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-12-08T08:35:50.650 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-12-08T08:35:50.650 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-12-08T08:35:50.652 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-12-08T08:35:50.653 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-12-08T08:35:50.656 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-12-08T08:35:50.657 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-12-08T08:35:50.736 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-12-08T08:35:50.746 INFO:teuthology.orchestra.run.smithi016.stdout:========================================================================================== 2023-12-08T08:35:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-12-08T08:35:50.747 INFO:teuthology.orchestra.run.smithi016.stdout:========================================================================================== 2023-12-08T08:35:50.747 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-12-08T08:35:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-12-08T08:35:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 6.5 k 2023-12-08T08:35:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 5.2 M 2023-12-08T08:35:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 860 k 2023-12-08T08:35:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 141 k 2023-12-08T08:35:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 1.5 M 2023-12-08T08:35:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 133 k 2023-12-08T08:35:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 1.8 M 2023-12-08T08:35:50.747 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 7.4 M 2023-12-08T08:35:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 50 k 2023-12-08T08:35:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 8.6 M 2023-12-08T08:35:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 41 M 2023-12-08T08:35:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 262 k 2023-12-08T08:35:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 801 k 2023-12-08T08:35:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 217 k 2023-12-08T08:35:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 18 k 2023-12-08T08:35:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-12-08T08:35:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-12-08T08:35:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-12-08T08:35:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 31 k 2023-12-08T08:35:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 719 k 2023-12-08T08:35:50.748 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 126 k 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 161 k 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 320 k 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 298 k 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 99 k 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 85 k 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 3.0 M 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 170 k 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-12-08T08:35:50.749 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2023-12-08T08:35:50.750 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.45.7-1.el9 baseos 2.3 M 2023-12-08T08:35:50.750 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.45.7-1.el9 baseos 1.8 M 2023-12-08T08:35:50.750 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.45.7-1.el9 baseos 43 k 2023-12-08T08:35:50.750 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.45.7-1.el9 baseos 246 k 2023-12-08T08:35:50.750 INFO:teuthology.orchestra.run.smithi016.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-12-08T08:35:50.750 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-12-08T08:35:50.750 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-12-08T08:35:50.750 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-12-08T08:35:50.750 INFO:teuthology.orchestra.run.smithi016.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-12-08T08:35:50.750 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-12-08T08:35:50.750 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 3.3 M 2023-12-08T08:35:50.750 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 3.0 M 2023-12-08T08:35:50.751 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-12-08T08:35:50.751 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-12-08T08:35:50.751 INFO:teuthology.orchestra.run.smithi016.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-12-08T08:35:50.751 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-12-08T08:35:50.751 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-12-08T08:35:50.751 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-12-08T08:35:50.751 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 19 M 2023-12-08T08:35:50.751 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 28 k 2023-12-08T08:35:50.751 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 2.1 M 2023-12-08T08:35:50.751 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 243 k 2023-12-08T08:35:50.751 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 4.4 M 2023-12-08T08:35:50.751 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 18 M 2023-12-08T08:35:50.751 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 15 k 2023-12-08T08:35:50.752 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 24 k 2023-12-08T08:35:50.752 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-12-08T08:35:50.752 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-12-08T08:35:50.752 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-12-08T08:35:50.752 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-12-08T08:35:50.752 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-12-08T08:35:50.752 INFO:teuthology.orchestra.run.smithi016.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-12-08T08:35:50.752 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2023-12-08T08:35:50.752 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2023-12-08T08:35:50.752 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 164 k 2023-12-08T08:35:50.752 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-12-08T08:35:50.752 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-12-08T08:35:50.753 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-12-08T08:35:50.753 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-12-08T08:35:50.753 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-12-08T08:35:50.753 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 476 k 2023-12-08T08:35:50.753 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-12-08T08:35:50.753 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 4.8 M 2023-12-08T08:35:50.753 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-12-08T08:35:50.753 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-12-08T08:35:50.753 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-12-08T08:35:50.753 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-12-08T08:35:50.753 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-12-08T08:35:50.753 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-12-08T08:35:50.753 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2023-12-08T08:35:50.753 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-12-08T08:35:50.754 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-12-08T08:35:50.754 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-12-08T08:35:50.754 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-12-08T08:35:50.754 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-12-08T08:35:50.754 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 45 k 2023-12-08T08:35:50.754 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 129 k 2023-12-08T08:35:50.754 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-12-08T08:35:50.754 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-12-08T08:35:50.754 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-12-08T08:35:50.755 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-12-08T08:35:50.755 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-12-08T08:35:50.755 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-12-08T08:35:50.755 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-12-08T08:35:50.755 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-12-08T08:35:50.755 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-12-08T08:35:50.755 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-12-08T08:35:50.755 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-12-08T08:35:50.755 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-12-08T08:35:50.755 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-12-08T08:35:50.755 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-12-08T08:35:50.755 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-12-08T08:35:50.755 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-12-08T08:35:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-12-08T08:35:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-12-08T08:35:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-12-08T08:35:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-12-08T08:35:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-12-08T08:35:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-12-08T08:35:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-12-08T08:35:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-12-08T08:35:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-12-08T08:35:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-12-08T08:35:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-12-08T08:35:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-12-08T08:35:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-12-08T08:35:50.756 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-12-08T08:35:50.757 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-12-08T08:35:50.757 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-12-08T08:35:50.757 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-12-08T08:35:50.757 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-12-08T08:35:50.757 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-12-08T08:35:50.757 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-12-08T08:35:50.757 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-12-08T08:35:50.757 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-12-08T08:35:50.757 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-12-08T08:35:50.757 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-12-08T08:35:50.757 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-12-08T08:35:50.757 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-12-08T08:35:50.758 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-12-08T08:35:50.758 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-12-08T08:35:50.758 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-12-08T08:35:50.758 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-12-08T08:35:50.759 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-12-08T08:35:50.759 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-12-08T08:35:50.759 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-12-08T08:35:50.759 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-12-08T08:35:50.759 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-12-08T08:35:50.759 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-12-08T08:35:50.759 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-12-08T08:35:50.759 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-12-08T08:35:50.759 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2023-12-08T08:35:50.759 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-12-08T08:35:50.759 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:35:50.759 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-12-08T08:35:50.759 INFO:teuthology.orchestra.run.smithi016.stdout:========================================================================================== 2023-12-08T08:35:50.760 INFO:teuthology.orchestra.run.smithi016.stdout:Install 132 Packages 2023-12-08T08:35:50.760 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 15 Packages 2023-12-08T08:35:50.760 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:35:50.760 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 204 M 2023-12-08T08:35:50.760 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-12-08T08:35:51.600 INFO:teuthology.orchestra.run.smithi016.stdout:(1/147): ceph-19.0.0-46.ga2cd52c0.el9.x86_64.rp 26 kB/s | 6.5 kB 00:00 2023-12-08T08:35:51.755 INFO:teuthology.orchestra.run.smithi112.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 15 kB 00:00 2023-12-08T08:35:52.509 INFO:teuthology.orchestra.run.smithi016.stdout:(2/147): ceph-common-19.0.0-46.ga2cd52c0.el9.x8 16 MB/s | 19 MB 00:01 2023-12-08T08:35:52.559 INFO:teuthology.orchestra.run.smithi016.stdout:(3/147): ceph-immutable-object-cache-19.0.0-46. 2.8 MB/s | 141 kB 00:00 2023-12-08T08:35:52.626 INFO:teuthology.orchestra.run.smithi016.stdout:(4/147): ceph-fuse-19.0.0-46.ga2cd52c0.el9.x86_ 839 kB/s | 860 kB 00:01 2023-12-08T08:35:52.693 INFO:teuthology.orchestra.run.smithi016.stdout:(5/147): ceph-mds-19.0.0-46.ga2cd52c0.el9.x86_6 16 MB/s | 2.1 MB 00:00 2023-12-08T08:35:52.927 INFO:teuthology.orchestra.run.smithi016.stdout:(6/147): ceph-mon-19.0.0-46.ga2cd52c0.el9.x86_6 19 MB/s | 4.4 MB 00:00 2023-12-08T08:35:53.138 INFO:teuthology.orchestra.run.smithi112.stdout:Extra Packages for Enterprise Linux 16 MB/s | 20 MB 00:01 2023-12-08T08:35:53.602 INFO:teuthology.orchestra.run.smithi016.stdout:(7/147): ceph-mgr-19.0.0-46.ga2cd52c0.el9.x86_6 1.5 MB/s | 1.5 MB 00:00 2023-12-08T08:35:53.811 INFO:teuthology.orchestra.run.smithi016.stdout:(8/147): ceph-osd-19.0.0-46.ga2cd52c0.el9.x86_6 20 MB/s | 18 MB 00:00 2023-12-08T08:35:53.861 INFO:teuthology.orchestra.run.smithi016.stdout:(9/147): ceph-selinux-19.0.0-46.ga2cd52c0.el9.x 485 kB/s | 24 kB 00:00 2023-12-08T08:35:54.086 INFO:teuthology.orchestra.run.smithi016.stdout:(10/147): ceph-base-19.0.0-46.ga2cd52c0.el9.x86 1.9 MB/s | 5.2 MB 00:02 2023-12-08T08:35:54.208 INFO:teuthology.orchestra.run.smithi016.stdout:(11/147): cephfs-mirror-19.0.0-46.ga2cd52c0.el9 1.8 MB/s | 217 kB 00:00 2023-12-08T08:35:54.325 INFO:teuthology.orchestra.run.smithi016.stdout:(12/147): libcephfs-devel-19.0.0-46.ga2cd52c0.e 267 kB/s | 31 kB 00:00 2023-12-08T08:35:54.600 INFO:teuthology.orchestra.run.smithi016.stdout:(13/147): libcephfs2-19.0.0-46.ga2cd52c0.el9.x8 2.6 MB/s | 719 kB 00:00 2023-12-08T08:35:54.709 INFO:teuthology.orchestra.run.smithi016.stdout:(14/147): libcephsqlite-19.0.0-46.ga2cd52c0.el9 1.5 MB/s | 164 kB 00:00 2023-12-08T08:35:54.809 INFO:teuthology.orchestra.run.smithi016.stdout:(15/147): librados-devel-19.0.0-46.ga2cd52c0.el 1.2 MB/s | 126 kB 00:00 2023-12-08T08:35:55.001 INFO:teuthology.orchestra.run.smithi016.stdout:(16/147): libradosstriper1-19.0.0-46.ga2cd52c0. 2.4 MB/s | 476 kB 00:00 2023-12-08T08:35:56.177 INFO:teuthology.orchestra.run.smithi016.stdout:(17/147): ceph-test-19.0.0-46.ga2cd52c0.el9.x86 18 MB/s | 41 MB 00:02 2023-12-08T08:35:56.269 INFO:teuthology.orchestra.run.smithi016.stdout:(18/147): ceph-radosgw-19.0.0-46.ga2cd52c0.el9. 3.2 MB/s | 8.6 MB 00:02 2023-12-08T08:35:56.294 INFO:teuthology.orchestra.run.smithi016.stdout:(19/147): python3-ceph-argparse-19.0.0-46.ga2cd 386 kB/s | 45 kB 00:00 2023-12-08T08:35:56.361 INFO:teuthology.orchestra.run.smithi016.stdout:(20/147): librgw2-19.0.0-46.ga2cd52c0.el9.x86_6 3.5 MB/s | 4.8 MB 00:01 2023-12-08T08:35:56.386 INFO:teuthology.orchestra.run.smithi016.stdout:(21/147): python3-ceph-common-19.0.0-46.ga2cd52 1.1 MB/s | 129 kB 00:00 2023-12-08T08:35:56.411 INFO:teuthology.orchestra.run.smithi016.stdout:(22/147): python3-cephfs-19.0.0-46.ga2cd52c0.el 1.3 MB/s | 161 kB 00:00 2023-12-08T08:35:56.461 INFO:teuthology.orchestra.run.smithi016.stdout:(23/147): python3-rgw-19.0.0-46.ga2cd52c0.el9.x 1.9 MB/s | 99 kB 00:00 2023-12-08T08:35:56.487 INFO:teuthology.orchestra.run.smithi016.stdout:(24/147): python3-rbd-19.0.0-46.ga2cd52c0.el9.x 2.9 MB/s | 298 kB 00:00 2023-12-08T08:35:56.520 INFO:teuthology.orchestra.run.smithi016.stdout:(25/147): python3-rados-19.0.0-46.ga2cd52c0.el9 2.0 MB/s | 320 kB 00:00 2023-12-08T08:35:56.545 INFO:teuthology.orchestra.run.smithi016.stdout:(26/147): rbd-fuse-19.0.0-46.ga2cd52c0.el9.x86_ 1.0 MB/s | 85 kB 00:00 2023-12-08T08:35:56.612 INFO:teuthology.orchestra.run.smithi016.stdout:(27/147): rbd-nbd-19.0.0-46.ga2cd52c0.el9.x86_6 1.8 MB/s | 170 kB 00:00 2023-12-08T08:35:56.629 INFO:teuthology.orchestra.run.smithi016.stdout:(28/147): ceph-grafana-dashboards-19.0.0-46.ga2 336 kB/s | 28 kB 00:00 2023-12-08T08:35:56.680 INFO:teuthology.orchestra.run.smithi016.stdout:(29/147): rbd-mirror-19.0.0-46.ga2cd52c0.el9.x8 16 MB/s | 3.0 MB 00:00 2023-12-08T08:35:56.705 INFO:teuthology.orchestra.run.smithi016.stdout:(30/147): ceph-mgr-cephadm-19.0.0-46.ga2cd52c0. 1.4 MB/s | 133 kB 00:00 2023-12-08T08:35:56.788 INFO:teuthology.orchestra.run.smithi016.stdout:(31/147): ceph-mgr-modules-core-19.0.0-46.ga2cd 2.9 MB/s | 243 kB 00:00 2023-12-08T08:35:56.872 INFO:teuthology.orchestra.run.smithi016.stdout:(32/147): ceph-mgr-rook-19.0.0-46.ga2cd52c0.el9 603 kB/s | 50 kB 00:00 2023-12-08T08:35:56.906 INFO:teuthology.orchestra.run.smithi016.stdout:(33/147): ceph-mgr-dashboard-19.0.0-46.ga2cd52c 6.4 MB/s | 1.8 MB 00:00 2023-12-08T08:35:56.939 INFO:teuthology.orchestra.run.smithi016.stdout:(34/147): ceph-prometheus-alerts-19.0.0-46.ga2c 226 kB/s | 15 kB 00:00 2023-12-08T08:35:56.998 INFO:teuthology.orchestra.run.smithi016.stdout:(35/147): ceph-volume-19.0.0-46.ga2cd52c0.el9.n 2.8 MB/s | 262 kB 00:00 2023-12-08T08:35:57.123 INFO:teuthology.orchestra.run.smithi016.stdout:(36/147): ceph-mgr-diskprediction-local-19.0.0- 17 MB/s | 7.4 MB 00:00 2023-12-08T08:35:57.140 INFO:teuthology.orchestra.run.smithi016.stdout:(37/147): cephfs-top-19.0.0-46.ga2cd52c0.el9.no 129 kB/s | 18 kB 00:00 2023-12-08T08:35:57.182 INFO:teuthology.orchestra.run.smithi016.stdout:(38/147): cephadm-19.0.0-46.ga2cd52c0.el9.noarc 3.2 MB/s | 801 kB 00:00 2023-12-08T08:35:57.332 INFO:teuthology.orchestra.run.smithi016.stdout:(39/147): boost-program-options-1.75.0-8.el9.x8 706 kB/s | 106 kB 00:00 2023-12-08T08:35:57.391 INFO:teuthology.orchestra.run.smithi016.stdout:(40/147): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 662 kB 00:00 2023-12-08T08:35:57.441 INFO:teuthology.orchestra.run.smithi016.stdout:(41/147): bison-3.7.4-5.el9.x86_64.rpm 3.0 MB/s | 920 kB 00:00 2023-12-08T08:35:57.458 INFO:teuthology.orchestra.run.smithi016.stdout:(42/147): elfutils-libelf-devel-0.190-1.el9.x86 173 kB/s | 21 kB 00:00 2023-12-08T08:35:57.483 INFO:teuthology.orchestra.run.smithi016.stdout:(43/147): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 307 kB 00:00 2023-12-08T08:35:57.500 INFO:teuthology.orchestra.run.smithi016.stdout:(44/147): flexiblas-3.0.4-8.el9.x86_64.rpm 766 kB/s | 32 kB 00:00 2023-12-08T08:35:57.550 INFO:teuthology.orchestra.run.smithi016.stdout:(45/147): flexiblas-openblas-openmp-3.0.4-8.el9 256 kB/s | 17 kB 00:00 2023-12-08T08:35:57.651 INFO:teuthology.orchestra.run.smithi016.stdout:(46/147): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2023-12-08T08:35:57.676 INFO:teuthology.orchestra.run.smithi016.stdout:(47/147): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 186 kB 00:00 2023-12-08T08:35:57.701 INFO:teuthology.orchestra.run.smithi016.stdout:(48/147): libaio-devel-0.3.111-13.el9.x86_64.rp 67 kB/s | 10 kB 00:00 2023-12-08T08:35:57.726 INFO:teuthology.orchestra.run.smithi016.stdout:(49/147): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2023-12-08T08:35:57.752 INFO:teuthology.orchestra.run.smithi016.stdout:(50/147): librabbitmq-0.11.0-7.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2023-12-08T08:35:57.777 INFO:teuthology.orchestra.run.smithi016.stdout:(51/147): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2023-12-08T08:35:57.802 INFO:teuthology.orchestra.run.smithi016.stdout:(52/147): libtool-2.4.6-45.el9.x86_64.rpm 7.5 MB/s | 578 kB 00:00 2023-12-08T08:35:57.828 INFO:teuthology.orchestra.run.smithi016.stdout:(53/147): libuuid-devel-2.37.4-15.el9.x86_64.rp 497 kB/s | 24 kB 00:00 2023-12-08T08:35:57.853 INFO:teuthology.orchestra.run.smithi016.stdout:(54/147): libzstd-devel-1.5.1-2.el9.x86_64.rpm 623 kB/s | 46 kB 00:00 2023-12-08T08:35:57.855 INFO:teuthology.orchestra.run.smithi112.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2023-12-08T08:35:57.878 INFO:teuthology.orchestra.run.smithi016.stdout:(55/147): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2023-12-08T08:35:57.903 INFO:teuthology.orchestra.run.smithi016.stdout:(56/147): openblas-0.3.21-2.el9.x86_64.rpm 692 kB/s | 34 kB 00:00 2023-12-08T08:35:57.929 INFO:teuthology.orchestra.run.smithi016.stdout:(57/147): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2023-12-08T08:35:58.022 INFO:teuthology.orchestra.run.smithi016.stdout:(58/147): openblas-openmp-0.3.21-2.el9.x86_64.r 33 MB/s | 4.7 MB 00:00 2023-12-08T08:35:58.062 INFO:teuthology.orchestra.run.smithi016.stdout:(59/147): postgresql-13.11-1.el9.x86_64.rpm 9.7 MB/s | 1.5 MB 00:00 2023-12-08T08:35:58.096 INFO:teuthology.orchestra.run.smithi016.stdout:(60/147): postgresql-contrib-13.11-1.el9.x86_64 4.8 MB/s | 811 kB 00:00 2023-12-08T08:35:58.121 INFO:teuthology.orchestra.run.smithi016.stdout:(61/147): postgresql-private-libs-13.11-1.el9.x 1.4 MB/s | 134 kB 00:00 2023-12-08T08:35:58.338 INFO:teuthology.orchestra.run.smithi016.stdout:(62/147): postgresql-server-13.11-1.el9.x86_64. 24 MB/s | 5.7 MB 00:00 2023-12-08T08:35:58.413 INFO:teuthology.orchestra.run.smithi016.stdout:(63/147): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2023-12-08T08:35:58.438 INFO:teuthology.orchestra.run.smithi016.stdout:(64/147): python3-jinja2-2.11.3-4.el9.noarch.rp 721 kB/s | 228 kB 00:00 2023-12-08T08:35:58.472 INFO:teuthology.orchestra.run.smithi016.stdout:(65/147): python3-jsonpatch-1.21-16.el9.noarch. 197 kB/s | 24 kB 00:00 2023-12-08T08:35:58.489 INFO:teuthology.orchestra.run.smithi016.stdout:(66/147): python3-jsonpointer-2.0-4.el9.noarch. 360 kB/s | 18 kB 00:00 2023-12-08T08:35:58.523 INFO:teuthology.orchestra.run.smithi016.stdout:(67/147): python3-mako-1.1.4-6.el9.noarch.rpm 1.7 MB/s | 144 kB 00:00 2023-12-08T08:35:58.540 INFO:teuthology.orchestra.run.smithi016.stdout:(68/147): python3-markupsafe-1.1.1-12.el9.x86_6 413 kB/s | 32 kB 00:00 2023-12-08T08:35:58.574 INFO:teuthology.orchestra.run.smithi016.stdout:(69/147): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 217 kB 00:00 2023-12-08T08:35:58.700 INFO:teuthology.orchestra.run.smithi016.stdout:(70/147): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 4.8 MB 00:00 2023-12-08T08:35:58.725 INFO:teuthology.orchestra.run.smithi016.stdout:(71/147): python3-oauthlib-3.1.1-5.el9.noarch.r 951 kB/s | 174 kB 00:00 2023-12-08T08:35:58.750 INFO:teuthology.orchestra.run.smithi016.stdout:(72/147): python3-prettytable-0.7.2-27.el9.noar 238 kB/s | 41 kB 00:00 2023-12-08T08:35:58.775 INFO:teuthology.orchestra.run.smithi016.stdout:(73/147): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2023-12-08T08:35:58.792 INFO:teuthology.orchestra.run.smithi016.stdout:(74/147): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2023-12-08T08:35:58.817 INFO:teuthology.orchestra.run.smithi016.stdout:(75/147): python3-pyasn1-modules-0.4.8-6.el9.no 3.1 MB/s | 211 kB 00:00 2023-12-08T08:35:58.834 INFO:teuthology.orchestra.run.smithi016.stdout:(76/147): python3-requests-oauthlib-1.3.0-12.el 730 kB/s | 43 kB 00:00 2023-12-08T08:35:58.976 INFO:teuthology.orchestra.run.smithi016.stdout:(77/147): python3-toml-0.10.2-6.el9.noarch.rpm 278 kB/s | 39 kB 00:00 2023-12-08T08:35:59.001 INFO:teuthology.orchestra.run.smithi016.stdout:(78/147): socat-1.7.4.1-5.el9.x86_64.rpm 1.8 MB/s | 300 kB 00:00 2023-12-08T08:35:59.169 INFO:teuthology.orchestra.run.smithi016.stdout:(79/147): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2023-12-08T08:35:59.185 INFO:teuthology.orchestra.run.smithi016.stdout:(80/147): uuid-1.6.2-55.el9.x86_64.rpm 270 kB/s | 56 kB 00:00 2023-12-08T08:35:59.210 INFO:teuthology.orchestra.run.smithi016.stdout:(81/147): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 412 kB/s | 85 kB 00:00 2023-12-08T08:35:59.227 INFO:teuthology.orchestra.run.smithi016.stdout:(82/147): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2023-12-08T08:35:59.246 INFO:teuthology.orchestra.run.smithi016.stdout:(83/147): zlib-devel-1.2.11-40.el9.x86_64.rpm 756 kB/s | 44 kB 00:00 2023-12-08T08:35:59.269 INFO:teuthology.orchestra.run.smithi016.stdout:(84/147): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2023-12-08T08:35:59.295 INFO:teuthology.orchestra.run.smithi016.stdout:(85/147): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2023-12-08T08:35:59.328 INFO:teuthology.orchestra.run.smithi016.stdout:(86/147): libgfortran-11.4.1-2.3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2023-12-08T08:35:59.353 INFO:teuthology.orchestra.run.smithi016.stdout:(87/147): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 241 kB 00:00 2023-12-08T08:35:59.395 INFO:teuthology.orchestra.run.smithi016.stdout:(88/147): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2023-12-08T08:35:59.421 INFO:teuthology.orchestra.run.smithi016.stdout:(89/147): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2023-12-08T08:35:59.446 INFO:teuthology.orchestra.run.smithi016.stdout:(90/147): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2023-12-08T08:35:59.471 INFO:teuthology.orchestra.run.smithi016.stdout:(91/147): python3-requests-2.25.1-7.el9.noarch. 1.5 MB/s | 113 kB 00:00 2023-12-08T08:35:59.496 INFO:teuthology.orchestra.run.smithi016.stdout:(92/147): python3-urllib3-1.26.5-4.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2023-12-08T08:35:59.780 INFO:teuthology.orchestra.run.smithi016.stdout:(93/147): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.1 MB/s | 309 kB 00:00 2023-12-08T08:36:00.005 INFO:teuthology.orchestra.run.smithi016.stdout:(94/147): libarrow-9.0.0-10.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2023-12-08T08:36:00.139 INFO:teuthology.orchestra.run.smithi016.stdout:(95/147): libarrow-doc-9.0.0-10.el9.noarch.rpm 195 kB/s | 26 kB 00:00 2023-12-08T08:36:00.197 INFO:teuthology.orchestra.run.smithi016.stdout:(96/147): liboath-2.6.7-2.el9.x86_64.rpm 833 kB/s | 49 kB 00:00 2023-12-08T08:36:00.256 INFO:teuthology.orchestra.run.smithi016.stdout:(97/147): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2023-12-08T08:36:00.360 INFO:teuthology.orchestra.run.smithi112.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-08T08:36:00.361 INFO:teuthology.orchestra.run.smithi112.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-08T08:36:00.373 INFO:teuthology.orchestra.run.smithi016.stdout:(98/147): python3-devel-3.9.17-2.el9.x86_64.rpm 272 kB/s | 245 kB 00:00 2023-12-08T08:36:00.374 INFO:teuthology.orchestra.run.smithi112.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-12-08T08:36:00.375 INFO:teuthology.orchestra.run.smithi112.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-12-08T08:36:00.376 INFO:teuthology.orchestra.run.smithi112.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-12-08T08:36:00.377 INFO:teuthology.orchestra.run.smithi112.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-12-08T08:36:00.379 INFO:teuthology.orchestra.run.smithi112.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-12-08T08:36:00.382 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-12-08T08:36:00.383 INFO:teuthology.orchestra.run.smithi112.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-12-08T08:36:00.456 INFO:teuthology.orchestra.run.smithi112.stdout:Dependencies resolved. 2023-12-08T08:36:00.476 INFO:teuthology.orchestra.run.smithi112.stdout:========================================================================================== 2023-12-08T08:36:00.476 INFO:teuthology.orchestra.run.smithi112.stdout: Package Arch Version Repository Size 2023-12-08T08:36:00.476 INFO:teuthology.orchestra.run.smithi112.stdout:========================================================================================== 2023-12-08T08:36:00.477 INFO:teuthology.orchestra.run.smithi112.stdout:Installing: 2023-12-08T08:36:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-12-08T08:36:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: ceph x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 6.5 k 2023-12-08T08:36:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 5.2 M 2023-12-08T08:36:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 860 k 2023-12-08T08:36:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 141 k 2023-12-08T08:36:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 1.5 M 2023-12-08T08:36:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 133 k 2023-12-08T08:36:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 1.8 M 2023-12-08T08:36:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 7.4 M 2023-12-08T08:36:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 50 k 2023-12-08T08:36:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 8.6 M 2023-12-08T08:36:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 41 M 2023-12-08T08:36:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 262 k 2023-12-08T08:36:00.478 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 801 k 2023-12-08T08:36:00.478 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 217 k 2023-12-08T08:36:00.478 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 18 k 2023-12-08T08:36:00.478 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-12-08T08:36:00.478 INFO:teuthology.orchestra.run.smithi112.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-12-08T08:36:00.478 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-12-08T08:36:00.478 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 31 k 2023-12-08T08:36:00.478 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2 x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 719 k 2023-12-08T08:36:00.478 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 126 k 2023-12-08T08:36:00.478 INFO:teuthology.orchestra.run.smithi112.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-12-08T08:36:00.478 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-12-08T08:36:00.478 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-12-08T08:36:00.478 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-12-08T08:36:00.478 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-12-08T08:36:00.479 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-12-08T08:36:00.479 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 161 k 2023-12-08T08:36:00.479 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 320 k 2023-12-08T08:36:00.479 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 298 k 2023-12-08T08:36:00.479 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 99 k 2023-12-08T08:36:00.479 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 85 k 2023-12-08T08:36:00.479 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 3.0 M 2023-12-08T08:36:00.479 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 170 k 2023-12-08T08:36:00.479 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-12-08T08:36:00.479 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrading: 2023-12-08T08:36:00.479 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager x86_64 1:1.45.7-1.el9 baseos 2.3 M 2023-12-08T08:36:00.479 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm x86_64 1:1.45.7-1.el9 baseos 1.8 M 2023-12-08T08:36:00.479 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team x86_64 1:1.45.7-1.el9 baseos 43 k 2023-12-08T08:36:00.479 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui x86_64 1:1.45.7-1.el9 baseos 246 k 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout: librados2 x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 3.3 M 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1 x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 3.0 M 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout:Installing dependencies: 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-12-08T08:36:00.480 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 19 M 2023-12-08T08:36:00.481 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 28 k 2023-12-08T08:36:00.481 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 2.1 M 2023-12-08T08:36:00.481 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 243 k 2023-12-08T08:36:00.481 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 4.4 M 2023-12-08T08:36:00.481 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 18 M 2023-12-08T08:36:00.481 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts noarch 2:19.0.0-46.ga2cd52c0.el9 ceph-noarch 15 k 2023-12-08T08:36:00.481 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 24 k 2023-12-08T08:36:00.481 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-12-08T08:36:00.481 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-12-08T08:36:00.481 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-12-08T08:36:00.481 INFO:teuthology.orchestra.run.smithi112.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-12-08T08:36:00.481 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-12-08T08:36:00.481 INFO:teuthology.orchestra.run.smithi112.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-12-08T08:36:00.481 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2023-12-08T08:36:00.482 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2023-12-08T08:36:00.482 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 164 k 2023-12-08T08:36:00.482 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-12-08T08:36:00.482 INFO:teuthology.orchestra.run.smithi112.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-12-08T08:36:00.482 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-12-08T08:36:00.482 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-12-08T08:36:00.482 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-12-08T08:36:00.482 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1 x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 476 k 2023-12-08T08:36:00.482 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-12-08T08:36:00.482 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2 x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 4.8 M 2023-12-08T08:36:00.482 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-12-08T08:36:00.482 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-12-08T08:36:00.482 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-12-08T08:36:00.482 INFO:teuthology.orchestra.run.smithi112.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-12-08T08:36:00.483 INFO:teuthology.orchestra.run.smithi112.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-12-08T08:36:00.483 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-12-08T08:36:00.483 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2023-12-08T08:36:00.483 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-12-08T08:36:00.483 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-12-08T08:36:00.483 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-12-08T08:36:00.483 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-12-08T08:36:00.483 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-12-08T08:36:00.483 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 45 k 2023-12-08T08:36:00.483 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common x86_64 2:19.0.0-46.ga2cd52c0.el9 ceph 129 k 2023-12-08T08:36:00.483 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-12-08T08:36:00.483 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-12-08T08:36:00.483 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-12-08T08:36:00.484 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-12-08T08:36:00.484 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-12-08T08:36:00.484 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-12-08T08:36:00.484 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-12-08T08:36:00.484 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-12-08T08:36:00.484 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-12-08T08:36:00.484 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-12-08T08:36:00.484 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-12-08T08:36:00.484 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-12-08T08:36:00.484 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-12-08T08:36:00.484 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-12-08T08:36:00.485 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-12-08T08:36:00.485 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-12-08T08:36:00.485 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-12-08T08:36:00.485 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-12-08T08:36:00.485 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-12-08T08:36:00.485 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-12-08T08:36:00.485 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-12-08T08:36:00.485 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-12-08T08:36:00.485 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-12-08T08:36:00.485 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-12-08T08:36:00.485 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-12-08T08:36:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-12-08T08:36:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-12-08T08:36:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-12-08T08:36:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-12-08T08:36:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-12-08T08:36:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-12-08T08:36:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-12-08T08:36:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-12-08T08:36:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-12-08T08:36:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-12-08T08:36:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-12-08T08:36:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-12-08T08:36:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-12-08T08:36:00.486 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-12-08T08:36:00.487 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-12-08T08:36:00.487 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-12-08T08:36:00.487 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-12-08T08:36:00.487 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-12-08T08:36:00.487 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-12-08T08:36:00.487 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-12-08T08:36:00.487 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-12-08T08:36:00.487 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-12-08T08:36:00.487 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-12-08T08:36:00.488 INFO:teuthology.orchestra.run.smithi112.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-12-08T08:36:00.488 INFO:teuthology.orchestra.run.smithi112.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-12-08T08:36:00.488 INFO:teuthology.orchestra.run.smithi112.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-12-08T08:36:00.488 INFO:teuthology.orchestra.run.smithi112.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-12-08T08:36:00.488 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-12-08T08:36:00.488 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-12-08T08:36:00.488 INFO:teuthology.orchestra.run.smithi112.stdout:Installing weak dependencies: 2023-12-08T08:36:00.488 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-12-08T08:36:00.488 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:36:00.488 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction Summary 2023-12-08T08:36:00.488 INFO:teuthology.orchestra.run.smithi112.stdout:========================================================================================== 2023-12-08T08:36:00.489 INFO:teuthology.orchestra.run.smithi112.stdout:Install 132 Packages 2023-12-08T08:36:00.489 INFO:teuthology.orchestra.run.smithi112.stdout:Upgrade 15 Packages 2023-12-08T08:36:00.489 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:36:00.489 INFO:teuthology.orchestra.run.smithi112.stdout:Total download size: 204 M 2023-12-08T08:36:00.489 INFO:teuthology.orchestra.run.smithi112.stdout:Downloading Packages: 2023-12-08T08:36:00.648 INFO:teuthology.orchestra.run.smithi016.stdout:(99/147): python3-asyncssh-2.13.2-3.el9.noarch. 1.9 MB/s | 548 kB 00:00 2023-12-08T08:36:00.707 INFO:teuthology.orchestra.run.smithi016.stdout:(100/147): openssl-devel-3.0.7-25.el9.x86_64.rp 3.3 MB/s | 4.1 MB 00:01 2023-12-08T08:36:00.732 INFO:teuthology.orchestra.run.smithi016.stdout:(101/147): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2023-12-08T08:36:00.832 INFO:teuthology.orchestra.run.smithi016.stdout:(102/147): python3-cachetools-4.2.4-1.el9.noarc 258 kB/s | 32 kB 00:00 2023-12-08T08:36:00.958 INFO:teuthology.orchestra.run.smithi016.stdout:(103/147): python3-certifi-2021.10.8-2.el9.noar 66 kB/s | 15 kB 00:00 2023-12-08T08:36:00.983 INFO:teuthology.orchestra.run.smithi016.stdout:(104/147): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2023-12-08T08:36:01.015 INFO:teuthology.orchestra.run.smithi112.stdout:(1/147): ceph-19.0.0-46.ga2cd52c0.el9.x86_64.rp 19 kB/s | 6.5 kB 00:00 2023-12-08T08:36:01.025 INFO:teuthology.orchestra.run.smithi016.stdout:(105/147): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2023-12-08T08:36:01.084 INFO:teuthology.orchestra.run.smithi016.stdout:(106/147): python3-google-auth-2.23.4-1.el9.noa 1.9 MB/s | 197 kB 00:00 2023-12-08T08:36:01.101 INFO:teuthology.orchestra.run.smithi016.stdout:(107/147): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2023-12-08T08:36:01.134 INFO:teuthology.orchestra.run.smithi016.stdout:(108/147): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2023-12-08T08:36:01.168 INFO:teuthology.orchestra.run.smithi016.stdout:(109/147): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2023-12-08T08:36:01.193 INFO:teuthology.orchestra.run.smithi016.stdout:(110/147): python3-jaraco-functools-3.5.0-2.el9 333 kB/s | 19 kB 00:00 2023-12-08T08:36:01.218 INFO:teuthology.orchestra.run.smithi016.stdout:(111/147): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2023-12-08T08:36:01.276 INFO:teuthology.orchestra.run.smithi016.stdout:(112/147): python3-jwt+crypto-2.4.0-1.el9.noarc 109 kB/s | 9.0 kB 00:00 2023-12-08T08:36:01.294 INFO:teuthology.orchestra.run.smithi016.stdout:(113/147): python3-jwt-2.4.0-1.el9.noarch.rpm 544 kB/s | 41 kB 00:00 2023-12-08T08:36:01.328 INFO:teuthology.orchestra.run.smithi016.stdout:(114/147): parquet-libs-9.0.0-10.el9.x86_64.rpm 782 kB/s | 838 kB 00:01 2023-12-08T08:36:01.361 INFO:teuthology.orchestra.run.smithi016.stdout:(115/147): python3-logutils-0.3.5-21.el9.noarch 691 kB/s | 46 kB 00:00 2023-12-08T08:36:01.386 INFO:teuthology.orchestra.run.smithi016.stdout:(116/147): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2023-12-08T08:36:01.420 INFO:teuthology.orchestra.run.smithi016.stdout:(117/147): python3-kubernetes-26.1.0-2.el9.noar 7.1 MB/s | 1.0 MB 00:00 2023-12-08T08:36:01.437 INFO:teuthology.orchestra.run.smithi016.stdout:(118/147): python3-natsort-7.1.1-5.el9.noarch.r 765 kB/s | 58 kB 00:00 2023-12-08T08:36:01.479 INFO:teuthology.orchestra.run.smithi016.stdout:(119/147): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2023-12-08T08:36:01.496 INFO:teuthology.orchestra.run.smithi016.stdout:(120/147): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2023-12-08T08:36:01.521 INFO:teuthology.orchestra.run.smithi016.stdout:(121/147): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2023-12-08T08:36:01.538 INFO:teuthology.orchestra.run.smithi016.stdout:(122/147): python3-repoze-lru-0.7-16.el9.noarch 523 kB/s | 31 kB 00:00 2023-12-08T08:36:01.563 INFO:teuthology.orchestra.run.smithi016.stdout:(123/147): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2023-12-08T08:36:01.588 INFO:teuthology.orchestra.run.smithi016.stdout:(124/147): python3-rsa-4.9-2.el9.noarch.rpm 880 kB/s | 59 kB 00:00 2023-12-08T08:36:01.605 INFO:teuthology.orchestra.run.smithi016.stdout:(125/147): python3-tempora-5.0.0-2.el9.noarch.r 533 kB/s | 36 kB 00:00 2023-12-08T08:36:01.622 INFO:teuthology.orchestra.run.smithi016.stdout:(126/147): python3-typing-extensions-4.4.0-2.el 867 kB/s | 51 kB 00:00 2023-12-08T08:36:01.657 INFO:teuthology.orchestra.run.smithi112.stdout:(2/147): ceph-fuse-19.0.0-46.ga2cd52c0.el9.x86_ 1.3 MB/s | 860 kB 00:00 2023-12-08T08:36:01.672 INFO:teuthology.orchestra.run.smithi016.stdout:(127/147): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2023-12-08T08:36:01.698 INFO:teuthology.orchestra.run.smithi016.stdout:(128/147): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2023-12-08T08:36:01.723 INFO:teuthology.orchestra.run.smithi016.stdout:(129/147): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2023-12-08T08:36:01.740 INFO:teuthology.orchestra.run.smithi016.stdout:(130/147): python3-zc-lockfile-2.0-10.el9.noarc 297 kB/s | 20 kB 00:00 2023-12-08T08:36:01.774 INFO:teuthology.orchestra.run.smithi112.stdout:(3/147): ceph-immutable-object-cache-19.0.0-46. 1.2 MB/s | 141 kB 00:00 2023-12-08T08:36:01.857 INFO:teuthology.orchestra.run.smithi016.stdout:(131/147): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2023-12-08T08:36:01.916 INFO:teuthology.orchestra.run.smithi016.stdout:(132/147): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2023-12-08T08:36:01.949 INFO:teuthology.orchestra.run.smithi016.stdout:(133/147): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-12-08T08:36:01.975 INFO:teuthology.orchestra.run.smithi016.stdout:(134/147): acl-2.3.1-4.el9.x86_64.rpm 2.7 MB/s | 69 kB 00:00 2023-12-08T08:36:02.008 INFO:teuthology.orchestra.run.smithi016.stdout:(135/147): elfutils-debuginfod-client-0.190-1.e 1.1 MB/s | 36 kB 00:00 2023-12-08T08:36:02.042 INFO:teuthology.orchestra.run.smithi016.stdout:(136/147): elfutils-libelf-0.190-1.el9.x86_64.r 5.6 MB/s | 192 kB 00:00 2023-12-08T08:36:02.084 INFO:teuthology.orchestra.run.smithi016.stdout:(137/147): elfutils-libs-0.190-1.el9.x86_64.rpm 6.0 MB/s | 255 kB 00:00 2023-12-08T08:36:02.118 INFO:teuthology.orchestra.run.smithi016.stdout:(138/147): libacl-2.3.1-4.el9.x86_64.rpm 659 kB/s | 22 kB 00:00 2023-12-08T08:36:02.160 INFO:teuthology.orchestra.run.smithi016.stdout:(139/147): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2023-12-08T08:36:02.424 INFO:teuthology.orchestra.run.smithi112.stdout:(4/147): ceph-base-19.0.0-46.ga2cd52c0.el9.x86_ 3.0 MB/s | 5.2 MB 00:01 2023-12-08T08:36:02.493 INFO:teuthology.orchestra.run.smithi016.stdout:(140/147): NetworkManager-1.45.7-1.el9.x86_64.r 6.8 MB/s | 2.3 MB 00:00 2023-12-08T08:36:02.702 INFO:teuthology.orchestra.run.smithi016.stdout:(141/147): NetworkManager-libnm-1.45.7-1.el9.x8 8.5 MB/s | 1.8 MB 00:00 2023-12-08T08:36:02.760 INFO:teuthology.orchestra.run.smithi016.stdout:(142/147): NetworkManager-team-1.45.7-1.el9.x86 741 kB/s | 43 kB 00:00 2023-12-08T08:36:02.819 INFO:teuthology.orchestra.run.smithi016.stdout:(143/147): NetworkManager-tui-1.45.7-1.el9.x86_ 4.1 MB/s | 246 kB 00:00 2023-12-08T08:36:02.953 INFO:teuthology.orchestra.run.smithi016.stdout:(144/147): librados2-19.0.0-46.ga2cd52c0.el9.x8 2.7 MB/s | 3.3 MB 00:01 2023-12-08T08:36:03.019 INFO:teuthology.orchestra.run.smithi016.stdout:(145/147): openssl-3.0.7-25.el9.x86_64.rpm 6.1 MB/s | 1.2 MB 00:00 2023-12-08T08:36:03.278 INFO:teuthology.orchestra.run.smithi016.stdout:(146/147): openssl-libs-3.0.7-25.el9.x86_64.rpm 6.6 MB/s | 2.2 MB 00:00 2023-12-08T08:36:03.366 INFO:teuthology.orchestra.run.smithi112.stdout:(5/147): ceph-common-19.0.0-46.ga2cd52c0.el9.x8 7.0 MB/s | 19 MB 00:02 2023-12-08T08:36:03.399 INFO:teuthology.orchestra.run.smithi112.stdout:(6/147): ceph-mgr-19.0.0-46.ga2cd52c0.el9.x86_6 1.5 MB/s | 1.5 MB 00:00 2023-12-08T08:36:03.508 INFO:teuthology.orchestra.run.smithi112.stdout:(7/147): ceph-mds-19.0.0-46.ga2cd52c0.el9.x86_6 1.2 MB/s | 2.1 MB 00:01 2023-12-08T08:36:03.791 INFO:teuthology.orchestra.run.smithi112.stdout:(8/147): ceph-mon-19.0.0-46.ga2cd52c0.el9.x86_6 10 MB/s | 4.4 MB 00:00 2023-12-08T08:36:03.867 INFO:teuthology.orchestra.run.smithi112.stdout:(9/147): ceph-selinux-19.0.0-46.ga2cd52c0.el9.x 322 kB/s | 24 kB 00:00 2023-12-08T08:36:04.427 INFO:teuthology.orchestra.run.smithi112.stdout:(10/147): ceph-radosgw-19.0.0-46.ga2cd52c0.el9. 9.4 MB/s | 8.6 MB 00:00 2023-12-08T08:36:04.484 INFO:teuthology.orchestra.run.smithi112.stdout:(11/147): cephfs-mirror-19.0.0-46.ga2cd52c0.el9 3.7 MB/s | 217 kB 00:00 2023-12-08T08:36:04.551 INFO:teuthology.orchestra.run.smithi112.stdout:(12/147): libcephfs-devel-19.0.0-46.ga2cd52c0.e 468 kB/s | 31 kB 00:00 2023-12-08T08:36:04.685 INFO:teuthology.orchestra.run.smithi112.stdout:(13/147): libcephfs2-19.0.0-46.ga2cd52c0.el9.x8 5.3 MB/s | 719 kB 00:00 2023-12-08T08:36:04.760 INFO:teuthology.orchestra.run.smithi112.stdout:(14/147): libcephsqlite-19.0.0-46.ga2cd52c0.el9 2.1 MB/s | 164 kB 00:00 2023-12-08T08:36:04.835 INFO:teuthology.orchestra.run.smithi112.stdout:(15/147): librados-devel-19.0.0-46.ga2cd52c0.el 1.6 MB/s | 126 kB 00:00 2023-12-08T08:36:04.919 INFO:teuthology.orchestra.run.smithi112.stdout:(16/147): libradosstriper1-19.0.0-46.ga2cd52c0. 5.6 MB/s | 476 kB 00:00 2023-12-08T08:36:05.569 INFO:teuthology.orchestra.run.smithi112.stdout:(17/147): librgw2-19.0.0-46.ga2cd52c0.el9.x86_6 7.3 MB/s | 4.8 MB 00:00 2023-12-08T08:36:05.636 INFO:teuthology.orchestra.run.smithi112.stdout:(18/147): python3-ceph-argparse-19.0.0-46.ga2cd 680 kB/s | 45 kB 00:00 2023-12-08T08:36:05.711 INFO:teuthology.orchestra.run.smithi112.stdout:(19/147): python3-ceph-common-19.0.0-46.ga2cd52 1.7 MB/s | 129 kB 00:00 2023-12-08T08:36:05.803 INFO:teuthology.orchestra.run.smithi112.stdout:(20/147): python3-cephfs-19.0.0-46.ga2cd52c0.el 1.7 MB/s | 161 kB 00:00 2023-12-08T08:36:05.887 INFO:teuthology.orchestra.run.smithi112.stdout:(21/147): python3-rados-19.0.0-46.ga2cd52c0.el9 3.8 MB/s | 320 kB 00:00 2023-12-08T08:36:06.054 INFO:teuthology.orchestra.run.smithi112.stdout:(22/147): ceph-osd-19.0.0-46.ga2cd52c0.el9.x86_ 6.7 MB/s | 18 MB 00:02 2023-12-08T08:36:06.079 INFO:teuthology.orchestra.run.smithi112.stdout:(23/147): python3-rbd-19.0.0-46.ga2cd52c0.el9.x 1.5 MB/s | 298 kB 00:00 2023-12-08T08:36:06.129 INFO:teuthology.orchestra.run.smithi112.stdout:(24/147): python3-rgw-19.0.0-46.ga2cd52c0.el9.x 1.3 MB/s | 99 kB 00:00 2023-12-08T08:36:06.155 INFO:teuthology.orchestra.run.smithi112.stdout:(25/147): rbd-fuse-19.0.0-46.ga2cd52c0.el9.x86_ 1.1 MB/s | 85 kB 00:00 2023-12-08T08:36:06.230 INFO:teuthology.orchestra.run.smithi112.stdout:(26/147): rbd-nbd-19.0.0-46.ga2cd52c0.el9.x86_6 2.2 MB/s | 170 kB 00:00 2023-12-08T08:36:06.297 INFO:teuthology.orchestra.run.smithi112.stdout:(27/147): ceph-grafana-dashboards-19.0.0-46.ga2 422 kB/s | 28 kB 00:00 2023-12-08T08:36:06.362 INFO:teuthology.orchestra.run.smithi112.stdout:(28/147): ceph-mgr-cephadm-19.0.0-46.ga2cd52c0. 2.0 MB/s | 133 kB 00:00 2023-12-08T08:36:06.513 INFO:teuthology.orchestra.run.smithi112.stdout:(29/147): rbd-mirror-19.0.0-46.ga2cd52c0.el9.x8 7.9 MB/s | 3.0 MB 00:00 2023-12-08T08:36:06.571 INFO:teuthology.orchestra.run.smithi112.stdout:(30/147): ceph-mgr-dashboard-19.0.0-46.ga2cd52c 8.5 MB/s | 1.8 MB 00:00 2023-12-08T08:36:06.630 INFO:teuthology.orchestra.run.smithi112.stdout:(31/147): ceph-mgr-modules-core-19.0.0-46.ga2cd 4.1 MB/s | 243 kB 00:00 2023-12-08T08:36:06.689 INFO:teuthology.orchestra.run.smithi112.stdout:(32/147): ceph-mgr-rook-19.0.0-46.ga2cd52c0.el9 857 kB/s | 50 kB 00:00 2023-12-08T08:36:06.747 INFO:teuthology.orchestra.run.smithi112.stdout:(33/147): ceph-prometheus-alerts-19.0.0-46.ga2c 259 kB/s | 15 kB 00:00 2023-12-08T08:36:07.140 INFO:teuthology.orchestra.run.smithi112.stdout:(34/147): ceph-test-19.0.0-46.ga2cd52c0.el9.x86 13 MB/s | 41 MB 00:03 2023-12-08T08:36:07.165 INFO:teuthology.orchestra.run.smithi112.stdout:(35/147): ceph-volume-19.0.0-46.ga2cd52c0.el9.n 629 kB/s | 262 kB 00:00 2023-12-08T08:36:07.206 INFO:teuthology.orchestra.run.smithi112.stdout:(36/147): cephfs-top-19.0.0-46.ga2cd52c0.el9.no 439 kB/s | 18 kB 00:00 2023-12-08T08:36:07.219 INFO:teuthology.orchestra.run.smithi016.stdout:(147/147): librbd1-19.0.0-46.ga2cd52c0.el9.x86_ 577 kB/s | 3.0 MB 00:05 2023-12-08T08:36:07.225 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-12-08T08:36:07.225 INFO:teuthology.orchestra.run.smithi016.stdout:Total 12 MB/s | 204 MB 00:16 2023-12-08T08:36:07.315 INFO:teuthology.orchestra.run.smithi112.stdout:(37/147): ceph-mgr-diskprediction-local-19.0.0- 9.2 MB/s | 7.4 MB 00:00 2023-12-08T08:36:07.382 INFO:teuthology.orchestra.run.smithi112.stdout:(38/147): cephadm-19.0.0-46.ga2cd52c0.el9.noarc 3.2 MB/s | 801 kB 00:00 2023-12-08T08:36:07.396 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-12-08T08:36:07.532 INFO:teuthology.orchestra.run.smithi112.stdout:(39/147): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 662 kB 00:00 2023-12-08T08:36:07.566 INFO:teuthology.orchestra.run.smithi112.stdout:(40/147): bison-3.7.4-5.el9.x86_64.rpm 4.0 MB/s | 920 kB 00:00 2023-12-08T08:36:07.591 INFO:teuthology.orchestra.run.smithi112.stdout:(41/147): boost-program-options-1.75.0-8.el9.x8 580 kB/s | 106 kB 00:00 2023-12-08T08:36:07.601 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-12-08T08:36:07.601 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-12-08T08:36:07.616 INFO:teuthology.orchestra.run.smithi112.stdout:(42/147): elfutils-libelf-devel-0.190-1.el9.x86 260 kB/s | 21 kB 00:00 2023-12-08T08:36:07.641 INFO:teuthology.orchestra.run.smithi112.stdout:(43/147): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2023-12-08T08:36:07.667 INFO:teuthology.orchestra.run.smithi112.stdout:(44/147): flexiblas-3.0.4-8.el9.x86_64.rpm 420 kB/s | 32 kB 00:00 2023-12-08T08:36:07.759 INFO:teuthology.orchestra.run.smithi112.stdout:(45/147): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2023-12-08T08:36:07.801 INFO:teuthology.orchestra.run.smithi112.stdout:(46/147): flexiblas-openblas-openmp-3.0.4-8.el9 130 kB/s | 17 kB 00:00 2023-12-08T08:36:07.826 INFO:teuthology.orchestra.run.smithi112.stdout:(47/147): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 186 kB 00:00 2023-12-08T08:36:07.851 INFO:teuthology.orchestra.run.smithi112.stdout:(48/147): libaio-devel-0.3.111-13.el9.x86_64.rp 109 kB/s | 10 kB 00:00 2023-12-08T08:36:07.877 INFO:teuthology.orchestra.run.smithi112.stdout:(49/147): librabbitmq-0.11.0-7.el9.x86_64.rpm 883 kB/s | 44 kB 00:00 2023-12-08T08:36:07.902 INFO:teuthology.orchestra.run.smithi112.stdout:(50/147): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 159 kB 00:00 2023-12-08T08:36:07.927 INFO:teuthology.orchestra.run.smithi112.stdout:(51/147): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2023-12-08T08:36:07.952 INFO:teuthology.orchestra.run.smithi112.stdout:(52/147): libtool-2.4.6-45.el9.x86_64.rpm 7.7 MB/s | 578 kB 00:00 2023-12-08T08:36:07.978 INFO:teuthology.orchestra.run.smithi112.stdout:(53/147): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 46 kB 00:00 2023-12-08T08:36:07.994 INFO:teuthology.orchestra.run.smithi112.stdout:(54/147): libuuid-devel-2.37.4-15.el9.x86_64.rp 370 kB/s | 24 kB 00:00 2023-12-08T08:36:08.019 INFO:teuthology.orchestra.run.smithi112.stdout:(55/147): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.1 MB/s | 282 kB 00:00 2023-12-08T08:36:08.045 INFO:teuthology.orchestra.run.smithi112.stdout:(56/147): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2023-12-08T08:36:08.062 INFO:teuthology.orchestra.run.smithi112.stdout:(57/147): openblas-0.3.21-2.el9.x86_64.rpm 827 kB/s | 34 kB 00:00 2023-12-08T08:36:08.154 INFO:teuthology.orchestra.run.smithi112.stdout:(58/147): openblas-openmp-0.3.21-2.el9.x86_64.r 36 MB/s | 4.7 MB 00:00 2023-12-08T08:36:08.196 INFO:teuthology.orchestra.run.smithi112.stdout:(59/147): postgresql-13.11-1.el9.x86_64.rpm 10 MB/s | 1.5 MB 00:00 2023-12-08T08:36:08.238 INFO:teuthology.orchestra.run.smithi112.stdout:(60/147): postgresql-contrib-13.11-1.el9.x86_64 4.6 MB/s | 811 kB 00:00 2023-12-08T08:36:08.263 INFO:teuthology.orchestra.run.smithi112.stdout:(61/147): postgresql-private-libs-13.11-1.el9.x 1.2 MB/s | 134 kB 00:00 2023-12-08T08:36:08.380 INFO:teuthology.orchestra.run.smithi112.stdout:(62/147): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2023-12-08T08:36:08.405 INFO:teuthology.orchestra.run.smithi112.stdout:(63/147): python3-jinja2-2.11.3-4.el9.noarch.rp 1.6 MB/s | 228 kB 00:00 2023-12-08T08:36:08.522 INFO:teuthology.orchestra.run.smithi112.stdout:(64/147): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2023-12-08T08:36:08.547 INFO:teuthology.orchestra.run.smithi112.stdout:(65/147): python3-jsonpatch-1.21-16.el9.noarch. 146 kB/s | 24 kB 00:00 2023-12-08T08:36:08.572 INFO:teuthology.orchestra.run.smithi112.stdout:(66/147): python3-jsonpointer-2.0-4.el9.noarch. 110 kB/s | 18 kB 00:00 2023-12-08T08:36:08.614 INFO:teuthology.orchestra.run.smithi112.stdout:(67/147): python3-markupsafe-1.1.1-12.el9.x86_6 762 kB/s | 32 kB 00:00 2023-12-08T08:36:08.639 INFO:teuthology.orchestra.run.smithi112.stdout:(68/147): python3-mako-1.1.4-6.el9.noarch.rpm 2.1 MB/s | 144 kB 00:00 2023-12-08T08:36:08.731 INFO:teuthology.orchestra.run.smithi112.stdout:(69/147): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2023-12-08T08:36:08.775 INFO:teuthology.orchestra.run.smithi112.stdout:(70/147): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2023-12-08T08:36:08.782 INFO:teuthology.orchestra.run.smithi112.stdout:(71/147): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 174 kB 00:00 2023-12-08T08:36:08.807 INFO:teuthology.orchestra.run.smithi112.stdout:(72/147): python3-prettytable-0.7.2-27.el9.noar 551 kB/s | 41 kB 00:00 2023-12-08T08:36:08.832 INFO:teuthology.orchestra.run.smithi112.stdout:(73/147): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2023-12-08T08:36:08.858 INFO:teuthology.orchestra.run.smithi112.stdout:(74/147): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 211 kB 00:00 2023-12-08T08:36:08.883 INFO:teuthology.orchestra.run.smithi112.stdout:(75/147): python3-pytz-2021.1-5.el9.noarch.rpm 628 kB/s | 47 kB 00:00 2023-12-08T08:36:08.908 INFO:teuthology.orchestra.run.smithi112.stdout:(76/147): python3-requests-oauthlib-1.3.0-12.el 859 kB/s | 43 kB 00:00 2023-12-08T08:36:09.067 INFO:teuthology.orchestra.run.smithi112.stdout:(77/147): python3-toml-0.10.2-6.el9.noarch.rpm 214 kB/s | 39 kB 00:00 2023-12-08T08:36:09.225 INFO:teuthology.orchestra.run.smithi112.stdout:(78/147): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2023-12-08T08:36:09.259 INFO:teuthology.orchestra.run.smithi112.stdout:(79/147): socat-1.7.4.1-5.el9.x86_64.rpm 863 kB/s | 300 kB 00:00 2023-12-08T08:36:09.284 INFO:teuthology.orchestra.run.smithi112.stdout:(80/147): uuid-1.6.2-55.el9.x86_64.rpm 262 kB/s | 56 kB 00:00 2023-12-08T08:36:09.309 INFO:teuthology.orchestra.run.smithi112.stdout:(81/147): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.7 MB/s | 85 kB 00:00 2023-12-08T08:36:09.335 INFO:teuthology.orchestra.run.smithi112.stdout:(82/147): xmlstarlet-1.6.1-20.el9.x86_64.rpm 832 kB/s | 63 kB 00:00 2023-12-08T08:36:09.360 INFO:teuthology.orchestra.run.smithi112.stdout:(83/147): zlib-devel-1.2.11-40.el9.x86_64.rpm 585 kB/s | 44 kB 00:00 2023-12-08T08:36:09.377 INFO:teuthology.orchestra.run.smithi112.stdout:(84/147): fuse-2.9.9-15.el9.x86_64.rpm 1.8 MB/s | 79 kB 00:00 2023-12-08T08:36:09.410 INFO:teuthology.orchestra.run.smithi112.stdout:(85/147): libgfortran-11.4.1-2.3.el9.x86_64.rpm 10 MB/s | 803 kB 00:00 2023-12-08T08:36:09.435 INFO:teuthology.orchestra.run.smithi112.stdout:(86/147): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2023-12-08T08:36:09.461 INFO:teuthology.orchestra.run.smithi112.stdout:(87/147): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.8 MB/s | 241 kB 00:00 2023-12-08T08:36:09.488 INFO:teuthology.orchestra.run.smithi112.stdout:(88/147): python3-ply-3.11-14.el9.noarch.rpm 1.9 MB/s | 103 kB 00:00 2023-12-08T08:36:09.516 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-12-08T08:36:09.517 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-12-08T08:36:09.522 INFO:teuthology.orchestra.run.smithi112.stdout:(89/147): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.1 MB 00:00 2023-12-08T08:36:09.547 INFO:teuthology.orchestra.run.smithi112.stdout:(90/147): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2023-12-08T08:36:09.572 INFO:teuthology.orchestra.run.smithi112.stdout:(91/147): python3-requests-2.25.1-7.el9.noarch. 1.3 MB/s | 113 kB 00:00 2023-12-08T08:36:09.597 INFO:teuthology.orchestra.run.smithi112.stdout:(92/147): python3-urllib3-1.26.5-4.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2023-12-08T08:36:10.139 INFO:teuthology.orchestra.run.smithi112.stdout:(93/147): python3-devel-3.9.17-2.el9.x86_64.rpm 433 kB/s | 245 kB 00:00 2023-12-08T08:36:10.248 INFO:teuthology.orchestra.run.smithi112.stdout:(94/147): gperftools-libs-2.9.1-2.el9.x86_64.rp 475 kB/s | 309 kB 00:00 2023-12-08T08:36:10.381 INFO:teuthology.orchestra.run.smithi112.stdout:(95/147): libarrow-doc-9.0.0-10.el9.noarch.rpm 243 kB/s | 26 kB 00:00 2023-12-08T08:36:10.490 INFO:teuthology.orchestra.run.smithi112.stdout:(96/147): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2023-12-08T08:36:10.549 INFO:teuthology.orchestra.run.smithi112.stdout:(97/147): openssl-devel-3.0.7-25.el9.x86_64.rpm 4.1 MB/s | 4.1 MB 00:01 2023-12-08T08:36:10.616 INFO:teuthology.orchestra.run.smithi112.stdout:(98/147): libunwind-1.6.2-1.el9.x86_64.rpm 538 kB/s | 67 kB 00:00 2023-12-08T08:36:10.832 INFO:teuthology.orchestra.run.smithi112.stdout:(99/147): python3-asyncssh-2.13.2-3.el9.noarch. 2.5 MB/s | 548 kB 00:00 2023-12-08T08:36:10.958 INFO:teuthology.orchestra.run.smithi112.stdout:(100/147): python3-bcrypt-3.2.2-1.el9.x86_64.rp 348 kB/s | 43 kB 00:00 2023-12-08T08:36:11.066 INFO:teuthology.orchestra.run.smithi112.stdout:(101/147): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2023-12-08T08:36:11.167 INFO:teuthology.orchestra.run.smithi112.stdout:(102/147): python3-certifi-2021.10.8-2.el9.noar 150 kB/s | 15 kB 00:00 2023-12-08T08:36:11.259 INFO:teuthology.orchestra.run.smithi112.stdout:(103/147): libarrow-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 4.4 MB 00:01 2023-12-08T08:36:11.284 INFO:teuthology.orchestra.run.smithi112.stdout:(104/147): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2023-12-08T08:36:11.376 INFO:teuthology.orchestra.run.smithi112.stdout:(105/147): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2023-12-08T08:36:11.401 INFO:teuthology.orchestra.run.smithi112.stdout:(106/147): python3-google-auth-2.23.4-1.el9.noa 1.6 MB/s | 197 kB 00:00 2023-12-08T08:36:11.434 INFO:teuthology.orchestra.run.smithi112.stdout:(107/147): parquet-libs-9.0.0-10.el9.x86_64.rpm 946 kB/s | 838 kB 00:00 2023-12-08T08:36:11.485 INFO:teuthology.orchestra.run.smithi112.stdout:(108/147): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2023-12-08T08:36:11.510 INFO:teuthology.orchestra.run.smithi112.stdout:(109/147): python3-jaraco-classes-3.2.1-5.el9.n 162 kB/s | 18 kB 00:00 2023-12-08T08:36:11.544 INFO:teuthology.orchestra.run.smithi112.stdout:(110/147): python3-jaraco-collections-3.0.0-8.e 212 kB/s | 23 kB 00:00 2023-12-08T08:36:11.586 INFO:teuthology.orchestra.run.smithi112.stdout:(111/147): python3-jaraco-functools-3.5.0-2.el9 193 kB/s | 19 kB 00:00 2023-12-08T08:36:11.611 INFO:teuthology.orchestra.run.smithi112.stdout:(112/147): python3-jaraco-text-3.2.0-6.el9.noar 195 kB/s | 20 kB 00:00 2023-12-08T08:36:11.645 INFO:teuthology.orchestra.run.smithi112.stdout:(113/147): python3-jwt+crypto-2.4.0-1.el9.noarc 90 kB/s | 9.0 kB 00:00 2023-12-08T08:36:11.687 INFO:teuthology.orchestra.run.smithi112.stdout:(114/147): python3-jwt-2.4.0-1.el9.noarch.rpm 407 kB/s | 41 kB 00:00 2023-12-08T08:36:11.754 INFO:teuthology.orchestra.run.smithi112.stdout:(115/147): python3-logutils-0.3.5-21.el9.noarch 427 kB/s | 46 kB 00:00 2023-12-08T08:36:11.796 INFO:teuthology.orchestra.run.smithi112.stdout:(116/147): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2023-12-08T08:36:11.821 INFO:teuthology.orchestra.run.smithi112.stdout:(117/147): python3-more-itertools-8.12.0-2.el9. 589 kB/s | 79 kB 00:00 2023-12-08T08:36:11.863 INFO:teuthology.orchestra.run.smithi112.stdout:(118/147): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2023-12-08T08:36:11.922 INFO:teuthology.orchestra.run.smithi112.stdout:(119/147): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2023-12-08T08:36:11.947 INFO:teuthology.orchestra.run.smithi112.stdout:(120/147): python3-portend-3.1.0-2.el9.noarch.r 131 kB/s | 16 kB 00:00 2023-12-08T08:36:12.005 INFO:teuthology.orchestra.run.smithi112.stdout:(121/147): python3-pyOpenSSL-21.0.0-1.el9.noarc 632 kB/s | 90 kB 00:00 2023-12-08T08:36:12.047 INFO:teuthology.orchestra.run.smithi112.stdout:(122/147): python3-repoze-lru-0.7-16.el9.noarch 245 kB/s | 31 kB 00:00 2023-12-08T08:36:12.073 INFO:teuthology.orchestra.run.smithi112.stdout:(123/147): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2023-12-08T08:36:12.115 INFO:teuthology.orchestra.run.smithi112.stdout:(124/147): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2023-12-08T08:36:12.148 INFO:teuthology.orchestra.run.smithi112.stdout:(125/147): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2023-12-08T08:36:12.173 INFO:teuthology.orchestra.run.smithi112.stdout:(126/147): python3-typing-extensions-4.4.0-2.el 508 kB/s | 51 kB 00:00 2023-12-08T08:36:12.232 INFO:teuthology.orchestra.run.smithi112.stdout:(127/147): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2023-12-08T08:36:12.257 INFO:teuthology.orchestra.run.smithi112.stdout:(128/147): python3-websocket-client-1.2.3-2.el9 824 kB/s | 90 kB 00:00 2023-12-08T08:36:12.282 INFO:teuthology.orchestra.run.smithi112.stdout:(129/147): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2023-12-08T08:36:12.333 INFO:teuthology.orchestra.run.smithi112.stdout:(130/147): python3-zc-lockfile-2.0-10.el9.noarc 198 kB/s | 20 kB 00:00 2023-12-08T08:36:12.366 INFO:teuthology.orchestra.run.smithi112.stdout:(131/147): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2023-12-08T08:36:12.425 INFO:teuthology.orchestra.run.smithi112.stdout:(132/147): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2023-12-08T08:36:12.459 INFO:teuthology.orchestra.run.smithi112.stdout:(133/147): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-12-08T08:36:12.492 INFO:teuthology.orchestra.run.smithi112.stdout:(134/147): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2023-12-08T08:36:12.526 INFO:teuthology.orchestra.run.smithi112.stdout:(135/147): elfutils-debuginfod-client-0.190-1.e 1.1 MB/s | 36 kB 00:00 2023-12-08T08:36:12.559 INFO:teuthology.orchestra.run.smithi112.stdout:(136/147): elfutils-libelf-0.190-1.el9.x86_64.r 5.7 MB/s | 192 kB 00:00 2023-12-08T08:36:12.601 INFO:teuthology.orchestra.run.smithi112.stdout:(137/147): elfutils-libs-0.190-1.el9.x86_64.rpm 6.0 MB/s | 255 kB 00:00 2023-12-08T08:36:12.635 INFO:teuthology.orchestra.run.smithi112.stdout:(138/147): libacl-2.3.1-4.el9.x86_64.rpm 658 kB/s | 22 kB 00:00 2023-12-08T08:36:12.685 INFO:teuthology.orchestra.run.smithi112.stdout:(139/147): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2023-12-08T08:36:12.744 INFO:teuthology.orchestra.run.smithi112.stdout:(140/147): librados2-19.0.0-46.ga2cd52c0.el9.x8 8.1 MB/s | 3.3 MB 00:00 2023-12-08T08:36:12.844 INFO:teuthology.orchestra.run.smithi112.stdout:(141/147): librbd1-19.0.0-46.ga2cd52c0.el9.x86_ 6.3 MB/s | 3.0 MB 00:00 2023-12-08T08:36:13.286 INFO:teuthology.orchestra.run.smithi112.stdout:(142/147): NetworkManager-team-1.45.7-1.el9.x86 104 kB/s | 43 kB 00:00 2023-12-08T08:36:13.612 INFO:teuthology.orchestra.run.smithi112.stdout:(143/147): NetworkManager-1.45.7-1.el9.x86_64.r 2.4 MB/s | 2.3 MB 00:00 2023-12-08T08:36:13.653 INFO:teuthology.orchestra.run.smithi112.stdout:(144/147): NetworkManager-libnm-1.45.7-1.el9.x8 2.0 MB/s | 1.8 MB 00:00 2023-12-08T08:36:13.678 INFO:teuthology.orchestra.run.smithi112.stdout:(145/147): NetworkManager-tui-1.45.7-1.el9.x86_ 630 kB/s | 246 kB 00:00 2023-12-08T08:36:13.754 INFO:teuthology.orchestra.run.smithi112.stdout:(146/147): openssl-3.0.7-25.el9.x86_64.rpm 8.6 MB/s | 1.2 MB 00:00 2023-12-08T08:36:13.854 INFO:teuthology.orchestra.run.smithi112.stdout:(147/147): openssl-libs-3.0.7-25.el9.x86_64.rpm 12 MB/s | 2.2 MB 00:00 2023-12-08T08:36:13.864 INFO:teuthology.orchestra.run.smithi112.stdout:-------------------------------------------------------------------------------- 2023-12-08T08:36:13.865 INFO:teuthology.orchestra.run.smithi112.stdout:Total 15 MB/s | 204 MB 00:13 2023-12-08T08:36:14.036 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction check 2023-12-08T08:36:14.220 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction check succeeded. 2023-12-08T08:36:14.220 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction test 2023-12-08T08:36:15.758 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-12-08T08:36:15.866 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/162 2023-12-08T08:36:15.899 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/162 2023-12-08T08:36:15.938 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/162 2023-12-08T08:36:15.975 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/162 2023-12-08T08:36:15.999 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/162 2023-12-08T08:36:16.052 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/162 2023-12-08T08:36:16.052 INFO:teuthology.orchestra.run.smithi112.stdout:Transaction test succeeded. 2023-12-08T08:36:16.053 INFO:teuthology.orchestra.run.smithi112.stdout:Running transaction 2023-12-08T08:36:16.393 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/162 2023-12-08T08:36:16.450 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.0.0-46.ga2cd52c0.el9.x86_64 8/162 2023-12-08T08:36:16.573 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.0.0-46.ga2cd52c0.el9.x86_64 8/162 2023-12-08T08:36:16.601 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.0.0-46.ga2cd52c0.el9.x86_64 9/162 2023-12-08T08:36:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.0.0-46.ga2cd52c0.el9.x86_64 9/162 2023-12-08T08:36:16.739 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.0.0-46.ga2cd52c0.el9.x86_64 10/162 2023-12-08T08:36:16.778 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/162 2023-12-08T08:36:16.975 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/162 2023-12-08T08:36:17.020 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/162 2023-12-08T08:36:17.048 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.0.0-46.ga2cd52c0.el9.x86_64 14/162 2023-12-08T08:36:17.159 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.0.0-46.ga2cd52c0.el9.x86_64 14/162 2023-12-08T08:36:17.184 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.0.0-46.ga2cd52c0.el9.x86_6 15/162 2023-12-08T08:36:17.398 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.0.0-46.ga2cd52c0.el9.x86_6 15/162 2023-12-08T08:36:17.505 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/162 2023-12-08T08:36:17.549 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/162 2023-12-08T08:36:17.638 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/162 2023-12-08T08:36:17.683 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.45.7-1.el9.x86_64 17/162 2023-12-08T08:36:17.887 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/162 2023-12-08T08:36:17.989 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/162 2023-12-08T08:36:18.021 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/162 2023-12-08T08:36:18.095 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/162 2023-12-08T08:36:18.146 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/162 2023-12-08T08:36:18.175 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/162 2023-12-08T08:36:18.219 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/162 2023-12-08T08:36:18.259 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/162 2023-12-08T08:36:18.333 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/162 2023-12-08T08:36:18.375 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/162 2023-12-08T08:36:18.402 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/162 2023-12-08T08:36:18.474 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/162 2023-12-08T08:36:18.507 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.0.0-46.ga2cd52c0.el9.x8 29/162 2023-12-08T08:36:18.551 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.0.0-46.ga2cd52c0.el9. 30/162 2023-12-08T08:36:18.627 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.0.0-46.ga2cd52c0.el9.x86_64 31/162 2023-12-08T08:36:18.764 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/162 2023-12-08T08:36:18.812 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/162 2023-12-08T08:36:19.372 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/162 2023-12-08T08:36:19.450 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/162 2023-12-08T08:36:19.487 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/162 2023-12-08T08:36:19.529 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/162 2023-12-08T08:36:19.558 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.0.0-46.ga2cd52c0.el9.x86_64 38/162 2023-12-08T08:36:19.600 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/162 2023-12-08T08:36:19.738 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/162 2023-12-08T08:36:19.784 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/162 2023-12-08T08:36:19.894 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/162 2023-12-08T08:36:19.929 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 43/162 2023-12-08T08:36:19.985 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/162 2023-12-08T08:36:20.058 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/162 2023-12-08T08:36:20.096 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/162 2023-12-08T08:36:20.133 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/162 2023-12-08T08:36:20.180 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/162 2023-12-08T08:36:20.225 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/162 2023-12-08T08:36:20.267 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/162 2023-12-08T08:36:20.407 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/162 2023-12-08T08:36:20.444 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/162 2023-12-08T08:36:20.485 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/162 2023-12-08T08:36:20.602 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/162 2023-12-08T08:36:20.636 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/162 2023-12-08T08:36:20.678 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/162 2023-12-08T08:36:20.709 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/162 2023-12-08T08:36:20.807 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 58/162 2023-12-08T08:36:20.854 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/162 2023-12-08T08:36:20.912 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/162 2023-12-08T08:36:20.992 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/162 2023-12-08T08:36:21.191 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/162 2023-12-08T08:36:21.326 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/162 2023-12-08T08:36:21.616 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/162 2023-12-08T08:36:21.662 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/162 2023-12-08T08:36:21.691 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/162 2023-12-08T08:36:21.732 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/162 2023-12-08T08:36:21.765 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/162 2023-12-08T08:36:21.797 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/162 2023-12-08T08:36:21.815 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/162 2023-12-08T08:36:21.844 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/162 2023-12-08T08:36:21.879 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/162 2023-12-08T08:36:21.908 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/162 2023-12-08T08:36:21.984 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/162 2023-12-08T08:36:22.014 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/162 2023-12-08T08:36:22.039 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/162 2023-12-08T08:36:22.081 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/162 2023-12-08T08:36:22.115 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/162 2023-12-08T08:36:22.207 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/162 2023-12-08T08:36:22.236 INFO:teuthology.orchestra.run.smithi112.stdout: Preparing : 1/1 2023-12-08T08:36:22.242 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/162 2023-12-08T08:36:22.349 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/162 2023-12-08T08:36:22.376 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/162 2023-12-08T08:36:22.403 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/162 2023-12-08T08:36:22.408 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/162 2023-12-08T08:36:22.464 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/162 2023-12-08T08:36:22.499 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/162 2023-12-08T08:36:22.525 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/162 2023-12-08T08:36:22.563 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/162 2023-12-08T08:36:22.892 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/162 2023-12-08T08:36:22.943 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librados2-2:19.0.0-46.ga2cd52c0.el9.x86_64 8/162 2023-12-08T08:36:23.047 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:19.0.0-46.ga2cd52c0.el9.x86_64 8/162 2023-12-08T08:36:23.078 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs2-2:19.0.0-46.ga2cd52c0.el9.x86_64 9/162 2023-12-08T08:36:23.153 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephfs2-2:19.0.0-46.ga2cd52c0.el9.x86_64 9/162 2023-12-08T08:36:23.174 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/162 2023-12-08T08:36:23.191 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rados-2:19.0.0-46.ga2cd52c0.el9.x86_64 10/162 2023-12-08T08:36:23.216 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/162 2023-12-08T08:36:23.234 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/162 2023-12-08T08:36:23.249 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/162 2023-12-08T08:36:23.299 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/162 2023-12-08T08:36:23.416 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/162 2023-12-08T08:36:23.463 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/162 2023-12-08T08:36:23.483 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/162 2023-12-08T08:36:23.491 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephsqlite-2:19.0.0-46.ga2cd52c0.el9.x86_64 14/162 2023-12-08T08:36:23.513 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/162 2023-12-08T08:36:23.592 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libcephsqlite-2:19.0.0-46.ga2cd52c0.el9.x86_64 14/162 2023-12-08T08:36:23.594 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/162 2023-12-08T08:36:23.617 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libradosstriper1-2:19.0.0-46.ga2cd52c0.el9.x86_6 15/162 2023-12-08T08:36:23.826 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: libradosstriper1-2:19.0.0-46.ga2cd52c0.el9.x86_6 15/162 2023-12-08T08:36:23.948 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/162 2023-12-08T08:36:23.991 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/162 2023-12-08T08:36:24.091 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/162 2023-12-08T08:36:24.142 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-1:1.45.7-1.el9.x86_64 17/162 2023-12-08T08:36:24.295 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/162 2023-12-08T08:36:24.348 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/162 2023-12-08T08:36:24.406 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/162 2023-12-08T08:36:24.449 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/162 2023-12-08T08:36:24.488 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/162 2023-12-08T08:36:24.573 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/162 2023-12-08T08:36:24.628 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/162 2023-12-08T08:36:24.659 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/162 2023-12-08T08:36:24.701 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/162 2023-12-08T08:36:24.750 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/162 2023-12-08T08:36:24.814 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/162 2023-12-08T08:36:24.849 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/162 2023-12-08T08:36:24.885 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/162 2023-12-08T08:36:24.953 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/162 2023-12-08T08:36:24.990 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-common-2:19.0.0-46.ga2cd52c0.el9.x8 29/162 2023-12-08T08:36:25.033 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ceph-argparse-2:19.0.0-46.ga2cd52c0.el9. 30/162 2023-12-08T08:36:25.104 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cephfs-2:19.0.0-46.ga2cd52c0.el9.x86_64 31/162 2023-12-08T08:36:25.238 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/162 2023-12-08T08:36:25.292 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/162 2023-12-08T08:36:25.718 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/162 2023-12-08T08:36:25.805 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/162 2023-12-08T08:36:25.807 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/162 2023-12-08T08:36:25.839 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/162 2023-12-08T08:36:25.869 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : jq-1.6-16.el9.x86_64 95/162 2023-12-08T08:36:25.889 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/162 2023-12-08T08:36:25.897 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/162 2023-12-08T08:36:25.930 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/162 2023-12-08T08:36:25.931 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 97/162 2023-12-08T08:36:25.978 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/162 2023-12-08T08:36:26.015 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librados-devel-2:19.0.0-46.ga2cd52c0.el9.x86_64 38/162 2023-12-08T08:36:26.057 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/162 2023-12-08T08:36:26.197 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/162 2023-12-08T08:36:26.241 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/162 2023-12-08T08:36:26.249 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/162 2023-12-08T08:36:26.277 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.0.0-46.ga2cd52c0.el9.x86_64 99/162 2023-12-08T08:36:26.346 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/162 2023-12-08T08:36:26.368 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.0.0-46.ga2cd52c0.el9.x86_64 99/162 2023-12-08T08:36:26.385 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 43/162 2023-12-08T08:36:26.398 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.0.0-46.ga2cd52c0.el9.x86_64 100/162 2023-12-08T08:36:26.427 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/162 2023-12-08T08:36:26.439 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/162 2023-12-08T08:36:26.511 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/162 2023-12-08T08:36:26.544 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/162 2023-12-08T08:36:26.589 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/162 2023-12-08T08:36:26.635 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/162 2023-12-08T08:36:26.680 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/162 2023-12-08T08:36:26.722 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/162 2023-12-08T08:36:26.853 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/162 2023-12-08T08:36:26.873 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/162 2023-12-08T08:36:26.891 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/162 2023-12-08T08:36:26.905 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.0.0-46.ga2cd52c0.el9.x86_64 103/162 2023-12-08T08:36:26.932 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/162 2023-12-08T08:36:26.968 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.0.0-46.ga2cd52c0.el9.x86_64 103/162 2023-12-08T08:36:26.990 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.0.0-46.ga2cd52c0.el9.x86_64 104/162 2023-12-08T08:36:27.024 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/162 2023-12-08T08:36:27.059 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/162 2023-12-08T08:36:27.124 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/162 2023-12-08T08:36:27.157 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/162 2023-12-08T08:36:27.249 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 58/162 2023-12-08T08:36:27.292 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/162 2023-12-08T08:36:27.342 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/162 2023-12-08T08:36:27.565 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/162 2023-12-08T08:36:27.752 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/162 2023-12-08T08:36:27.805 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/162 2023-12-08T08:36:27.934 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/162 2023-12-08T08:36:27.974 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/162 2023-12-08T08:36:28.003 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/162 2023-12-08T08:36:28.037 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/162 2023-12-08T08:36:28.070 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/162 2023-12-08T08:36:28.100 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/162 2023-12-08T08:36:28.128 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/162 2023-12-08T08:36:28.165 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/162 2023-12-08T08:36:28.199 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/162 2023-12-08T08:36:28.229 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/162 2023-12-08T08:36:28.305 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/162 2023-12-08T08:36:28.340 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/162 2023-12-08T08:36:28.373 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/162 2023-12-08T08:36:28.414 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/162 2023-12-08T08:36:28.449 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/162 2023-12-08T08:36:28.528 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/162 2023-12-08T08:36:28.559 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/162 2023-12-08T08:36:28.660 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/162 2023-12-08T08:36:28.716 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/162 2023-12-08T08:36:29.148 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-46.ga2cd52c0.el9.x86_64 105/162 2023-12-08T08:36:29.186 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.0.0-46.ga2cd52c0.el9.x86_64 105/162 2023-12-08T08:36:29.429 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/162 2023-12-08T08:36:29.472 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/162 2023-12-08T08:36:29.505 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/162 2023-12-08T08:36:29.538 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/162 2023-12-08T08:36:29.708 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/162 2023-12-08T08:36:29.745 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/162 2023-12-08T08:36:29.822 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/162 2023-12-08T08:36:29.841 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.0.0-46.ga2cd52c0.el9.x86_64 105/162 2023-12-08T08:36:29.873 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.0.0-46.ga2cd52c0.el9.x86_64 106/162 2023-12-08T08:36:29.989 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.0.0-46.ga2cd52c0.el9.x86_64 106/162 2023-12-08T08:36:29.989 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-12-08T08:36:29.989 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-12-08T08:36:29.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:36:30.025 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.0.0-46.ga2cd52c0.el9.x86_64 107/162 2023-12-08T08:36:30.460 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/162 2023-12-08T08:36:30.573 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/162 2023-12-08T08:36:31.792 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/162 2023-12-08T08:36:31.869 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/162 2023-12-08T08:36:31.901 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/162 2023-12-08T08:36:31.931 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : jq-1.6-16.el9.x86_64 95/162 2023-12-08T08:36:31.960 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/162 2023-12-08T08:36:31.993 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 97/162 2023-12-08T08:36:32.294 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/162 2023-12-08T08:36:32.322 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : librbd1-2:19.0.0-46.ga2cd52c0.el9.x86_64 99/162 2023-12-08T08:36:32.413 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:19.0.0-46.ga2cd52c0.el9.x86_64 99/162 2023-12-08T08:36:32.444 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rbd-2:19.0.0-46.ga2cd52c0.el9.x86_64 100/162 2023-12-08T08:36:32.480 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/162 2023-12-08T08:36:32.900 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/162 2023-12-08T08:36:32.934 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : librgw2-2:19.0.0-46.ga2cd52c0.el9.x86_64 103/162 2023-12-08T08:36:33.005 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librgw2-2:19.0.0-46.ga2cd52c0.el9.x86_64 103/162 2023-12-08T08:36:33.027 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : python3-rgw-2:19.0.0-46.ga2cd52c0.el9.x86_64 104/162 2023-12-08T08:36:35.053 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-46.ga2cd52c0.el9.x86_64 105/162 2023-12-08T08:36:35.090 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-common-2:19.0.0-46.ga2cd52c0.el9.x86_64 105/162 2023-12-08T08:36:35.630 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-common-2:19.0.0-46.ga2cd52c0.el9.x86_64 105/162 2023-12-08T08:36:35.668 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-base-2:19.0.0-46.ga2cd52c0.el9.x86_64 106/162 2023-12-08T08:36:35.758 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-base-2:19.0.0-46.ga2cd52c0.el9.x86_64 106/162 2023-12-08T08:36:35.758 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-12-08T08:36:35.758 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-12-08T08:36:35.758 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:36:35.794 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-selinux-2:19.0.0-46.ga2cd52c0.el9.x86_64 107/162 2023-12-08T08:36:42.071 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.0.0-46.ga2cd52c0.el9.x86_64 107/162 2023-12-08T08:36:42.071 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2023-12-08T08:36:42.072 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2023-12-08T08:36:42.072 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2023-12-08T08:36:42.072 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2023-12-08T08:36:42.072 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2023-12-08T08:36:42.072 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2023-12-08T08:36:42.072 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2023-12-08T08:36:42.072 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:36:42.239 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.0.0-46.ga2cd52c0.el9.noarch 108/162 2023-12-08T08:36:42.255 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.0.0-46.ga2cd52c0.el9.noarch 108/162 2023-12-08T08:36:42.255 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:42.255 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-12-08T08:36:42.255 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:36:43.884 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.0.0-46.ga2cd52c0.el9.x86_64 109/162 2023-12-08T08:36:43.912 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.0.0-46.ga2cd52c0.el9.x86_64 109/162 2023-12-08T08:36:43.912 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:43.912 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-12-08T08:36:43.912 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-08T08:36:43.912 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-08T08:36:43.912 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:36:44.126 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.0.0-46.ga2cd52c0.el9.x86_64 110/162 2023-12-08T08:36:44.150 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.0.0-46.ga2cd52c0.el9.x86_64 110/162 2023-12-08T08:36:44.150 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:44.151 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-12-08T08:36:44.151 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-08T08:36:44.151 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-08T08:36:44.151 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:36:44.619 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.0.0-46.ga2cd52c0.el9.x86_64 111/162 2023-12-08T08:36:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.0.0-46.ga2cd52c0.el9.x86_64 111/162 2023-12-08T08:36:44.643 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:44.643 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-12-08T08:36:44.643 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-08T08:36:44.643 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-08T08:36:44.643 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:36:44.746 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 112/162 2023-12-08T08:36:45.159 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.0.0-46.ga2cd52c0.el9.noarch 113/162 2023-12-08T08:36:45.203 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.0.0-46.ga2cd52c0.el9.noarch 113/162 2023-12-08T08:36:45.233 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-46.ga2cd52c0.el9 114/162 2023-12-08T08:36:45.295 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-46.ga2cd52c0.el 115/162 2023-12-08T08:36:45.326 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-46.ga2cd52c0.el9.noarc 116/162 2023-12-08T08:36:45.565 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-46.ga2cd52c0.el9.noarc 116/162 2023-12-08T08:36:45.610 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-46.ga2cd52c0.el9.noa 117/162 2023-12-08T08:36:46.627 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-46.ga2cd52c0.el9.noa 117/162 2023-12-08T08:36:46.661 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-46.ga2cd5 118/162 2023-12-08T08:36:46.765 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-46.ga2cd5 118/162 2023-12-08T08:36:46.924 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-46.ga2cd52c0.el9. 119/162 2023-12-08T08:36:46.954 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.0.0-46.ga2cd52c0.el9.x86_64 120/162 2023-12-08T08:36:46.979 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.0.0-46.ga2cd52c0.el9.x86_64 120/162 2023-12-08T08:36:46.979 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:46.979 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-12-08T08:36:46.979 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-08T08:36:46.979 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-08T08:36:46.979 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:36:47.030 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.0.0-46.ga2cd52c0.el9.noarch 121/162 2023-12-08T08:36:47.041 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-46.ga2cd52c0.el9.noarch 121/162 2023-12-08T08:36:47.092 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.0.0-46.ga2cd52c0.el9.x86_64 122/162 2023-12-08T08:36:47.336 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-selinux-2:19.0.0-46.ga2cd52c0.el9.x86_64 107/162 2023-12-08T08:36:47.336 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /sys 2023-12-08T08:36:47.336 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /proc 2023-12-08T08:36:47.336 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /mnt 2023-12-08T08:36:47.337 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /var/tmp 2023-12-08T08:36:47.337 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /home 2023-12-08T08:36:47.337 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /root 2023-12-08T08:36:47.337 INFO:teuthology.orchestra.run.smithi112.stdout:skipping the directory /tmp 2023-12-08T08:36:47.337 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:36:47.496 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-volume-2:19.0.0-46.ga2cd52c0.el9.noarch 108/162 2023-12-08T08:36:47.512 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-volume-2:19.0.0-46.ga2cd52c0.el9.noarch 108/162 2023-12-08T08:36:47.512 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:47.512 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-12-08T08:36:47.512 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:36:47.909 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-45.el9.x86_64 123/162 2023-12-08T08:36:47.940 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.0.0-46.ga2cd52c0.el9.x86_64 124/162 2023-12-08T08:36:47.965 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-46.ga2cd52c0.el9.x86_64 124/162 2023-12-08T08:36:47.965 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:47.965 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-12-08T08:36:47.965 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-08T08:36:47.965 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-08T08:36:47.965 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:36:48.017 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-46.ga2cd52c 125/162 2023-12-08T08:36:48.040 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-46.ga2cd52c 125/162 2023-12-08T08:36:48.040 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:48.040 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-12-08T08:36:48.040 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:36:48.092 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 126/162 2023-12-08T08:36:48.116 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 126/162 2023-12-08T08:36:48.116 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:48.116 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-12-08T08:36:48.116 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-08T08:36:48.116 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-08T08:36:48.116 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:36:48.426 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 127/162 2023-12-08T08:36:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 127/162 2023-12-08T08:36:48.450 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:48.450 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-12-08T08:36:48.450 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-08T08:36:48.450 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-08T08:36:48.450 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:36:49.074 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-osd-2:19.0.0-46.ga2cd52c0.el9.x86_64 109/162 2023-12-08T08:36:49.100 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-osd-2:19.0.0-46.ga2cd52c0.el9.x86_64 109/162 2023-12-08T08:36:49.100 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:49.100 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-12-08T08:36:49.100 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-08T08:36:49.100 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-08T08:36:49.100 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:36:49.308 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mds-2:19.0.0-46.ga2cd52c0.el9.x86_64 110/162 2023-12-08T08:36:49.331 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mds-2:19.0.0-46.ga2cd52c0.el9.x86_64 110/162 2023-12-08T08:36:49.331 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:49.331 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-12-08T08:36:49.331 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-08T08:36:49.331 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-08T08:36:49.331 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:36:49.776 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mon-2:19.0.0-46.ga2cd52c0.el9.x86_64 111/162 2023-12-08T08:36:49.798 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mon-2:19.0.0-46.ga2cd52c0.el9.x86_64 111/162 2023-12-08T08:36:49.798 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:49.799 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-12-08T08:36:49.799 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-08T08:36:49.799 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-08T08:36:49.799 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:36:49.894 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : automake-1.16.2-8.el9.noarch 112/162 2023-12-08T08:36:50.355 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephadm-2:19.0.0-46.ga2cd52c0.el9.noarch 113/162 2023-12-08T08:36:50.400 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephadm-2:19.0.0-46.ga2cd52c0.el9.noarch 113/162 2023-12-08T08:36:50.422 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-46.ga2cd52c0.el9 114/162 2023-12-08T08:36:50.474 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-46.ga2cd52c0.el 115/162 2023-12-08T08:36:50.498 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-46.ga2cd52c0.el9.noarc 116/162 2023-12-08T08:36:50.724 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-46.ga2cd52c0.el9.noarc 116/162 2023-12-08T08:36:50.766 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-46.ga2cd52c0.el9.noa 117/162 2023-12-08T08:36:51.727 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-46.ga2cd52c0.el9.noa 117/162 2023-12-08T08:36:51.758 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-46.ga2cd5 118/162 2023-12-08T08:36:51.857 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-46.ga2cd5 118/162 2023-12-08T08:36:52.007 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-46.ga2cd52c0.el9. 119/162 2023-12-08T08:36:52.035 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-2:19.0.0-46.ga2cd52c0.el9.x86_64 120/162 2023-12-08T08:36:52.058 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-2:19.0.0-46.ga2cd52c0.el9.x86_64 120/162 2023-12-08T08:36:52.058 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:52.058 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-12-08T08:36:52.058 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-08T08:36:52.058 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-08T08:36:52.058 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:36:52.102 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-mgr-rook-2:19.0.0-46.ga2cd52c0.el9.noarch 121/162 2023-12-08T08:36:52.113 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-46.ga2cd52c0.el9.noarch 121/162 2023-12-08T08:36:52.157 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-2:19.0.0-46.ga2cd52c0.el9.x86_64 122/162 2023-12-08T08:36:52.505 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.0.0-46.ga2cd52c0.el9.x86_64 128/162 2023-12-08T08:36:52.555 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.0.0-46.ga2cd52c0.el9.x86_64 129/162 2023-12-08T08:36:52.597 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.0.0-46.ga2cd52c0.el9.x86_64 130/162 2023-12-08T08:36:52.628 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/162 2023-12-08T08:36:52.830 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 132/162 2023-12-08T08:36:52.923 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libtool-2.4.6-45.el9.x86_64 123/162 2023-12-08T08:36:52.954 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-radosgw-2:19.0.0-46.ga2cd52c0.el9.x86_64 124/162 2023-12-08T08:36:52.955 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 133/162 2023-12-08T08:36:52.976 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-46.ga2cd52c0.el9.x86_64 124/162 2023-12-08T08:36:52.976 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-12-08T08:36:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-08T08:36:52.977 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-08T08:36:52.977 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:36:52.992 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.0.0-46.ga2cd52c0.el9.x86_64 134/162 2023-12-08T08:36:53.021 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-46.ga2cd52c 125/162 2023-12-08T08:36:53.025 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.0.0-46.ga2cd52c0.el9.x86_64 135/162 2023-12-08T08:36:53.043 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-46.ga2cd52c 125/162 2023-12-08T08:36:53.043 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:53.043 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-12-08T08:36:53.044 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:36:53.088 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 126/162 2023-12-08T08:36:53.091 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 136/162 2023-12-08T08:36:53.111 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 126/162 2023-12-08T08:36:53.111 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:53.111 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-12-08T08:36:53.111 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-08T08:36:53.111 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-08T08:36:53.112 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:36:53.151 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-12-08T08:36:53.194 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 138/162 2023-12-08T08:36:53.223 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.45.7-1.el9.x86_64 139/162 2023-12-08T08:36:53.260 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.45.7-1.el9.x86_64 140/162 2023-12-08T08:36:53.423 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 127/162 2023-12-08T08:36:53.446 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: rbd-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 127/162 2023-12-08T08:36:53.446 INFO:teuthology.orchestra.run.smithi112.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-08T08:36:53.446 INFO:teuthology.orchestra.run.smithi112.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-12-08T08:36:53.446 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-08T08:36:53.446 INFO:teuthology.orchestra.run.smithi112.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-08T08:36:53.446 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:36:53.896 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-12-08T08:36:53.935 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 141/162 2023-12-08T08:36:53.955 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-12-08T08:36:54.845 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.0.0-46.ga2cd52c0.el9.noarch 142/162 2023-12-08T08:36:54.966 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 143/162 2023-12-08T08:36:55.023 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 144/162 2023-12-08T08:36:55.066 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 145/162 2023-12-08T08:36:55.166 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 146/162 2023-12-08T08:36:55.202 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 147/162 2023-12-08T08:36:55.227 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/162 2023-12-08T08:36:55.253 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/162 2023-12-08T08:36:55.279 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 150/162 2023-12-08T08:36:55.279 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-12-08T08:36:55.347 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-12-08T08:36:55.375 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 152/162 2023-12-08T08:36:55.407 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 153/162 2023-12-08T08:36:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 154/162 2023-12-08T08:36:55.433 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/162 2023-12-08T08:36:55.517 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/162 2023-12-08T08:36:55.542 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-12-08T08:36:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-12-08T08:36:55.549 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-12-08T08:36:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-12-08T08:36:55.737 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-12-08T08:36:55.821 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-12-08T08:36:55.844 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 159/162 2023-12-08T08:36:55.870 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 160/162 2023-12-08T08:36:55.895 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 161/162 2023-12-08T08:36:55.963 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/162 2023-12-08T08:36:57.292 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-test-2:19.0.0-46.ga2cd52c0.el9.x86_64 128/162 2023-12-08T08:36:57.333 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-fuse-2:19.0.0-46.ga2cd52c0.el9.x86_64 129/162 2023-12-08T08:36:57.374 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : rbd-nbd-2:19.0.0-46.ga2cd52c0.el9.x86_64 130/162 2023-12-08T08:36:57.406 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/162 2023-12-08T08:36:57.582 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 132/162 2023-12-08T08:36:57.699 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 133/162 2023-12-08T08:36:57.730 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : ceph-fuse-2:19.0.0-46.ga2cd52c0.el9.x86_64 134/162 2023-12-08T08:36:57.762 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libcephfs-devel-2:19.0.0-46.ga2cd52c0.el9.x86_64 135/162 2023-12-08T08:36:57.825 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 136/162 2023-12-08T08:36:57.892 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-12-08T08:36:57.930 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 138/162 2023-12-08T08:36:57.960 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-team-1:1.45.7-1.el9.x86_64 139/162 2023-12-08T08:36:58.005 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : NetworkManager-tui-1:1.45.7-1.el9.x86_64 140/162 2023-12-08T08:36:58.605 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-12-08T08:36:58.646 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 141/162 2023-12-08T08:36:58.666 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-12-08T08:36:59.483 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : cephfs-top-2:19.0.0-46.ga2cd52c0.el9.noarch 142/162 2023-12-08T08:36:59.574 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 143/162 2023-12-08T08:36:59.642 INFO:teuthology.orchestra.run.smithi112.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 144/162 2023-12-08T08:36:59.684 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 145/162 2023-12-08T08:36:59.774 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : flex-2.6.4-9.el9.x86_64 146/162 2023-12-08T08:36:59.803 INFO:teuthology.orchestra.run.smithi112.stdout: Installing : bison-3.7.4-5.el9.x86_64 147/162 2023-12-08T08:36:59.829 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/162 2023-12-08T08:36:59.854 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/162 2023-12-08T08:36:59.881 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 150/162 2023-12-08T08:36:59.881 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-12-08T08:36:59.989 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-12-08T08:37:00.017 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 152/162 2023-12-08T08:37:00.054 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 153/162 2023-12-08T08:37:00.082 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 154/162 2023-12-08T08:37:00.082 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/162 2023-12-08T08:37:00.149 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/162 2023-12-08T08:37:00.174 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-12-08T08:37:00.189 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-12-08T08:37:00.189 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-12-08T08:37:00.369 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-12-08T08:37:00.369 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-12-08T08:37:00.383 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 162/162 2023-12-08T08:37:00.383 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.0.0-46.ga2cd52c0.el9.x86_64 1/162 2023-12-08T08:37:00.383 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.0.0-46.ga2cd52c0.el9.x86_64 2/162 2023-12-08T08:37:00.383 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.0.0-46.ga2cd52c0.el9.x86_64 3/162 2023-12-08T08:37:00.383 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.0.0-46.ga2cd52c0.el9.x86_64 4/162 2023-12-08T08:37:00.384 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-46.ga2cd52c 5/162 2023-12-08T08:37:00.384 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.0.0-46.ga2cd52c0.el9.x86_64 6/162 2023-12-08T08:37:00.384 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.0.0-46.ga2cd52c0.el9.x86_64 7/162 2023-12-08T08:37:00.384 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.0.0-46.ga2cd52c0.el9.x86_64 8/162 2023-12-08T08:37:00.384 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.0.0-46.ga2cd52c0.el9.x86_64 9/162 2023-12-08T08:37:00.384 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.0.0-46.ga2cd52c0.el9.x86_64 10/162 2023-12-08T08:37:00.384 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.0.0-46.ga2cd52c0.el9.x86_64 11/162 2023-12-08T08:37:00.384 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.0.0-46.ga2cd52c0.el9.x86_64 12/162 2023-12-08T08:37:00.384 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 13/162 2023-12-08T08:37:00.384 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.0.0-46.ga2cd52c0.el9.x86_64 14/162 2023-12-08T08:37:00.384 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.0.0-46.ga2cd52c0.el9.x86_64 15/162 2023-12-08T08:37:00.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.0.0-46.ga2cd52c0.el9.x86_64 16/162 2023-12-08T08:37:00.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.0.0-46.ga2cd52c0.el9.x86_64 17/162 2023-12-08T08:37:00.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.0.0-46.ga2cd52c0.el9.x86_6 18/162 2023-12-08T08:37:00.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.0.0-46.ga2cd52c0.el9.x86_64 19/162 2023-12-08T08:37:00.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.0.0-46.ga2cd52c0.el9. 20/162 2023-12-08T08:37:00.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.0.0-46.ga2cd52c0.el9.x8 21/162 2023-12-08T08:37:00.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.0.0-46.ga2cd52c0.el9.x86_64 22/162 2023-12-08T08:37:00.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.0.0-46.ga2cd52c0.el9.x86_64 23/162 2023-12-08T08:37:00.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.0.0-46.ga2cd52c0.el9.x86_64 24/162 2023-12-08T08:37:00.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.0.0-46.ga2cd52c0.el9.x86_64 25/162 2023-12-08T08:37:00.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.0.0-46.ga2cd52c0.el9.x86_64 26/162 2023-12-08T08:37:00.399 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 27/162 2023-12-08T08:37:00.399 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.0.0-46.ga2cd52c0.el9.x86_64 28/162 2023-12-08T08:37:00.399 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-46.ga2cd52c0.el 29/162 2023-12-08T08:37:00.399 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-46.ga2cd52c0.el9.noarc 30/162 2023-12-08T08:37:00.399 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-46.ga2cd52c0.el9.noa 31/162 2023-12-08T08:37:00.399 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-46.ga2cd5 32/162 2023-12-08T08:37:00.399 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-46.ga2cd52c0.el9. 33/162 2023-12-08T08:37:00.399 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.0.0-46.ga2cd52c0.el9.noarch 34/162 2023-12-08T08:37:00.399 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-46.ga2cd52c0.el9 35/162 2023-12-08T08:37:00.399 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.0.0-46.ga2cd52c0.el9.noarch 36/162 2023-12-08T08:37:00.400 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.0.0-46.ga2cd52c0.el9.noarch 37/162 2023-12-08T08:37:00.400 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.0.0-46.ga2cd52c0.el9.noarch 38/162 2023-12-08T08:37:00.400 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/162 2023-12-08T08:37:00.400 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/162 2023-12-08T08:37:00.400 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/162 2023-12-08T08:37:00.400 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/162 2023-12-08T08:37:00.400 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/162 2023-12-08T08:37:00.400 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/162 2023-12-08T08:37:00.400 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/162 2023-12-08T08:37:00.400 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/162 2023-12-08T08:37:00.400 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : jq-1.6-16.el9.x86_64 47/162 2023-12-08T08:37:00.401 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/162 2023-12-08T08:37:00.401 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/162 2023-12-08T08:37:00.401 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/162 2023-12-08T08:37:00.401 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/162 2023-12-08T08:37:00.401 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/162 2023-12-08T08:37:00.401 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/162 2023-12-08T08:37:00.401 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/162 2023-12-08T08:37:00.401 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/162 2023-12-08T08:37:00.401 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/162 2023-12-08T08:37:00.401 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/162 2023-12-08T08:37:00.401 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/162 2023-12-08T08:37:00.402 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/162 2023-12-08T08:37:00.402 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/162 2023-12-08T08:37:00.402 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/162 2023-12-08T08:37:00.402 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/162 2023-12-08T08:37:00.402 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/162 2023-12-08T08:37:00.402 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/162 2023-12-08T08:37:00.402 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/162 2023-12-08T08:37:00.402 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/162 2023-12-08T08:37:00.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/162 2023-12-08T08:37:00.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/162 2023-12-08T08:37:00.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/162 2023-12-08T08:37:00.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/162 2023-12-08T08:37:00.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/162 2023-12-08T08:37:00.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/162 2023-12-08T08:37:00.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/162 2023-12-08T08:37:00.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/162 2023-12-08T08:37:00.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/162 2023-12-08T08:37:00.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/162 2023-12-08T08:37:00.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/162 2023-12-08T08:37:00.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/162 2023-12-08T08:37:00.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/162 2023-12-08T08:37:00.403 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/162 2023-12-08T08:37:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/162 2023-12-08T08:37:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/162 2023-12-08T08:37:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/162 2023-12-08T08:37:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/162 2023-12-08T08:37:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/162 2023-12-08T08:37:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/162 2023-12-08T08:37:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/162 2023-12-08T08:37:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/162 2023-12-08T08:37:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/162 2023-12-08T08:37:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/162 2023-12-08T08:37:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/162 2023-12-08T08:37:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/162 2023-12-08T08:37:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/162 2023-12-08T08:37:00.404 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 96/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 97/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 100/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/162 2023-12-08T08:37:00.405 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/162 2023-12-08T08:37:00.406 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/162 2023-12-08T08:37:00.406 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/162 2023-12-08T08:37:00.406 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/162 2023-12-08T08:37:00.406 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/162 2023-12-08T08:37:00.406 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/162 2023-12-08T08:37:00.406 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/162 2023-12-08T08:37:00.406 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/162 2023-12-08T08:37:00.406 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/162 2023-12-08T08:37:00.406 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/162 2023-12-08T08:37:00.406 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/162 2023-12-08T08:37:00.406 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/162 2023-12-08T08:37:00.406 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/162 2023-12-08T08:37:00.406 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/162 2023-12-08T08:37:00.406 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.0.0-46.ga2cd52c0.el9.x86_64 133/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.0.0-46.ga2cd52c0.el9.x86_64 135/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-12-08T08:37:00.407 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : acl-2.3.1-4.el9.x86_64 139/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : acl-2.3.1-3.el9.x86_64 140/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 141/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 142/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 143/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 144/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 145/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 146/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 147/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 148/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 149/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 150/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.45.7-1.el9.x86_64 151/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 152/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 153/162 2023-12-08T08:37:00.408 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 154/162 2023-12-08T08:37:00.409 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.45.7-1.el9.x86_64 155/162 2023-12-08T08:37:00.409 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-12-08T08:37:00.409 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.45.7-1.el9.x86_64 157/162 2023-12-08T08:37:00.409 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 158/162 2023-12-08T08:37:00.409 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 159/162 2023-12-08T08:37:00.409 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 160/162 2023-12-08T08:37:00.409 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 161/162 2023-12-08T08:37:00.453 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-12-08T08:37:00.477 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 159/162 2023-12-08T08:37:00.502 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 160/162 2023-12-08T08:37:00.528 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 161/162 2023-12-08T08:37:00.584 INFO:teuthology.orchestra.run.smithi112.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/162 2023-12-08T08:37:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 162/162 2023-12-08T08:37:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:02.779 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2023-12-08T08:37:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.45.7-1.el9.x86_64 2023-12-08T08:37:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 2023-12-08T08:37:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.45.7-1.el9.x86_64 2023-12-08T08:37:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.45.7-1.el9.x86_64 2023-12-08T08:37:02.779 INFO:teuthology.orchestra.run.smithi016.stdout: acl-2.3.1-4.el9.x86_64 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-2.3.1-4.el9.x86_64 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-12-08T08:37:02.780 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-12-08T08:37:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2023-12-08T08:37:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-12-08T08:37:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-12-08T08:37:02.782 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-12-08T08:37:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-15.el9.x86_64 2023-12-08T08:37:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-12-08T08:37:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: jq-1.6-16.el9.x86_64 2023-12-08T08:37:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-12-08T08:37:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-10.el9.x86_64 2023-12-08T08:37:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-10.el9.noarch 2023-12-08T08:37:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-12-08T08:37:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2023-12-08T08:37:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-12-08T08:37:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-12-08T08:37:02.783 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-12-08T08:37:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-12-08T08:37:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-45.el9.x86_64 2023-12-08T08:37:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2023-12-08T08:37:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-12-08T08:37:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-12-08T08:37:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-12-08T08:37:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-12-08T08:37:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.21-2.el9.x86_64 2023-12-08T08:37:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-12-08T08:37:02.784 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-12-08T08:37:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-10.el9.x86_64 2023-12-08T08:37:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.11-1.el9.x86_64 2023-12-08T08:37:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-12-08T08:37:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-12-08T08:37:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.11-1.el9.x86_64 2023-12-08T08:37:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-12-08T08:37:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2023-12-08T08:37:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-12-08T08:37:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-12-08T08:37:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-12-08T08:37:02.785 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-12-08T08:37:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-12-08T08:37:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-12-08T08:37:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-12-08T08:37:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-12-08T08:37:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-12-08T08:37:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-12-08T08:37:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-12-08T08:37:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-12-08T08:37:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-12-08T08:37:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-12-08T08:37:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-12-08T08:37:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-12-08T08:37:02.786 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-12-08T08:37:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-12-08T08:37:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-12-08T08:37:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-12-08T08:37:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-12-08T08:37:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2023-12-08T08:37:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-12-08T08:37:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-12-08T08:37:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-12-08T08:37:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-12-08T08:37:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-12-08T08:37:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-12-08T08:37:02.787 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-7.el9.noarch 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2023-12-08T08:37:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2023-12-08T08:37:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-12-08T08:37:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-12-08T08:37:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2023-12-08T08:37:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-12-08T08:37:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-12-08T08:37:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.7-6.el9.noarch 2023-12-08T08:37:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-12-08T08:37:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-12-08T08:37:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-12-08T08:37:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:02.789 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-3.el9.x86_64 2023-12-08T08:37:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-5.el9.x86_64 2023-12-08T08:37:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2023-12-08T08:37:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2023-12-08T08:37:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-12-08T08:37:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-12-08T08:37:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-12-08T08:37:02.790 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:02.790 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-12-08T08:37:03.345 DEBUG:teuthology.parallel:result is None 2023-12-08T08:37:05.157 INFO:teuthology.orchestra.run.smithi112.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 162/162 2023-12-08T08:37:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-2:19.0.0-46.ga2cd52c0.el9.x86_64 1/162 2023-12-08T08:37:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-base-2:19.0.0-46.ga2cd52c0.el9.x86_64 2/162 2023-12-08T08:37:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-common-2:19.0.0-46.ga2cd52c0.el9.x86_64 3/162 2023-12-08T08:37:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-fuse-2:19.0.0-46.ga2cd52c0.el9.x86_64 4/162 2023-12-08T08:37:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-46.ga2cd52c 5/162 2023-12-08T08:37:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mds-2:19.0.0-46.ga2cd52c0.el9.x86_64 6/162 2023-12-08T08:37:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-2:19.0.0-46.ga2cd52c0.el9.x86_64 7/162 2023-12-08T08:37:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mon-2:19.0.0-46.ga2cd52c0.el9.x86_64 8/162 2023-12-08T08:37:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-osd-2:19.0.0-46.ga2cd52c0.el9.x86_64 9/162 2023-12-08T08:37:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-radosgw-2:19.0.0-46.ga2cd52c0.el9.x86_64 10/162 2023-12-08T08:37:05.158 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-selinux-2:19.0.0-46.ga2cd52c0.el9.x86_64 11/162 2023-12-08T08:37:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-test-2:19.0.0-46.ga2cd52c0.el9.x86_64 12/162 2023-12-08T08:37:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 13/162 2023-12-08T08:37:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs-devel-2:19.0.0-46.ga2cd52c0.el9.x86_64 14/162 2023-12-08T08:37:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephfs2-2:19.0.0-46.ga2cd52c0.el9.x86_64 15/162 2023-12-08T08:37:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libcephsqlite-2:19.0.0-46.ga2cd52c0.el9.x86_64 16/162 2023-12-08T08:37:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados-devel-2:19.0.0-46.ga2cd52c0.el9.x86_64 17/162 2023-12-08T08:37:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libradosstriper1-2:19.0.0-46.ga2cd52c0.el9.x86_6 18/162 2023-12-08T08:37:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librgw2-2:19.0.0-46.ga2cd52c0.el9.x86_64 19/162 2023-12-08T08:37:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-argparse-2:19.0.0-46.ga2cd52c0.el9. 20/162 2023-12-08T08:37:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ceph-common-2:19.0.0-46.ga2cd52c0.el9.x8 21/162 2023-12-08T08:37:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cephfs-2:19.0.0-46.ga2cd52c0.el9.x86_64 22/162 2023-12-08T08:37:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rados-2:19.0.0-46.ga2cd52c0.el9.x86_64 23/162 2023-12-08T08:37:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rbd-2:19.0.0-46.ga2cd52c0.el9.x86_64 24/162 2023-12-08T08:37:05.159 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rgw-2:19.0.0-46.ga2cd52c0.el9.x86_64 25/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-fuse-2:19.0.0-46.ga2cd52c0.el9.x86_64 26/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 27/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : rbd-nbd-2:19.0.0-46.ga2cd52c0.el9.x86_64 28/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-46.ga2cd52c0.el 29/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-46.ga2cd52c0.el9.noarc 30/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-46.ga2cd52c0.el9.noa 31/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-46.ga2cd5 32/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-46.ga2cd52c0.el9. 33/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-mgr-rook-2:19.0.0-46.ga2cd52c0.el9.noarch 34/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-46.ga2cd52c0.el9 35/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : ceph-volume-2:19.0.0-46.ga2cd52c0.el9.noarch 36/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephadm-2:19.0.0-46.ga2cd52c0.el9.noarch 37/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : cephfs-top-2:19.0.0-46.ga2cd52c0.el9.noarch 38/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/162 2023-12-08T08:37:05.160 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/162 2023-12-08T08:37:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/162 2023-12-08T08:37:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/162 2023-12-08T08:37:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/162 2023-12-08T08:37:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/162 2023-12-08T08:37:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/162 2023-12-08T08:37:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : jq-1.6-16.el9.x86_64 47/162 2023-12-08T08:37:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/162 2023-12-08T08:37:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/162 2023-12-08T08:37:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/162 2023-12-08T08:37:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/162 2023-12-08T08:37:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/162 2023-12-08T08:37:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/162 2023-12-08T08:37:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/162 2023-12-08T08:37:05.161 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/162 2023-12-08T08:37:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/162 2023-12-08T08:37:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/162 2023-12-08T08:37:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/162 2023-12-08T08:37:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/162 2023-12-08T08:37:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/162 2023-12-08T08:37:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/162 2023-12-08T08:37:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/162 2023-12-08T08:37:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/162 2023-12-08T08:37:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/162 2023-12-08T08:37:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/162 2023-12-08T08:37:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/162 2023-12-08T08:37:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/162 2023-12-08T08:37:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/162 2023-12-08T08:37:05.162 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/162 2023-12-08T08:37:05.163 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/162 2023-12-08T08:37:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/162 2023-12-08T08:37:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/162 2023-12-08T08:37:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/162 2023-12-08T08:37:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/162 2023-12-08T08:37:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/162 2023-12-08T08:37:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/162 2023-12-08T08:37:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/162 2023-12-08T08:37:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/162 2023-12-08T08:37:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/162 2023-12-08T08:37:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/162 2023-12-08T08:37:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/162 2023-12-08T08:37:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 96/162 2023-12-08T08:37:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 97/162 2023-12-08T08:37:05.164 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/162 2023-12-08T08:37:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/162 2023-12-08T08:37:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 100/162 2023-12-08T08:37:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/162 2023-12-08T08:37:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/162 2023-12-08T08:37:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/162 2023-12-08T08:37:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/162 2023-12-08T08:37:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/162 2023-12-08T08:37:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/162 2023-12-08T08:37:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/162 2023-12-08T08:37:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/162 2023-12-08T08:37:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/162 2023-12-08T08:37:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/162 2023-12-08T08:37:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/162 2023-12-08T08:37:05.165 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/162 2023-12-08T08:37:05.166 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:19.0.0-46.ga2cd52c0.el9.x86_64 133/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:19.0.0-46.ga2cd52c0.el9.x86_64 135/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : acl-2.3.1-4.el9.x86_64 139/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : acl-2.3.1-3.el9.x86_64 140/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 141/162 2023-12-08T08:37:05.167 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 142/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 143/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 144/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 145/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 146/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 147/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 148/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 149/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 150/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.45.7-1.el9.x86_64 151/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 152/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 153/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 154/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.45.7-1.el9.x86_64 155/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-12-08T08:37:05.168 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.45.7-1.el9.x86_64 157/162 2023-12-08T08:37:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 158/162 2023-12-08T08:37:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 159/162 2023-12-08T08:37:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 160/162 2023-12-08T08:37:05.169 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 161/162 2023-12-08T08:37:07.854 INFO:teuthology.orchestra.run.smithi112.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 162/162 2023-12-08T08:37:07.854 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:07.854 INFO:teuthology.orchestra.run.smithi112.stdout:Upgraded: 2023-12-08T08:37:07.854 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-1:1.45.7-1.el9.x86_64 2023-12-08T08:37:07.854 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 2023-12-08T08:37:07.854 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-team-1:1.45.7-1.el9.x86_64 2023-12-08T08:37:07.855 INFO:teuthology.orchestra.run.smithi112.stdout: NetworkManager-tui-1:1.45.7-1.el9.x86_64 2023-12-08T08:37:07.855 INFO:teuthology.orchestra.run.smithi112.stdout: acl-2.3.1-4.el9.x86_64 2023-12-08T08:37:07.855 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-12-08T08:37:07.855 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-12-08T08:37:07.855 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-12-08T08:37:07.855 INFO:teuthology.orchestra.run.smithi112.stdout: libacl-2.3.1-4.el9.x86_64 2023-12-08T08:37:07.855 INFO:teuthology.orchestra.run.smithi112.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-12-08T08:37:07.855 INFO:teuthology.orchestra.run.smithi112.stdout: librados2-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.855 INFO:teuthology.orchestra.run.smithi112.stdout: librbd1-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.855 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-12-08T08:37:07.855 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-12-08T08:37:07.855 INFO:teuthology.orchestra.run.smithi112.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-12-08T08:37:07.855 INFO:teuthology.orchestra.run.smithi112.stdout:Installed: 2023-12-08T08:37:07.855 INFO:teuthology.orchestra.run.smithi112.stdout: automake-1.16.2-8.el9.noarch 2023-12-08T08:37:07.856 INFO:teuthology.orchestra.run.smithi112.stdout: bison-3.7.4-5.el9.x86_64 2023-12-08T08:37:07.856 INFO:teuthology.orchestra.run.smithi112.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-12-08T08:37:07.856 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.856 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-base-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.856 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-common-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.856 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-fuse-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.856 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-grafana-dashboards-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:07.856 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-immutable-object-cache-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.856 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mds-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.856 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.856 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-cephadm-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:07.856 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-dashboard-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:07.856 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-diskprediction-local-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:07.856 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-modules-core-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:07.857 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mgr-rook-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:07.857 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-mon-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.857 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-osd-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.857 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-prometheus-alerts-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:07.857 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-radosgw-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.857 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-selinux-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.857 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-test-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.857 INFO:teuthology.orchestra.run.smithi112.stdout: ceph-volume-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:07.857 INFO:teuthology.orchestra.run.smithi112.stdout: cephadm-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:07.857 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.857 INFO:teuthology.orchestra.run.smithi112.stdout: cephfs-top-2:19.0.0-46.ga2cd52c0.el9.noarch 2023-12-08T08:37:07.857 INFO:teuthology.orchestra.run.smithi112.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-12-08T08:37:07.858 INFO:teuthology.orchestra.run.smithi112.stdout: flex-2.6.4-9.el9.x86_64 2023-12-08T08:37:07.858 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-12-08T08:37:07.858 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-12-08T08:37:07.858 INFO:teuthology.orchestra.run.smithi112.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-12-08T08:37:07.858 INFO:teuthology.orchestra.run.smithi112.stdout: fuse-2.9.9-15.el9.x86_64 2023-12-08T08:37:07.858 INFO:teuthology.orchestra.run.smithi112.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-12-08T08:37:07.858 INFO:teuthology.orchestra.run.smithi112.stdout: jq-1.6-16.el9.x86_64 2023-12-08T08:37:07.858 INFO:teuthology.orchestra.run.smithi112.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-12-08T08:37:07.858 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-9.0.0-10.el9.x86_64 2023-12-08T08:37:07.858 INFO:teuthology.orchestra.run.smithi112.stdout: libarrow-doc-9.0.0-10.el9.noarch 2023-12-08T08:37:07.858 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs-devel-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.858 INFO:teuthology.orchestra.run.smithi112.stdout: libcephfs2-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.858 INFO:teuthology.orchestra.run.smithi112.stdout: libcephsqlite-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.858 INFO:teuthology.orchestra.run.smithi112.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-12-08T08:37:07.859 INFO:teuthology.orchestra.run.smithi112.stdout: liboath-2.6.7-2.el9.x86_64 2023-12-08T08:37:07.859 INFO:teuthology.orchestra.run.smithi112.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-12-08T08:37:07.859 INFO:teuthology.orchestra.run.smithi112.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-12-08T08:37:07.859 INFO:teuthology.orchestra.run.smithi112.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-12-08T08:37:07.859 INFO:teuthology.orchestra.run.smithi112.stdout: librados-devel-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.859 INFO:teuthology.orchestra.run.smithi112.stdout: libradosstriper1-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.859 INFO:teuthology.orchestra.run.smithi112.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-12-08T08:37:07.859 INFO:teuthology.orchestra.run.smithi112.stdout: librgw2-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.859 INFO:teuthology.orchestra.run.smithi112.stdout: libtool-2.4.6-45.el9.x86_64 2023-12-08T08:37:07.859 INFO:teuthology.orchestra.run.smithi112.stdout: libunwind-1.6.2-1.el9.x86_64 2023-12-08T08:37:07.859 INFO:teuthology.orchestra.run.smithi112.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-12-08T08:37:07.859 INFO:teuthology.orchestra.run.smithi112.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-12-08T08:37:07.859 INFO:teuthology.orchestra.run.smithi112.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-12-08T08:37:07.859 INFO:teuthology.orchestra.run.smithi112.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-0.3.21-2.el9.x86_64 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: parquet-libs-9.0.0-10.el9.x86_64 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-13.11-1.el9.x86_64 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: postgresql-server-13.11-1.el9.x86_64 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: python3-babel-2.9.1-2.el9.noarch 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-argparse-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ceph-common-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.860 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cephfs-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.861 INFO:teuthology.orchestra.run.smithi112.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-12-08T08:37:07.861 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-12-08T08:37:07.861 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-12-08T08:37:07.861 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-12-08T08:37:07.861 INFO:teuthology.orchestra.run.smithi112.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-12-08T08:37:07.861 INFO:teuthology.orchestra.run.smithi112.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-12-08T08:37:07.861 INFO:teuthology.orchestra.run.smithi112.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-12-08T08:37:07.861 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-12-08T08:37:07.861 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-12-08T08:37:07.861 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-12-08T08:37:07.861 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-12-08T08:37:07.861 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-12-08T08:37:07.861 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-12-08T08:37:07.861 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-12-08T08:37:07.862 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-12-08T08:37:07.862 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-12-08T08:37:07.862 INFO:teuthology.orchestra.run.smithi112.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-12-08T08:37:07.862 INFO:teuthology.orchestra.run.smithi112.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-12-08T08:37:07.862 INFO:teuthology.orchestra.run.smithi112.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-12-08T08:37:07.863 INFO:teuthology.orchestra.run.smithi112.stdout: python3-mako-1.1.4-6.el9.noarch 2023-12-08T08:37:07.863 INFO:teuthology.orchestra.run.smithi112.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-12-08T08:37:07.863 INFO:teuthology.orchestra.run.smithi112.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-12-08T08:37:07.863 INFO:teuthology.orchestra.run.smithi112.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-12-08T08:37:07.863 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-12-08T08:37:07.863 INFO:teuthology.orchestra.run.smithi112.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-12-08T08:37:07.863 INFO:teuthology.orchestra.run.smithi112.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-12-08T08:37:07.863 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-12-08T08:37:07.863 INFO:teuthology.orchestra.run.smithi112.stdout: python3-ply-3.11-14.el9.noarch 2023-12-08T08:37:07.863 INFO:teuthology.orchestra.run.smithi112.stdout: python3-portend-3.1.0-2.el9.noarch 2023-12-08T08:37:07.863 INFO:teuthology.orchestra.run.smithi112.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-12-08T08:37:07.863 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-12-08T08:37:07.863 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pycparser-2.20-6.el9.noarch 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-pytz-2021.1-5.el9.noarch 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rados-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rbd-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-2.25.1-7.el9.noarch 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rgw-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-routes-2.5.1-5.el9.noarch 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-rsa-4.9-2.el9.noarch 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-toml-0.10.2-6.el9.noarch 2023-12-08T08:37:07.864 INFO:teuthology.orchestra.run.smithi112.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-12-08T08:37:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-12-08T08:37:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: python3-webob-1.8.7-6.el9.noarch 2023-12-08T08:37:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-12-08T08:37:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-12-08T08:37:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-12-08T08:37:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-fuse-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-mirror-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: rbd-nbd-2:19.0.0-46.ga2cd52c0.el9.x86_64 2023-12-08T08:37:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: re2-1:20211101-3.el9.x86_64 2023-12-08T08:37:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: socat-1.7.4.1-5.el9.x86_64 2023-12-08T08:37:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: thrift-0.15.0-2.el9.x86_64 2023-12-08T08:37:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: uuid-1.6.2-55.el9.x86_64 2023-12-08T08:37:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-12-08T08:37:07.865 INFO:teuthology.orchestra.run.smithi112.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-12-08T08:37:07.866 INFO:teuthology.orchestra.run.smithi112.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-12-08T08:37:07.866 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:07.866 INFO:teuthology.orchestra.run.smithi112.stdout:Complete! 2023-12-08T08:37:08.417 DEBUG:teuthology.parallel:result is None 2023-12-08T08:37:08.417 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a2cd52c00149b5caff897ea7f79d0529b62620da 2023-12-08T08:37:08.637 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-08T08:37:08.663 INFO:teuthology.orchestra.run.smithi016.stdout:19.0.0-46.ga2cd52c0.el9 2023-12-08T08:37:08.663 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-46.ga2cd52c0.el9 2023-12-08T08:37:08.664 INFO:teuthology.task.install:The correct ceph version 19.0.0-46.ga2cd52c0 is installed. 2023-12-08T08:37:08.664 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a2cd52c00149b5caff897ea7f79d0529b62620da 2023-12-08T08:37:08.807 DEBUG:teuthology.orchestra.run.smithi112:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-08T08:37:08.829 INFO:teuthology.orchestra.run.smithi112.stdout:19.0.0-46.ga2cd52c0.el9 2023-12-08T08:37:08.829 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-46.ga2cd52c0.el9 2023-12-08T08:37:08.829 INFO:teuthology.task.install:The correct ceph version 19.0.0-46.ga2cd52c0 is installed. 2023-12-08T08:37:08.829 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-08T08:37:08.830 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:08.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-08T08:37:08.869 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:08.870 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-08T08:37:08.901 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-08T08:37:08.901 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:08.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2023-12-08T08:37:08.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-08T08:37:09.018 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:09.019 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/daemon-helper 2023-12-08T08:37:09.046 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-08T08:37:09.112 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-08T08:37:09.113 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:09.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-08T08:37:09.147 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-08T08:37:09.219 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:09.219 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-08T08:37:09.246 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-08T08:37:09.313 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-08T08:37:09.313 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:09.313 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2023-12-08T08:37:09.345 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-08T08:37:09.418 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:09.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/usr/bin/stdin-killer 2023-12-08T08:37:09.445 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-08T08:37:09.512 INFO:teuthology.run_tasks:Running task ceph... 2023-12-08T08:37:09.588 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-12-08T08:37:09.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2023-12-08T08:37:09.590 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 777 /var/log/ceph 2023-12-08T08:37:09.620 INFO:tasks.ceph:Disabling ceph logrotate... 2023-12-08T08:37:09.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-08T08:37:09.663 DEBUG:teuthology.orchestra.run.smithi112:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-08T08:37:09.693 INFO:tasks.ceph:Creating extra log directories... 2023-12-08T08:37:09.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-08T08:37:09.735 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-08T08:37:09.771 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-12-08T08:37:09.771 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-12-08T08:37:09.771 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-12-08T08:37:09.771 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:09.771 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-08T08:37:09.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-08T08:37:09.951 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-12-08T08:37:09.981 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:09.982 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-08T08:37:09.997 DEBUG:teuthology.orchestra.run.smithi112:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-08T08:37:10.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-12-08T08:37:10.145 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-12-08T08:37:10.145 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-12-08T08:37:10.145 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi/7483789', 'branch': 'wip-vshankar-testing-20231206.125818', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7483789', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-12-08_07:46:48-fs-wip-vshankar-testing-20231206.125818-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20231206.125818'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'a2cd52c00149b5caff897ea7f79d0529b62620da'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a2cd52c00149b5caff897ea7f79d0529b62620da'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20231206.125818', 'sha1': 'a2cd52c00149b5caff897ea7f79d0529b62620da'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 4099, 'sha1': 'a2cd52c00149b5caff897ea7f79d0529b62620da', 'sleep_before_teardown': 0, 'subset': '359/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20231206.125818', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a2cd52c00149b5caff897ea7f79d0529b62620da/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a2cd52c00149b5caff897ea7f79d0529b62620da', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi016.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF/oq4PR8P0nJ1+Zg8yIxbCvd5uplT1Kp/CNd9kDgv628WHtoI5X3/30LU03pgRbYeqM0LZ1UwK54Ov5z9suKKw=', 'smithi112.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGGxxAwWuNsIuQ1f/M3QMGpbiwr2qHjOiVWTolZZ5TlFO+WC81Y8dSPRPc2VYZtZkaCNTNp/UNxvCK+rcOPWngw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a2cd52c00149b5caff897ea7f79d0529b62620da'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'a2cd52c00149b5caff897ea7f79d0529b62620da', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'f2de108beb0a6dde7f3e6c96039071828a1f6269', 'timestamp': '2023-12-08_07:46:48', 'tube': 'smithi', 'use_shaman': True, 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-12-08T08:37:10.146 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-08T08:37:10.148 DEBUG:teuthology.orchestra.run.smithi112:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-08T08:37:10.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2023-12-08T08:37:10.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo install -d -m0777 -- /var/run/ceph 2023-12-08T08:37:10.261 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:10.261 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2023-12-08T08:37:10.287 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-08T08:37:10.287 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2023-12-08T08:37:10.343 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-08T08:37:10.343 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-08T08:37:10.343 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 665 Links: 1 2023-12-08T08:37:10.343 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-08T08:37:10.343 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-12-08T08:37:10.344 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-12-08 08:36:56.324484952 +0000 2023-12-08T08:37:10.344 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-12-08 08:34:54.003377085 +0000 2023-12-08T08:37:10.344 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-12-08 08:34:54.003377085 +0000 2023-12-08T08:37:10.344 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2023-12-08 08:34:54.003377085 +0000 2023-12-08T08:37:10.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-08T08:37:10.413 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-12-08T08:37:10.414 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-12-08T08:37:10.414 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 9.1525e-05 s, 5.6 MB/s 2023-12-08T08:37:10.415 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-08T08:37:10.471 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2023-12-08T08:37:10.527 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-08T08:37:10.527 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-08T08:37:10.527 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 683 Links: 1 2023-12-08T08:37:10.527 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-08T08:37:10.527 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-12-08T08:37:10.527 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-12-08 08:36:56.325484938 +0000 2023-12-08T08:37:10.527 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-12-08 08:34:54.504369846 +0000 2023-12-08T08:37:10.527 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-12-08 08:34:54.504369846 +0000 2023-12-08T08:37:10.527 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2023-12-08 08:34:54.504369846 +0000 2023-12-08T08:37:10.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-08T08:37:10.596 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-12-08T08:37:10.596 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-12-08T08:37:10.596 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 7.6281e-05 s, 6.7 MB/s 2023-12-08T08:37:10.597 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-08T08:37:10.653 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2023-12-08T08:37:10.709 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-08T08:37:10.709 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-08T08:37:10.709 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 700 Links: 1 2023-12-08T08:37:10.709 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-08T08:37:10.709 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-12-08T08:37:10.709 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-12-08 08:36:56.325484938 +0000 2023-12-08T08:37:10.709 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-12-08 08:34:55.017362433 +0000 2023-12-08T08:37:10.709 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-12-08 08:34:55.017362433 +0000 2023-12-08T08:37:10.709 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2023-12-08 08:34:55.017362433 +0000 2023-12-08T08:37:10.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-08T08:37:10.778 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-12-08T08:37:10.778 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-12-08T08:37:10.778 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 8.2405e-05 s, 6.2 MB/s 2023-12-08T08:37:10.779 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-08T08:37:10.836 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2023-12-08T08:37:10.891 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-08T08:37:10.891 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-08T08:37:10.891 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 722 Links: 1 2023-12-08T08:37:10.891 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-08T08:37:10.891 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-12-08T08:37:10.891 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-12-08 08:36:56.325484938 +0000 2023-12-08T08:37:10.891 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-12-08 08:34:55.523355122 +0000 2023-12-08T08:37:10.891 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-12-08 08:34:55.523355122 +0000 2023-12-08T08:37:10.891 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2023-12-08 08:34:55.523355122 +0000 2023-12-08T08:37:10.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-08T08:37:10.959 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-12-08T08:37:10.959 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-12-08T08:37:10.960 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 9.7122e-05 s, 5.3 MB/s 2023-12-08T08:37:10.961 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-08T08:37:11.017 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-08T08:37:11.017 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:11.017 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/scratch_devs of=/dev/stdout 2023-12-08T08:37:11.032 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-08T08:37:11.033 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_1 2023-12-08T08:37:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-08T08:37:11.088 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-08T08:37:11.088 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 665 Links: 1 2023-12-08T08:37:11.088 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-08T08:37:11.088 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-12-08T08:37:11.088 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-12-08 08:37:00.917991457 +0000 2023-12-08T08:37:11.089 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-12-08 08:33:43.616416364 +0000 2023-12-08T08:37:11.089 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-12-08 08:33:43.616416364 +0000 2023-12-08T08:37:11.089 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2023-12-08 08:33:43.616416364 +0000 2023-12-08T08:37:11.089 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-08T08:37:11.154 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-12-08T08:37:11.154 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-12-08T08:37:11.154 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000272542 s, 1.9 MB/s 2023-12-08T08:37:11.155 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-08T08:37:11.211 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_2 2023-12-08T08:37:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-08T08:37:11.266 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-08T08:37:11.266 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 681 Links: 1 2023-12-08T08:37:11.266 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-08T08:37:11.266 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-12-08T08:37:11.266 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-12-08 08:37:00.917991457 +0000 2023-12-08T08:37:11.266 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-12-08 08:33:44.119403095 +0000 2023-12-08T08:37:11.267 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-12-08 08:33:44.119403095 +0000 2023-12-08T08:37:11.267 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2023-12-08 08:33:44.119403095 +0000 2023-12-08T08:37:11.267 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-08T08:37:11.332 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-12-08T08:37:11.332 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-12-08T08:37:11.332 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000178761 s, 2.9 MB/s 2023-12-08T08:37:11.333 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-08T08:37:11.390 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_3 2023-12-08T08:37:11.445 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-08T08:37:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-08T08:37:11.446 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 702 Links: 1 2023-12-08T08:37:11.446 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-08T08:37:11.446 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-12-08T08:37:11.446 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-12-08 08:37:00.917991457 +0000 2023-12-08T08:37:11.446 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-12-08 08:33:44.603390327 +0000 2023-12-08T08:37:11.446 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-12-08 08:33:44.603390327 +0000 2023-12-08T08:37:11.446 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2023-12-08 08:33:44.603390327 +0000 2023-12-08T08:37:11.446 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-08T08:37:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-12-08T08:37:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-12-08T08:37:11.511 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000210355 s, 2.4 MB/s 2023-12-08T08:37:11.512 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-08T08:37:11.567 DEBUG:teuthology.orchestra.run.smithi112:> stat /dev/vg_nvme/lv_4 2023-12-08T08:37:11.621 INFO:teuthology.orchestra.run.smithi112.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-08T08:37:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-08T08:37:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:Device: 5h/5d Inode: 722 Links: 1 2023-12-08T08:37:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-08T08:37:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:Context: system_u:object_r:device_t:s0 2023-12-08T08:37:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:Access: 2023-12-08 08:37:00.917991457 +0000 2023-12-08T08:37:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:Modify: 2023-12-08 08:33:45.096377321 +0000 2023-12-08T08:37:11.622 INFO:teuthology.orchestra.run.smithi112.stdout:Change: 2023-12-08 08:33:45.096377321 +0000 2023-12-08T08:37:11.622 INFO:teuthology.orchestra.run.smithi112.stdout: Birth: 2023-12-08 08:33:45.096377321 +0000 2023-12-08T08:37:11.622 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-08T08:37:11.687 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records in 2023-12-08T08:37:11.687 INFO:teuthology.orchestra.run.smithi112.stderr:1+0 records out 2023-12-08T08:37:11.687 INFO:teuthology.orchestra.run.smithi112.stderr:512 bytes copied, 0.000173625 s, 2.9 MB/s 2023-12-08T08:37:11.688 DEBUG:teuthology.orchestra.run.smithi112:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-08T08:37:11.743 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-08T08:37:11.743 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-08T08:37:11.743 INFO:tasks.ceph:Generating config... 2023-12-08T08:37:11.745 INFO:tasks.ceph:[client] client mount timeout = 600 2023-12-08T08:37:11.745 INFO:tasks.ceph:[client] debug client = 20 2023-12-08T08:37:11.745 INFO:tasks.ceph:[client] debug ms = 1 2023-12-08T08:37:11.745 INFO:tasks.ceph:[client] pid file = 2023-12-08T08:37:11.745 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-12-08T08:37:11.745 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-12-08T08:37:11.745 INFO:tasks.ceph:[mds] debug mds = 20 2023-12-08T08:37:11.745 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-12-08T08:37:11.745 INFO:tasks.ceph:[mds] debug ms = 1 2023-12-08T08:37:11.745 INFO:tasks.ceph:[mds] mds debug frag = True 2023-12-08T08:37:11.745 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-12-08T08:37:11.746 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-12-08T08:37:11.746 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-12-08T08:37:11.746 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-12-08T08:37:11.746 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-12-08T08:37:11.746 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-12-08T08:37:11.746 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-12-08T08:37:11.746 INFO:tasks.ceph:[mgr] debug ms = 1 2023-12-08T08:37:11.746 INFO:tasks.ceph:[mon] debug mon = 20 2023-12-08T08:37:11.746 INFO:tasks.ceph:[mon] debug ms = 1 2023-12-08T08:37:11.746 INFO:tasks.ceph:[mon] debug paxos = 20 2023-12-08T08:37:11.746 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-12-08T08:37:11.746 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-12-08T08:37:11.746 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-12-08T08:37:11.746 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-12-08T08:37:11.747 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-12-08T08:37:11.747 INFO:tasks.ceph:[osd] debug ms = 1 2023-12-08T08:37:11.747 INFO:tasks.ceph:[osd] debug osd = 20 2023-12-08T08:37:11.747 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-12-08T08:37:11.747 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-12-08T08:37:11.747 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-12-08T08:37:11.747 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-12-08T08:37:11.747 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-12-08T08:37:11.747 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-12-08T08:37:11.747 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-12-08T08:37:11.747 INFO:tasks.ceph:Setting up mon.a... 2023-12-08T08:37:11.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-12-08T08:37:11.814 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2023-12-08T08:37:11.816 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-12-08T08:37:11.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-08T08:37:11.897 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16'), ('mon.b', '172.21.15.112'), ('mon.c', '[v2:172.21.15.112:3301,v1:172.21.15.112:6790]')] 2023-12-08T08:37:11.898 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-12-08T08:37:11.899 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:11.899 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-12-08T08:37:11.953 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --add b 172.21.15.112 --addv c '[v2:172.21.15.112:3301,v1:172.21.15.112:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-12-08T08:37:12.039 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-12-08T08:37:12.039 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid ead7ea87-fe4a-461b-916c-c6458a26d625 2023-12-08T08:37:12.039 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = pacific 2023-12-08T08:37:12.039 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2023-12-08T08:37:12.039 INFO:teuthology.orchestra.run.smithi016.stdout:fsid ead7ea87-fe4a-461b-916c-c6458a26d625 2023-12-08T08:37:12.039 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2023-12-08T08:37:12.037867+0000 2023-12-08T08:37:12.039 INFO:teuthology.orchestra.run.smithi016.stdout:created 2023-12-08T08:37:12.037867+0000 2023-12-08T08:37:12.039 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 16 (pacific) 2023-12-08T08:37:12.040 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2023-12-08T08:37:12.040 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2023-12-08T08:37:12.040 INFO:teuthology.orchestra.run.smithi016.stdout:1: [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] mon.b 2023-12-08T08:37:12.040 INFO:teuthology.orchestra.run.smithi016.stdout:2: [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] mon.c 2023-12-08T08:37:12.040 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-12-08T08:37:12.041 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-12-08T08:37:12.056 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ead7ea87-fe4a-461b-916c-c6458a26d625... 2023-12-08T08:37:12.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-08T08:37:12.099 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-08T08:37:12.150 INFO:teuthology.orchestra.run.smithi112.stdout:[global] 2023-12-08T08:37:12.150 INFO:teuthology.orchestra.run.smithi112.stdout: chdir = "" 2023-12-08T08:37:12.150 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-08T08:37:12.150 INFO:teuthology.orchestra.run.smithi112.stdout: auth supported = cephx 2023-12-08T08:37:12.150 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.150 INFO:teuthology.orchestra.run.smithi112.stdout: filestore xattr use omap = true 2023-12-08T08:37:12.150 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.150 INFO:teuthology.orchestra.run.smithi112.stdout: mon clock drift allowed = 1.000 2023-12-08T08:37:12.150 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.150 INFO:teuthology.orchestra.run.smithi112.stdout: osd crush chooseleaf type = 0 2023-12-08T08:37:12.150 INFO:teuthology.orchestra.run.smithi112.stdout: auth debug = true 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on old message = true 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: ms die on bug = true 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: mon pg warn max object skew = 0 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: # disable pg_autoscaler by default for new pools 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default size = 2 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow primary affinity = true 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd allow pg remap = true 2023-12-08T08:37:12.151 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on legacy crush tunables = false 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on crush straw calc version zero = false 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on no sortbitwise = false 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on osd down out interval zero = false 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: mon warn on too few osds = false 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: mon_allow_pool_size_one = true 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: osd default data pool replay window = 5 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: mon allow pool delete = true 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.152 INFO:teuthology.orchestra.run.smithi112.stdout: mon cluster log file level = debug 2023-12-08T08:37:12.153 INFO:teuthology.orchestra.run.smithi112.stdout: debug asserts on shutdown = true 2023-12-08T08:37:12.153 INFO:teuthology.orchestra.run.smithi112.stdout: mon health detail to clog = false 2023-12-08T08:37:12.153 INFO:teuthology.orchestra.run.smithi112.stdout: mon host = "172.21.15.16,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2023-12-08T08:37:12.153 INFO:teuthology.orchestra.run.smithi112.stdout: fsid = ead7ea87-fe4a-461b-916c-c6458a26d625 2023-12-08T08:37:12.153 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.153 INFO:teuthology.orchestra.run.smithi112.stdout:[osd] 2023-12-08T08:37:12.153 INFO:teuthology.orchestra.run.smithi112.stdout: osd journal size = 100 2023-12-08T08:37:12.153 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.153 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub load threshold = 5.0 2023-12-08T08:37:12.153 INFO:teuthology.orchestra.run.smithi112.stdout: osd scrub max interval = 600 2023-12-08T08:37:12.153 INFO:teuthology.orchestra.run.smithi112.stdout: osd mclock profile = high_recovery_ops 2023-12-08T08:37:12.153 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.153 INFO:teuthology.orchestra.run.smithi112.stdout: osd recover clone overlap = true 2023-12-08T08:37:12.153 INFO:teuthology.orchestra.run.smithi112.stdout: osd recovery max chunk = 1048576 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug shutdown = true 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug op order = true 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug verify stray on activate = true 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug trim objects = true 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: osd open classes on start = true 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug pg log writeout = true 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: osd deep scrub update digest min age = 30 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: osd map max advance = 10 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: journal zero on create = true 2023-12-08T08:37:12.154 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: filestore ondisk finisher threads = 3 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: filestore apply finisher threads = 3 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: bdev debug aio = true 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: osd debug misdirected ops = true 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore block size = 96636764160 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: bluestore fsck on mount = True 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluefs = 20 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: debug bluestore = 20 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: debug osd = 20 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: debug rocksdb = 10 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd backfillfull_ratio = 0.85 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd full ratio = 0.9 2023-12-08T08:37:12.155 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd nearfull ratio = 0.8 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout: osd failsafe full ratio = 0.95 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout: osd objectstore = bluestore 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr] 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout: debug mgr = 20 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout: mgr/telemetry/nag = false 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout:[mon] 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-12-08T08:37:12.156 INFO:teuthology.orchestra.run.smithi112.stdout: debug mon = 20 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: debug paxos = 20 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: debug auth = 20 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: mon data avail warn = 5 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: mon mgr mkfs grace = 240 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min pgs per osd = 4 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd reporter subtree level = osd 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: mon osd prime pg temp = true 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: mon reweight min bytes per osd = 10 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: auth mon ticket ttl = 660 # 11m 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: auth service ticket ttl = 240 # 4m 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: # don't complain about insecure global_id in the test suite 2023-12-08T08:37:12.157 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-08T08:37:12.158 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-08T08:37:12.158 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.158 INFO:teuthology.orchestra.run.smithi112.stdout: # 1m isn't quite enough 2023-12-08T08:37:12.158 INFO:teuthology.orchestra.run.smithi112.stdout: mon_down_mkfs_grace = 2m 2023-12-08T08:37:12.158 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.158 INFO:teuthology.orchestra.run.smithi112.stdout: mon_warn_on_filestore_osds = false 2023-12-08T08:37:12.158 INFO:teuthology.orchestra.run.smithi112.stdout: mon op complaint time = 120 2023-12-08T08:37:12.158 INFO:teuthology.orchestra.run.smithi112.stdout: 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout:[client] 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout: rgw cache enabled = true 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable ops log = true 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout: rgw enable usage log = true 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout: client mount timeout = 600 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout: debug client = 20 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout: pid file = "" 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.a] 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.b] 2023-12-08T08:37:12.159 INFO:teuthology.orchestra.run.smithi112.stdout:[mon.c] 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout:[mds] 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds = 20 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout: debug mds balancer = 20 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout: debug ms = 1 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug frag = True 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout: mds debug scatterstat = True 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout: mds op complaint time = 180 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout: mds verify scatter = True 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout: osd op complaint time = 180 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout: rados mon op timeout = 900 2023-12-08T08:37:12.160 INFO:teuthology.orchestra.run.smithi112.stdout: rados osd op timeout = 900 2023-12-08T08:37:12.161 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-12-08T08:37:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-12-08T08:37:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-08T08:37:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-12-08T08:37:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-12-08T08:37:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.161 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-12-08T08:37:12.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-12-08T08:37:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-12-08T08:37:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-12-08T08:37:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-12-08T08:37:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-12-08T08:37:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-12-08T08:37:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-12-08T08:37:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-08T08:37:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-08T08:37:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-12-08T08:37:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.163 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,172.21.15.112,[v2:172.21.15.112:3301,v1:172.21.15.112:6790]" 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = ead7ea87-fe4a-461b-916c-c6458a26d625 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.164 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug trim objects = true 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.165 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-12-08T08:37:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-12-08T08:37:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-12-08T08:37:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-12-08T08:37:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-12-08T08:37:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-12-08T08:37:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-12-08T08:37:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-12-08T08:37:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-12-08T08:37:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 20 2023-12-08T08:37:12.166 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 20 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 10 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-12-08T08:37:12.167 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.168 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-12-08T08:37:12.169 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = "" 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-12-08T08:37:12.170 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-12-08T08:37:12.171 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-12-08T08:37:12.171 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-12-08T08:37:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-12-08T08:37:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-12-08T08:37:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-12-08T08:37:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-12-08T08:37:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2023-12-08T08:37:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-12-08T08:37:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2023-12-08T08:37:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-12-08T08:37:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-12-08T08:37:12.171 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-12-08T08:37:12.175 INFO:tasks.ceph:Creating admin key on mon.a... 2023-12-08T08:37:12.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-12-08T08:37:12.275 INFO:tasks.ceph:Copying monmap to all nodes... 2023-12-08T08:37:12.275 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:12.275 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-12-08T08:37:12.290 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:12.290 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-12-08T08:37:12.345 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2023-12-08T08:37:12.346 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:12.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-08T08:37:12.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-08T08:37:12.432 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:12.432 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-08T08:37:12.488 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi112.front.sepia.ceph.com 2023-12-08T08:37:12.488 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:12.488 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-08T08:37:12.488 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-08T08:37:12.527 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:12.528 DEBUG:teuthology.orchestra.run.smithi112:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-08T08:37:12.583 INFO:tasks.ceph:Setting up mon nodes... 2023-12-08T08:37:12.583 INFO:tasks.ceph:Setting up mgr nodes... 2023-12-08T08:37:12.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-12-08T08:37:12.646 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-12-08T08:37:12.648 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-12-08T08:37:12.709 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-12-08T08:37:12.711 INFO:tasks.ceph:Setting up mds nodes... 2023-12-08T08:37:12.711 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-12-08T08:37:12.775 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-12-08T08:37:12.778 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-12-08T08:37:12.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-12-08T08:37:12.910 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-12-08T08:37:12.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-12-08T08:37:12.944 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-12-08T08:37:12.999 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-12-08T08:37:13.001 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-12-08T08:37:13.029 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-12-08T08:37:13.123 INFO:teuthology.orchestra.run.smithi112.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-12-08T08:37:13.125 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-12-08T08:37:13.152 INFO:tasks.ceph_client:Setting up client nodes... 2023-12-08T08:37:13.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-08T08:37:13.199 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-12-08T08:37:13.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-12-08T08:37:13.264 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-12-08T08:37:13.283 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-12-08T08:37:13.325 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-12-08T08:37:13.340 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-12-08T08:37:13.422 INFO:teuthology.orchestra.run.smithi112.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-12-08T08:37:13.437 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-12-08T08:37:13.437 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi112.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-08T08:37:13.437 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-12-08T08:37:13.468 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-08T08:37:13.469 INFO:tasks.ceph:role: osd.0 2023-12-08T08:37:13.469 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2023-12-08T08:37:13.469 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-08T08:37:13.539 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-08T08:37:13.539 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-08T08:37:13.539 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-08T08:37:13.539 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-08T08:37:13.539 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-08T08:37:13.539 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-12-08T08:37:13.540 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-08T08:37:13.540 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-08T08:37:13.540 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-08T08:37:13.540 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-08T08:37:13.586 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-12-08T08:37:13.588 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-12-08T08:37:13.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-12-08T08:37:13.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-12-08T08:37:13.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-12-08T08:37:13.813 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-08T08:37:13.813 INFO:tasks.ceph:role: osd.1 2023-12-08T08:37:13.813 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com 2023-12-08T08:37:13.813 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-08T08:37:13.883 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-08T08:37:13.883 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-08T08:37:13.883 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-08T08:37:13.883 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-08T08:37:13.883 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-08T08:37:13.884 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-12-08T08:37:13.884 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-08T08:37:13.884 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-08T08:37:13.884 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-08T08:37:13.884 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-08T08:37:13.926 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-12-08T08:37:13.928 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-12-08T08:37:13.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-12-08T08:37:14.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-12-08T08:37:14.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-12-08T08:37:14.152 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-08T08:37:14.152 INFO:tasks.ceph:role: osd.2 2023-12-08T08:37:14.152 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com 2023-12-08T08:37:14.152 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-08T08:37:14.222 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-08T08:37:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-08T08:37:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-08T08:37:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-08T08:37:14.222 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-08T08:37:14.222 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-12-08T08:37:14.222 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-08T08:37:14.223 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-08T08:37:14.223 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-08T08:37:14.223 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-08T08:37:14.264 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-12-08T08:37:14.266 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-12-08T08:37:14.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-12-08T08:37:14.341 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-12-08T08:37:14.416 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-12-08T08:37:14.486 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-08T08:37:14.486 INFO:tasks.ceph:role: osd.3 2023-12-08T08:37:14.486 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com 2023-12-08T08:37:14.486 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-08T08:37:14.556 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-08T08:37:14.557 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-08T08:37:14.557 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-08T08:37:14.557 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-08T08:37:14.557 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-08T08:37:14.557 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-12-08T08:37:14.557 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-08T08:37:14.557 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-08T08:37:14.557 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-08T08:37:14.557 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-08T08:37:14.597 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-12-08T08:37:14.599 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-12-08T08:37:14.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-12-08T08:37:14.674 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-12-08T08:37:14.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-08T08:37:14.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:14.845+0000 7fca3020f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-12-08T08:37:14.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:14.845+0000 7fca3020f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-12-08T08:37:14.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:14.845+0000 7fca3020f500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-08T08:37:14.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:14.845+0000 7fca3020f500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-08T08:37:14.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:14.845+0000 7fca3020f500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-08T08:37:14.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:14.846+0000 7fca3020f500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-12-08T08:37:17.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-12-08T08:37:17.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-08T08:37:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:17.819+0000 7f580841c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-12-08T08:37:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:17.820+0000 7f580841c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-12-08T08:37:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:17.820+0000 7f580841c500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-08T08:37:17.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:17.820+0000 7f580841c500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-08T08:37:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:17.820+0000 7f580841c500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-08T08:37:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:17.820+0000 7f580841c500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-12-08T08:37:20.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-12-08T08:37:20.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-08T08:37:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:20.797+0000 7fc93ce22500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-12-08T08:37:20.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:20.797+0000 7fc93ce22500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-12-08T08:37:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:20.797+0000 7fc93ce22500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-08T08:37:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:20.797+0000 7fc93ce22500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-08T08:37:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:20.797+0000 7fc93ce22500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-08T08:37:20.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:20.798+0000 7fc93ce22500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-12-08T08:37:23.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-12-08T08:37:23.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-08T08:37:23.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:23.772+0000 7fc80daf6500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-12-08T08:37:23.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:23.772+0000 7fc80daf6500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-12-08T08:37:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:23.772+0000 7fc80daf6500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-08T08:37:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:23.772+0000 7fc80daf6500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-08T08:37:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:23.772+0000 7fc80daf6500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-08T08:37:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:23.772+0000 7fc80daf6500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-12-08T08:37:26.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-12-08T08:37:26.656 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-12-08T08:37:26.688 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-08T08:37:26.688 INFO:tasks.ceph:role: osd.4 2023-12-08T08:37:26.688 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com 2023-12-08T08:37:26.688 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-08T08:37:26.766 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-08T08:37:26.766 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-08T08:37:26.766 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-08T08:37:26.766 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-08T08:37:26.766 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-08T08:37:26.766 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-12-08T08:37:26.766 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-08T08:37:26.766 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-08T08:37:26.766 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-08T08:37:26.766 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-08T08:37:27.277 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-12-08T08:37:27.279 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-12-08T08:37:27.279 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-12-08T08:37:27.414 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-12-08T08:37:27.452 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-12-08T08:37:27.521 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-08T08:37:27.521 INFO:tasks.ceph:role: osd.5 2023-12-08T08:37:27.521 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com 2023-12-08T08:37:27.521 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-08T08:37:27.598 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-08T08:37:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-08T08:37:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-08T08:37:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-08T08:37:27.598 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-08T08:37:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-12-08T08:37:27.598 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-08T08:37:27.598 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-08T08:37:27.598 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-08T08:37:27.598 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-08T08:37:28.044 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-12-08T08:37:28.046 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-12-08T08:37:28.046 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-12-08T08:37:28.139 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-12-08T08:37:28.169 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-12-08T08:37:28.236 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-08T08:37:28.236 INFO:tasks.ceph:role: osd.6 2023-12-08T08:37:28.236 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com 2023-12-08T08:37:28.236 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-08T08:37:28.310 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-08T08:37:28.310 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-08T08:37:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-08T08:37:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-08T08:37:28.311 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-08T08:37:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-12-08T08:37:28.311 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-08T08:37:28.311 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-08T08:37:28.311 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-08T08:37:28.311 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-08T08:37:28.803 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-12-08T08:37:28.805 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-12-08T08:37:28.806 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-12-08T08:37:28.959 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-12-08T08:37:28.990 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-12-08T08:37:29.057 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-08T08:37:29.057 INFO:tasks.ceph:role: osd.7 2023-12-08T08:37:29.057 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com 2023-12-08T08:37:29.057 DEBUG:teuthology.orchestra.run.smithi112:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-08T08:37:29.131 INFO:teuthology.orchestra.run.smithi112.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-08T08:37:29.132 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-08T08:37:29.132 INFO:teuthology.orchestra.run.smithi112.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-08T08:37:29.132 INFO:teuthology.orchestra.run.smithi112.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-08T08:37:29.132 INFO:teuthology.orchestra.run.smithi112.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-08T08:37:29.132 INFO:teuthology.orchestra.run.smithi112.stdout: = sunit=0 swidth=0 blks 2023-12-08T08:37:29.132 INFO:teuthology.orchestra.run.smithi112.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-08T08:37:29.132 INFO:teuthology.orchestra.run.smithi112.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-08T08:37:29.132 INFO:teuthology.orchestra.run.smithi112.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-08T08:37:29.132 INFO:teuthology.orchestra.run.smithi112.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-08T08:37:29.659 INFO:teuthology.orchestra.run.smithi112.stdout:Discarding blocks...Done. 2023-12-08T08:37:29.661 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi112.front.sepia.ceph.com -o noatime 2023-12-08T08:37:29.662 DEBUG:teuthology.orchestra.run.smithi112:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-12-08T08:37:29.802 DEBUG:teuthology.orchestra.run.smithi112:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-12-08T08:37:29.838 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-08T08:37:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:29.938+0000 7f1e33e23500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-12-08T08:37:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:29.938+0000 7f1e33e23500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-12-08T08:37:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:29.938+0000 7f1e33e23500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-08T08:37:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:29.938+0000 7f1e33e23500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-08T08:37:29.941 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:29.938+0000 7f1e33e23500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-08T08:37:29.942 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:29.939+0000 7f1e33e23500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-12-08T08:37:32.797 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-12-08T08:37:32.828 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-08T08:37:32.920 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:32.918+0000 7ff412623500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-12-08T08:37:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:32.918+0000 7ff412623500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-12-08T08:37:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:32.918+0000 7ff412623500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-08T08:37:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:32.918+0000 7ff412623500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-08T08:37:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:32.918+0000 7ff412623500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-08T08:37:32.921 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:32.919+0000 7ff412623500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-12-08T08:37:35.776 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-12-08T08:37:35.806 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-08T08:37:35.895 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:35.893+0000 7f78ae48a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-12-08T08:37:35.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:35.893+0000 7f78ae48a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-12-08T08:37:35.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:35.893+0000 7f78ae48a500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-08T08:37:35.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:35.893+0000 7f78ae48a500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-08T08:37:35.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:35.893+0000 7f78ae48a500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-08T08:37:35.896 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:35.893+0000 7f78ae48a500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-12-08T08:37:38.749 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-12-08T08:37:38.781 DEBUG:teuthology.orchestra.run.smithi112:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-08T08:37:38.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:38.867+0000 7f71ef614500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-12-08T08:37:38.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:38.867+0000 7f71ef614500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-12-08T08:37:38.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:38.867+0000 7f71ef614500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-08T08:37:38.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:38.867+0000 7f71ef614500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-08T08:37:38.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:38.867+0000 7f71ef614500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-08T08:37:38.870 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:38.868+0000 7f71ef614500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-12-08T08:37:40.147 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:37:40.150 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:37:41.722 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-12-08T08:37:41.749 INFO:tasks.ceph:Reading keys from all nodes... 2023-12-08T08:37:41.750 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:41.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-12-08T08:37:41.787 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:41.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-12-08T08:37:41.863 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:41.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-12-08T08:37:41.936 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:41.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-12-08T08:37:42.010 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:42.010 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-12-08T08:37:42.083 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:42.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-12-08T08:37:42.156 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:42.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-12-08T08:37:42.229 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:42.229 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-12-08T08:37:42.257 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:42.258 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-12-08T08:37:42.325 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:42.325 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-12-08T08:37:42.393 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:42.393 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-12-08T08:37:42.460 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:42.460 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-12-08T08:37:42.527 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:42.527 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-12-08T08:37:42.593 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:42.594 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-12-08T08:37:42.660 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:42.660 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-12-08T08:37:42.677 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:42.677 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-12-08T08:37:42.734 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:42.734 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-12-08T08:37:42.749 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:42.749 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-12-08T08:37:42.804 INFO:tasks.ceph:Adding keys to all mons... 2023-12-08T08:37:42.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-08T08:37:42.806 DEBUG:teuthology.orchestra.run.smithi112:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-08T08:37:42.847 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.y] 2023-12-08T08:37:42.847 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC41XJlcSdvJhAAXSnQAgTJ8oR3ZaObFc0fWw== 2023-12-08T08:37:42.848 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-12-08T08:37:42.848 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC41XJlEfknLhAAU0MfMMmtFok98FUnhfdwNg== 2023-12-08T08:37:42.848 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c] 2023-12-08T08:37:42.848 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC41XJleAE4NhAA6vzrXiYiWRXoRHnl8grXgA== 2023-12-08T08:37:42.848 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2023-12-08T08:37:42.848 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC61XJlChJvMhAA27OGl0JnI2oX15qX88jApw== 2023-12-08T08:37:42.848 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2023-12-08T08:37:42.848 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC91XJlST/oMBAA/jWxkH8dw+39wzBQJ2pBsQ== 2023-12-08T08:37:42.849 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2023-12-08T08:37:42.849 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDA1XJlEG+SLxAAgAN/6Fv/H7bMQHjlmUHgnQ== 2023-12-08T08:37:42.849 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2023-12-08T08:37:42.849 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDD1XJl4kQQLhAAsevNUXS8KlnqlhydabjVmw== 2023-12-08T08:37:42.849 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2023-12-08T08:37:42.849 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC41XJllUExKhAAqR4aRqeRBt4ibrlTNOD0xQ== 2023-12-08T08:37:42.849 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b] 2023-12-08T08:37:42.849 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC41XJl9kd7OxAACKRXjQrIJORGtWKvnIEB/A== 2023-12-08T08:37:42.849 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d] 2023-12-08T08:37:42.849 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC51XJlxKNJBxAAU7UWljdMrMl28PCh3lciDw== 2023-12-08T08:37:42.850 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.4] 2023-12-08T08:37:42.850 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDJ1XJlUSUBOBAAjg1B3YnxHRRfUyfjtVcsbA== 2023-12-08T08:37:42.850 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.5] 2023-12-08T08:37:42.850 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDM1XJlrpjLNhAAZ9K/AYfVZ6CEHwqDKED/+w== 2023-12-08T08:37:42.850 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.6] 2023-12-08T08:37:42.850 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDP1XJlGLJKNRAAP42GAoFBFBgKqP8646NM3w== 2023-12-08T08:37:42.850 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.7] 2023-12-08T08:37:42.850 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDS1XJlUZDEMxAAE5FgV0KgG4sKzIPFzbxKzA== 2023-12-08T08:37:42.851 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-12-08T08:37:42.851 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC51XJlZAHPCxAAZLN3jMh+Z9XRet/zdd6MzQ== 2023-12-08T08:37:42.851 INFO:teuthology.orchestra.run.smithi016.stdout:[client.2] 2023-12-08T08:37:42.851 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC51XJljiayDxAAetWjq+LR+1gYuybmNhrl7A== 2023-12-08T08:37:42.851 INFO:teuthology.orchestra.run.smithi016.stdout:[client.1] 2023-12-08T08:37:42.851 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC51XJllr1UExAAo9PcR6ozHGuR4DIMLF0vlg== 2023-12-08T08:37:42.851 INFO:teuthology.orchestra.run.smithi016.stdout:[client.3] 2023-12-08T08:37:42.851 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC51XJl2R4aGRAABYH/VhDDrsPHG9bbsDa52A== 2023-12-08T08:37:42.869 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.y] 2023-12-08T08:37:42.869 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC41XJlcSdvJhAAXSnQAgTJ8oR3ZaObFc0fWw== 2023-12-08T08:37:42.869 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.a] 2023-12-08T08:37:42.869 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC41XJlEfknLhAAU0MfMMmtFok98FUnhfdwNg== 2023-12-08T08:37:42.869 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.c] 2023-12-08T08:37:42.869 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC41XJleAE4NhAA6vzrXiYiWRXoRHnl8grXgA== 2023-12-08T08:37:42.869 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.0] 2023-12-08T08:37:42.869 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC61XJlChJvMhAA27OGl0JnI2oX15qX88jApw== 2023-12-08T08:37:42.869 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.1] 2023-12-08T08:37:42.869 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC91XJlST/oMBAA/jWxkH8dw+39wzBQJ2pBsQ== 2023-12-08T08:37:42.869 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.2] 2023-12-08T08:37:42.870 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDA1XJlEG+SLxAAgAN/6Fv/H7bMQHjlmUHgnQ== 2023-12-08T08:37:42.870 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.3] 2023-12-08T08:37:42.870 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDD1XJl4kQQLhAAsevNUXS8KlnqlhydabjVmw== 2023-12-08T08:37:42.870 INFO:teuthology.orchestra.run.smithi112.stdout:[mgr.x] 2023-12-08T08:37:42.870 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC41XJllUExKhAAqR4aRqeRBt4ibrlTNOD0xQ== 2023-12-08T08:37:42.870 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.b] 2023-12-08T08:37:42.870 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC41XJl9kd7OxAACKRXjQrIJORGtWKvnIEB/A== 2023-12-08T08:37:42.870 INFO:teuthology.orchestra.run.smithi112.stdout:[mds.d] 2023-12-08T08:37:42.870 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC51XJlxKNJBxAAU7UWljdMrMl28PCh3lciDw== 2023-12-08T08:37:42.870 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.4] 2023-12-08T08:37:42.870 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDJ1XJlUSUBOBAAjg1B3YnxHRRfUyfjtVcsbA== 2023-12-08T08:37:42.870 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.5] 2023-12-08T08:37:42.870 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDM1XJlrpjLNhAAZ9K/AYfVZ6CEHwqDKED/+w== 2023-12-08T08:37:42.871 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.6] 2023-12-08T08:37:42.871 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDP1XJlGLJKNRAAP42GAoFBFBgKqP8646NM3w== 2023-12-08T08:37:42.871 INFO:teuthology.orchestra.run.smithi112.stdout:[osd.7] 2023-12-08T08:37:42.871 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQDS1XJlUZDEMxAAE5FgV0KgG4sKzIPFzbxKzA== 2023-12-08T08:37:42.871 INFO:teuthology.orchestra.run.smithi112.stdout:[client.0] 2023-12-08T08:37:42.871 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC51XJlZAHPCxAAZLN3jMh+Z9XRet/zdd6MzQ== 2023-12-08T08:37:42.871 INFO:teuthology.orchestra.run.smithi112.stdout:[client.2] 2023-12-08T08:37:42.871 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC51XJljiayDxAAetWjq+LR+1gYuybmNhrl7A== 2023-12-08T08:37:42.871 INFO:teuthology.orchestra.run.smithi112.stdout:[client.1] 2023-12-08T08:37:42.871 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC51XJllr1UExAAo9PcR6ozHGuR4DIMLF0vlg== 2023-12-08T08:37:42.871 INFO:teuthology.orchestra.run.smithi112.stdout:[client.3] 2023-12-08T08:37:42.871 INFO:teuthology.orchestra.run.smithi112.stdout: key = AQC51XJl2R4aGRAABYH/VhDDrsPHG9bbsDa52A== 2023-12-08T08:37:42.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-08T08:37:42.895 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-08T08:37:42.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-08T08:37:42.966 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-08T08:37:43.024 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-08T08:37:43.026 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-08T08:37:43.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.085 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.139 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.141 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.198 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.253 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-08T08:37:43.310 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-08T08:37:43.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-08T08:37:43.366 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-08T08:37:43.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-08T08:37:43.425 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-08T08:37:43.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.484 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.539 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.541 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.597 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.652 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-08T08:37:43.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-08T08:37:43.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-08T08:37:43.765 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-08T08:37:43.767 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-08T08:37:43.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-08T08:37:43.825 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-08T08:37:43.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-08T08:37:43.884 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-08T08:37:43.940 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-12-08T08:37:43.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-12-08T08:37:43.971 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-08T08:37:44.072 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-12-08T08:37:44.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-12-08T08:37:44.130 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-08T08:37:44.226 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-12-08T08:37:44.253 DEBUG:teuthology.orchestra.run.smithi112:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-12-08T08:37:44.320 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-08T08:37:44.416 DEBUG:teuthology.orchestra.run.smithi112:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-12-08T08:37:44.443 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-08T08:37:44.445 DEBUG:teuthology.orchestra.run.smithi112:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-08T08:37:44.498 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-12-08T08:37:44.498 INFO:tasks.ceph.mon.a:Restarting daemon 2023-12-08T08:37:44.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-12-08T08:37:44.501 INFO:tasks.ceph.mon.a:Started 2023-12-08T08:37:44.501 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-08T08:37:44.501 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-12-08T08:37:44.540 INFO:tasks.ceph.mon.b:Started 2023-12-08T08:37:44.541 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-08T08:37:44.541 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-12-08T08:37:44.543 INFO:tasks.ceph.mon.c:Started 2023-12-08T08:37:44.543 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-12-08T08:37:44.543 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-12-08T08:37:44.543 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-12-08T08:37:44.545 INFO:tasks.ceph.mgr.y:Started 2023-12-08T08:37:44.545 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-12-08T08:37:44.545 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-12-08T08:37:44.547 INFO:tasks.ceph.mgr.x:Started 2023-12-08T08:37:44.547 INFO:tasks.ceph:Setting crush tunables to default 2023-12-08T08:37:44.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2023-12-08T08:37:44.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.651+0000 7fbefa0c2640 1 Processor -- start 2023-12-08T08:37:44.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.652+0000 7fbefa0c2640 1 -- start start 2023-12-08T08:37:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.652+0000 7fbefa0c2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.652+0000 7fbefa0c2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 0x7fbef40cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.652+0000 7fbefa0c2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 0x7fbef406ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.652+0000 7fbefa0c2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef406d030 con 0x7fbef40b4980 2023-12-08T08:37:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.652+0000 7fbefa0c2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef406d310 con 0x7fbef40cff00 2023-12-08T08:37:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.652+0000 7fbefa0c2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef406d5f0 con 0x7fbef40cf560 2023-12-08T08:37:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.652+0000 7fbef37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36530/0 (socket says 172.21.15.16:36530) 2023-12-08T08:37:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 learned_addr learned my addr 172.21.15.16:0/3867407712 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:37:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 0x7fbef406ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 0x7fbef40cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 msgr2=0x7fbef40cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-08T08:37:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 msgr2=0x7fbef40cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-08T08:37:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 msgr2=0x7fbef406ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-08T08:37:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 msgr2=0x7fbef406ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef2ffd640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 msgr2=0x7fbef40cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-08T08:37:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 0x7fbef406ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef2ffd640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 msgr2=0x7fbef40cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 0x7fbef40cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 0x7fbef406ddf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-08T08:37:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.653+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 0x7fbef40cf9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-08T08:37:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 0x7fbef406ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 msgr2=0x7fbef40cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-08T08:37:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 msgr2=0x7fbef40cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 0x7fbef40cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-08T08:37:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 msgr2=0x7fbef406ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-08T08:37:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 msgr2=0x7fbef406ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 0x7fbef406ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 0x7fbef406ddf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-08T08:37:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef2ffd640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 msgr2=0x7fbef40cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-08T08:37:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef2ffd640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 msgr2=0x7fbef40cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 0x7fbef40cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:44.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:44.854+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 0x7fbef40cf9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-08T08:37:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 msgr2=0x7fbef40cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-08T08:37:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 msgr2=0x7fbef40cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-08T08:37:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 0x7fbef406ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:45.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 0x7fbef40cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:45.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 msgr2=0x7fbef406ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-08T08:37:45.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 msgr2=0x7fbef406ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:45.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 0x7fbef406ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:45.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 0x7fbef406ddf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-08T08:37:45.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef2ffd640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 msgr2=0x7fbef40cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-08T08:37:45.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef2ffd640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 msgr2=0x7fbef40cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:45.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 0x7fbef40cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:45.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:45.255+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 0x7fbef40cf9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-08T08:37:46.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:46.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 msgr2=0x7fbef40cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-08T08:37:46.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 msgr2=0x7fbef40cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:46.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:46.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-08T08:37:46.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 0x7fbef406ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:46.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 0x7fbef40cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:46.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 msgr2=0x7fbef406ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-08T08:37:46.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 msgr2=0x7fbef406ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:46.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 0x7fbef406ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:46.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 0x7fbef406ddf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-08T08:37:46.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef2ffd640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 msgr2=0x7fbef40cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-08T08:37:46.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef2ffd640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 msgr2=0x7fbef40cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:46.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 0x7fbef40cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:46.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:46.056+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 0x7fbef40cf9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-08T08:37:47.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:47.653+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 msgr2=0x7fbef40cf9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-08T08:37:47.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:47.653+0000 7fbef17fa640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbef40cf560 0x7fbef40cf9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:47.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:47.653+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 msgr2=0x7fbef406ddf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-08T08:37:47.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:47.653+0000 7fbef17fa640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbef40cff00 0x7fbef406ddf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:47.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:47.653+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 msgr2=0x7fbef40cd8f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-08T08:37:47.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:47.653+0000 7fbef17fa640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbef40b4980 0x7fbef40cd8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:47.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:47.653+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef406d8d0 con 0x7fbedc007320 2023-12-08T08:37:47.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:47.653+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc00c070 con 0x7fbedc003670 2023-12-08T08:37:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 >> v1:172.21.15.112:6789/0 conn(0x7fbedc003670 legacy=0x7fbedc005af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 >> v1:172.21.15.16:6789/0 conn(0x7fbedc007320 legacy=0x7fbedc0097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:50.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 reap_dead start 2023-12-08T08:37:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef17fa640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbedc00c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef17fa640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 0x7fbedc005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc00d270 con 0x7fbedc006220 2023-12-08T08:37:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef406d030 con 0x7fbedc006600 2023-12-08T08:37:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef406d310 con 0x7fbedc00c3c0 2023-12-08T08:37:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 0x7fbedc005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbedc00c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 msgr2=0x7fbedc005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-08T08:37:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 msgr2=0x7fbedc005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 0x7fbedc005a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 0x7fbedc005a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-08T08:37:50.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 msgr2=0x7fbedc00c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-08T08:37:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 msgr2=0x7fbedc00c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbedc00c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:50.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.654+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbedc00c800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-08T08:37:50.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.855+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 0x7fbedc005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:50.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.855+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 msgr2=0x7fbedc005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-08T08:37:50.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.855+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 msgr2=0x7fbedc005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:50.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.855+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 0x7fbedc005a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:50.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.855+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 0x7fbedc005a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-08T08:37:50.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.855+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbedc00c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:50.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.855+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 msgr2=0x7fbedc00c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-08T08:37:50.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.855+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 msgr2=0x7fbedc00c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:50.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.855+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbedc00c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:50.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:50.855+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbedc00c800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-08T08:37:51.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:51.256+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 0x7fbedc005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:51.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:51.256+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 msgr2=0x7fbedc005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-08T08:37:51.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:51.256+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 msgr2=0x7fbedc005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:51.256+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 0x7fbedc005a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:51.256+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 0x7fbedc005a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-08T08:37:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:51.256+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbedc00c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:51.256+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 msgr2=0x7fbedc00c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-08T08:37:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:51.256+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 msgr2=0x7fbedc00c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:51.256+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbedc00c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:51.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:51.256+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbedc00c800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-08T08:37:52.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:52.058+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 0x7fbedc005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:52.058+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 msgr2=0x7fbedc005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-08T08:37:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:52.058+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 msgr2=0x7fbedc005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:52.058+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 0x7fbedc005a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:52.058+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 0x7fbedc005a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-08T08:37:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:52.058+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbedc00c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:52.058+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 msgr2=0x7fbedc00c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-08T08:37:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:52.058+0000 7fbef37fe640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 msgr2=0x7fbedc00c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:52.058+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbedc00c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:52.058+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbedc00c800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-08T08:37:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.654+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 msgr2=0x7fbedc00c800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-08T08:37:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.654+0000 7fbef17fa640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbedc00c800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.654+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 msgr2=0x7fbedc005a90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-08T08:37:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.654+0000 7fbef17fa640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.16:3300/0 conn(0x7fbedc006600 0x7fbedc005a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.654+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 >> v1:172.21.15.16:6789/0 conn(0x7fbedc006220 legacy=0x7fbedc0097c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.654+0000 7fbef17fa640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 0x7fbedc0145a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.654+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbedc006ab0 con 0x7fbedc006d90 2023-12-08T08:37:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.654+0000 7fbef17fa640 1 -- 172.21.15.16:0/3867407712 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef406d030 con 0x7fbedc00b800 2023-12-08T08:37:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.654+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 0x7fbedc0145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.654+0000 7fbef2ffd640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 msgr2=0x7fbedc0145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-08T08:37:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.654+0000 7fbef2ffd640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 msgr2=0x7fbedc0145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.654+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 0x7fbedc0145a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.654+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 0x7fbedc0145a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-08T08:37:53.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.855+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 0x7fbedc0145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:53.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.855+0000 7fbef2ffd640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 msgr2=0x7fbedc0145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-08T08:37:53.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.855+0000 7fbef2ffd640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 msgr2=0x7fbedc0145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:53.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.855+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 0x7fbedc0145a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:53.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:53.855+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 0x7fbedc0145a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-08T08:37:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.256+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 0x7fbedc0145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.256+0000 7fbef2ffd640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 msgr2=0x7fbedc0145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-08T08:37:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.256+0000 7fbef2ffd640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 msgr2=0x7fbedc0145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-08T08:37:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.256+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 0x7fbedc0145a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-08T08:37:54.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.256+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 0x7fbedc0145a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-08T08:37:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.878+0000 7fbef27fc640 1 -- 172.21.15.16:0/3867407712 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3297340853 0 0) 0x7fbedc006ab0 con 0x7fbedc006d90 2023-12-08T08:37:54.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.878+0000 7fbef27fc640 1 -- 172.21.15.16:0/3867407712 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbee0003520 con 0x7fbedc006d90 2023-12-08T08:37:54.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef27fc640 1 -- 172.21.15.16:0/3867407712 <== mon.1 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1919991369 0 0) 0x7fbee0003520 con 0x7fbedc006d90 2023-12-08T08:37:54.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef27fc640 1 -- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 msgr2=0x7fbedc0145a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-08T08:37:54.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef27fc640 1 --2- 172.21.15.16:0/3867407712 >> v2:172.21.15.112:3300/0 conn(0x7fbedc00b800 0x7fbedc0145a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:54.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef27fc640 1 -- 172.21.15.16:0/3867407712 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc00bd60 con 0x7fbedc006d90 2023-12-08T08:37:54.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef27fc640 1 -- 172.21.15.16:0/3867407712 <== mon.1 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7fbee40167e0 con 0x7fbedc006d90 2023-12-08T08:37:54.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef27fc640 1 -- 172.21.15.16:0/3867407712 >> v1:172.21.15.112:6789/0 conn(0x7fbedc006d90 legacy=0x7fbedc00b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:54.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 reap_dead start 2023-12-08T08:37:54.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef27fc640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee00039a0 0x7fbee0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:54.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef27fc640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbee00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:54.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef27fc640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee0004d50 0x7fbee00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:54.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef27fc640 1 -- 172.21.15.16:0/3867407712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef406d030 con 0x7fbee0004d50 2023-12-08T08:37:54.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef27fc640 1 -- 172.21.15.16:0/3867407712 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee000bbe0 con 0x7fbee00039a0 2023-12-08T08:37:54.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef27fc640 1 -- 172.21.15.16:0/3867407712 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee000bda0 con 0x7fbedc00c3c0 2023-12-08T08:37:54.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee0004d50 0x7fbee00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:54.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee00039a0 0x7fbee0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:54.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.885+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbee00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:54.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.886+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 msgr2=0x7fbee00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:37:54.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.886+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbee00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:54.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.886+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee00039a0 msgr2=0x7fbee0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:37:54.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.886+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee00039a0 0x7fbee0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:54.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.886+0000 7fbef3fff640 1 -- 172.21.15.16:0/3867407712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee0003520 con 0x7fbee0004d50 2023-12-08T08:37:54.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.886+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee00039a0 0x7fbee0003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:37:54.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.887+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee0004d50 0x7fbee00051d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbef4072aa0 tx=0x7fbee00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f7ff71c1699254 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:54.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.887+0000 7fbef27fc640 1 -- 172.21.15.16:0/3867407712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec031470 con 0x7fbee0004d50 2023-12-08T08:37:54.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.887+0000 7fbef27fc640 1 -- 172.21.15.16:0/3867407712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec011040 con 0x7fbee0004d50 2023-12-08T08:37:54.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.887+0000 7fbef27fc640 1 -- 172.21.15.16:0/3867407712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec031870 con 0x7fbee0004d50 2023-12-08T08:37:54.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.888+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3867407712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee0004d50 msgr2=0x7fbee00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:54.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.888+0000 7fbefa0c2640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee0004d50 0x7fbee00051d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fbef4072aa0 tx=0x7fbee00057f0 comp rx=0 tx=0).stop 2023-12-08T08:37:54.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.888+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3867407712 shutdown_connections 2023-12-08T08:37:54.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.888+0000 7fbefa0c2640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbee0004d50 0x7fbee00051d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:54.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.888+0000 7fbefa0c2640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbedc00c3c0 0x7fbee00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:54.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.888+0000 7fbefa0c2640 1 --2- 172.21.15.16:0/3867407712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee00039a0 0x7fbee0003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:54.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.888+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3867407712 >> 172.21.15.16:0/3867407712 conn(0x7fbef40b1a70 msgr2=0x7fbef40b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:54.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.888+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3867407712 shutdown_connections 2023-12-08T08:37:54.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.888+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3867407712 wait complete. 2023-12-08T08:37:54.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.889+0000 7fbefa0c2640 1 Processor -- start 2023-12-08T08:37:54.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.890+0000 7fbefa0c2640 1 -- start start 2023-12-08T08:37:54.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbefa0c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbedc00c3c0 0x7fbef4153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:54.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbefa0c2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee00039a0 0x7fbef40cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:54.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbefa0c2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbee0004d50 0x7fbef40ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:54.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbef37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbedc00c3c0 0x7fbef4153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:54.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbef37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbedc00c3c0 0x7fbef4153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57502/0 (socket says 172.21.15.16:57502) 2023-12-08T08:37:54.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbef37fe640 1 -- 172.21.15.16:0/3302162463 learned_addr learned my addr 172.21.15.16:0/3302162463 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:37:54.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3302162463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef4159990 con 0x7fbedc00c3c0 2023-12-08T08:37:54.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3302162463 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40c90f0 con 0x7fbee00039a0 2023-12-08T08:37:54.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3302162463 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40c9c70 con 0x7fbee0004d50 2023-12-08T08:37:54.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3302162463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee00039a0 0x7fbef40cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:54.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbef3fff640 1 --2- 172.21.15.16:0/3302162463 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbee0004d50 0x7fbef40ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:54.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbef37fe640 1 -- 172.21.15.16:0/3302162463 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbee0004d50 msgr2=0x7fbef40ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:54.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3302162463 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbee0004d50 0x7fbef40ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:54.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbef37fe640 1 -- 172.21.15.16:0/3302162463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee00039a0 msgr2=0x7fbef40cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:54.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3302162463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee00039a0 0x7fbef40cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:54.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbef37fe640 1 -- 172.21.15.16:0/3302162463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef4153bb0 con 0x7fbedc00c3c0 2023-12-08T08:37:54.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbef2ffd640 1 --2- 172.21.15.16:0/3302162463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee00039a0 0x7fbef40cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:37:54.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.891+0000 7fbef37fe640 1 --2- 172.21.15.16:0/3302162463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbedc00c3c0 0x7fbef4153670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbee4006a90 tx=0x7fbee4015730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:54.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.892+0000 7fbedbfff640 1 -- 172.21.15.16:0/3302162463 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee401e050 con 0x7fbedc00c3c0 2023-12-08T08:37:54.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.892+0000 7fbedbfff640 1 -- 172.21.15.16:0/3302162463 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee401f040 con 0x7fbedc00c3c0 2023-12-08T08:37:54.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.892+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3302162463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef40c9340 con 0x7fbedc00c3c0 2023-12-08T08:37:54.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.892+0000 7fbedbfff640 1 -- 172.21.15.16:0/3302162463 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee4004030 con 0x7fbedc00c3c0 2023-12-08T08:37:54.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.892+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3302162463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef40c9900 con 0x7fbedc00c3c0 2023-12-08T08:37:54.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.892+0000 7fbedbfff640 1 -- 172.21.15.16:0/3302162463 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbee4003610 con 0x7fbedc00c3c0 2023-12-08T08:37:54.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.892+0000 7fbedbfff640 1 -- 172.21.15.16:0/3302162463 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbee402d670 con 0x7fbedc00c3c0 2023-12-08T08:37:54.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.893+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3302162463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef4067ad0 con 0x7fbedc00c3c0 2023-12-08T08:37:54.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.896+0000 7fbedbfff640 1 -- 172.21.15.16:0/3302162463 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fbee4003a50 con 0x7fbedc00c3c0 2023-12-08T08:37:54.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.965+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3302162463 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fbef4067ad0 con 0x7fbedc00c3c0 2023-12-08T08:37:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.967+0000 7fbedbfff640 1 -- 172.21.15.16:0/3302162463 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbee4003bf0 con 0x7fbedc00c3c0 2023-12-08T08:37:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2023-12-08T08:37:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.968+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3302162463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbedc00c3c0 msgr2=0x7fbef4153670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.968+0000 7fbefa0c2640 1 --2- 172.21.15.16:0/3302162463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbedc00c3c0 0x7fbef4153670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbee4006a90 tx=0x7fbee4015730 comp rx=0 tx=0).stop 2023-12-08T08:37:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.968+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3302162463 shutdown_connections 2023-12-08T08:37:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.968+0000 7fbefa0c2640 1 --2- 172.21.15.16:0/3302162463 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbee0004d50 0x7fbef40ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.968+0000 7fbefa0c2640 1 --2- 172.21.15.16:0/3302162463 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbee00039a0 0x7fbef40cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.968+0000 7fbefa0c2640 1 --2- 172.21.15.16:0/3302162463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbedc00c3c0 0x7fbef4153670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.968+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3302162463 >> 172.21.15.16:0/3302162463 conn(0x7fbef40b1a70 msgr2=0x7fbef4142330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.968+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3302162463 shutdown_connections 2023-12-08T08:37:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:54.968+0000 7fbefa0c2640 1 -- 172.21.15.16:0/3302162463 wait complete. 2023-12-08T08:37:54.982 INFO:tasks.ceph:check_enable_crimson: False 2023-12-08T08:37:54.983 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-12-08T08:37:54.983 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:54.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-12-08T08:37:55.015 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:55.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-12-08T08:37:55.085 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:55.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-12-08T08:37:55.156 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:37:55.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-12-08T08:37:55.227 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:55.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-12-08T08:37:55.257 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:55.257 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-12-08T08:37:55.324 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:55.324 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-12-08T08:37:55.355 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:55.352+0000 7f898324c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-08T08:37:55.360 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:55.358+0000 7f091f230040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-08T08:37:55.387 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:37:55.387 DEBUG:teuthology.orchestra.run.smithi112:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-12-08T08:37:55.408 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:55.406+0000 7f898324c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-08T08:37:55.414 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:55.412+0000 7f091f230040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-08T08:37:55.440 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 1debed5b-b2a4-4603-84ce-36ea491b7332 0 2023-12-08T08:37:55.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.576+0000 7f8648905640 1 Processor -- start 2023-12-08T08:37:55.579 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.576+0000 7f8648905640 1 -- start start 2023-12-08T08:37:55.581 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f8648905640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86440737b0 0x7f864415af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f8648905640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86440b3f40 con 0x7f86440b2520 2023-12-08T08:37:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f8648905640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864415b490 con 0x7f8644157aa0 2023-12-08T08:37:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f8648905640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864415a500 con 0x7f86440737b0 2023-12-08T08:37:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f864259c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f8644157aa0 0x7f86440b3150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35936/0 (socket says 172.21.15.112:35936) 2023-12-08T08:37:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f864259c640 1 -- 172.21.15.112:0/3832927699 learned_addr learned my addr 172.21.15.112:0/3832927699 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-12-08T08:37:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f863bfff640 1 --2- 172.21.15.112:0/3832927699 >> v2:172.21.15.16:3300/0 conn(0x7f86440737b0 0x7f864415af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f8641d9b640 1 -- 172.21.15.112:0/3832927699 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556796963 0 0) 0x7f8634001110 con 0x7f8644157aa0 2023-12-08T08:37:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f8641d9b640 1 -- 172.21.15.112:0/3832927699 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8630005730 con 0x7f8644157aa0 2023-12-08T08:37:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f8641d9b640 1 -- 172.21.15.112:0/3832927699 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3713747679 0 0) 0x7f863c001490 con 0x7f86440b2520 2023-12-08T08:37:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f8641d9b640 1 -- 172.21.15.112:0/3832927699 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8634001110 con 0x7f86440b2520 2023-12-08T08:37:55.582 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f863bfff640 1 -- 172.21.15.112:0/3832927699 >> v1:172.21.15.112:6789/0 conn(0x7f8644157aa0 legacy=0x7f86440b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f8641d9b640 1 -- 172.21.15.112:0/3832927699 <== mon.1 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3474310515 0 0) 0x7f8630005730 con 0x7f8644157aa0 2023-12-08T08:37:55.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f863bfff640 1 -- 172.21.15.112:0/3832927699 >> v1:172.21.15.16:6789/0 conn(0x7f86440b2520 legacy=0x7f86440cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.577+0000 7f863bfff640 1 -- 172.21.15.112:0/3832927699 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864415aac0 con 0x7f86440737b0 2023-12-08T08:37:55.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.578+0000 7f863bfff640 1 --2- 172.21.15.112:0/3832927699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86440737b0 0x7f864415af50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f862c009720 tx=0x7f862c02b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99301abd65991bf4 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:55.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.578+0000 7f8641d9b640 1 -- 172.21.15.112:0/3832927699 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634001800 con 0x7f86440737b0 2023-12-08T08:37:55.583 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.578+0000 7f8641d9b640 1 -- 172.21.15.112:0/3832927699 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c031520 con 0x7f86440737b0 2023-12-08T08:37:55.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.578+0000 7f8641d9b640 1 -- 172.21.15.112:0/3832927699 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c0316b0 con 0x7f86440737b0 2023-12-08T08:37:55.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.579+0000 7f8648905640 1 -- 172.21.15.112:0/3832927699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86440737b0 msgr2=0x7f864415af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.579+0000 7f8648905640 1 --2- 172.21.15.112:0/3832927699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86440737b0 0x7f864415af50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f862c009720 tx=0x7f862c02b7c0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.579+0000 7f8648905640 1 -- 172.21.15.112:0/3832927699 shutdown_connections 2023-12-08T08:37:55.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.579+0000 7f8648905640 1 --2- 172.21.15.112:0/3832927699 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86440737b0 0x7f864415af50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.579+0000 7f8648905640 1 -- 172.21.15.112:0/3832927699 >> 172.21.15.112:0/3832927699 conn(0x7f86440cf4d0 msgr2=0x7f86440cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:55.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.579+0000 7f8648905640 1 -- 172.21.15.112:0/3832927699 shutdown_connections 2023-12-08T08:37:55.584 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.579+0000 7f8648905640 1 -- 172.21.15.112:0/3832927699 wait complete. 2023-12-08T08:37:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.580+0000 7f8648905640 1 Processor -- start 2023-12-08T08:37:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.580+0000 7f8648905640 1 -- start start 2023-12-08T08:37:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.581+0000 7f8648905640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86440737b0 0x7f864415e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.581+0000 7f8648905640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86440b2520 0x7f86440ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.581+0000 7f8648905640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8644157aa0 0x7f864414dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.581+0000 7f8648905640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86440caa70 con 0x7f8644157aa0 2023-12-08T08:37:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.581+0000 7f8648905640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86440cad40 con 0x7f86440b2520 2023-12-08T08:37:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.581+0000 7f8648905640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86440cb020 con 0x7f86440737b0 2023-12-08T08:37:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.581+0000 7f863bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86440b2520 0x7f86440ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.581+0000 7f863bfff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86440b2520 0x7f86440ca530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51642/0 (socket says 172.21.15.112:51642) 2023-12-08T08:37:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.581+0000 7f864259c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86440737b0 0x7f864415e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.581+0000 7f863bfff640 1 -- 172.21.15.112:0/949084047 learned_addr learned my addr 172.21.15.112:0/949084047 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-12-08T08:37:55.587 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.581+0000 7f8642d9d640 1 --2- 172.21.15.112:0/949084047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8644157aa0 0x7f864414dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.582+0000 7f863bfff640 1 -- 172.21.15.112:0/949084047 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86440737b0 msgr2=0x7f864415e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.582+0000 7f863bfff640 1 --2- 172.21.15.112:0/949084047 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86440737b0 0x7f864415e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.582+0000 7f863bfff640 1 -- 172.21.15.112:0/949084047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8644157aa0 msgr2=0x7f864414dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.582+0000 7f863bfff640 1 --2- 172.21.15.112:0/949084047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8644157aa0 0x7f864414dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.582+0000 7f863bfff640 1 -- 172.21.15.112:0/949084047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c009400 con 0x7f86440b2520 2023-12-08T08:37:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.582+0000 7f864259c640 1 --2- 172.21.15.112:0/949084047 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86440737b0 0x7f864415e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:37:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.582+0000 7f863bfff640 1 --2- 172.21.15.112:0/949084047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86440b2520 0x7f86440ca530 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f862c036040 tx=0x7f862c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.582+0000 7f863affd640 1 -- 172.21.15.112:0/949084047 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c0338e0 con 0x7f86440b2520 2023-12-08T08:37:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.582+0000 7f863affd640 1 -- 172.21.15.112:0/949084047 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c033a40 con 0x7f86440b2520 2023-12-08T08:37:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.582+0000 7f863affd640 1 -- 172.21.15.112:0/949084047 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c033dd0 con 0x7f86440b2520 2023-12-08T08:37:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.582+0000 7f8648905640 1 -- 172.21.15.112:0/949084047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86440cb300 con 0x7f86440b2520 2023-12-08T08:37:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.582+0000 7f8648905640 1 -- 172.21.15.112:0/949084047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864415b9b0 con 0x7f86440b2520 2023-12-08T08:37:55.588 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.582+0000 7f863affd640 1 -- 172.21.15.112:0/949084047 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f862c03a040 con 0x7f86440b2520 2023-12-08T08:37:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.582+0000 7f863affd640 1 -- 172.21.15.112:0/949084047 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f862c03c400 con 0x7f86440b2520 2023-12-08T08:37:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.583+0000 7f8648905640 1 -- 172.21.15.112:0/949084047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8644159f80 con 0x7f86440b2520 2023-12-08T08:37:55.589 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.585+0000 7f863affd640 1 -- 172.21.15.112:0/949084047 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f862c04d060 con 0x7f86440b2520 2023-12-08T08:37:55.658 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.655+0000 7f8648905640 1 -- 172.21.15.112:0/949084047 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1debed5b-b2a4-4603-84ce-36ea491b7332", "id": 0} v 0) v1 -- 0x7f8644067ad0 con 0x7f86440b2520 2023-12-08T08:37:55.660 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.658+0000 7f863affd640 1 -- 172.21.15.112:0/949084047 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1debed5b-b2a4-4603-84ce-36ea491b7332", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f862c043070 con 0x7f86440b2520 2023-12-08T08:37:55.661 INFO:teuthology.orchestra.run.smithi112.stdout:0 2023-12-08T08:37:55.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.659+0000 7f8648905640 1 -- 172.21.15.112:0/949084047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86440b2520 msgr2=0x7f86440ca530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.659+0000 7f8648905640 1 --2- 172.21.15.112:0/949084047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86440b2520 0x7f86440ca530 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f862c036040 tx=0x7f862c0046c0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.659+0000 7f8648905640 1 -- 172.21.15.112:0/949084047 shutdown_connections 2023-12-08T08:37:55.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.659+0000 7f8648905640 1 --2- 172.21.15.112:0/949084047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8644157aa0 0x7f864414dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.659+0000 7f8648905640 1 --2- 172.21.15.112:0/949084047 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86440b2520 0x7f86440ca530 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.659+0000 7f8648905640 1 --2- 172.21.15.112:0/949084047 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86440737b0 0x7f864415e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.659+0000 7f8648905640 1 -- 172.21.15.112:0/949084047 >> 172.21.15.112:0/949084047 conn(0x7f86440cf4d0 msgr2=0x7f86441529e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:55.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.659+0000 7f8648905640 1 -- 172.21.15.112:0/949084047 shutdown_connections 2023-12-08T08:37:55.662 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.659+0000 7f8648905640 1 -- 172.21.15.112:0/949084047 wait complete. 2023-12-08T08:37:55.672 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 33af8d04-7310-43c1-ac4b-205f09214b04 1 2023-12-08T08:37:55.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.769+0000 7f6dcf9e5640 1 Processor -- start 2023-12-08T08:37:55.772 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.769+0000 7f6dcf9e5640 1 -- start start 2023-12-08T08:37:55.775 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.770+0000 7f6dcf9e5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6dc806ac50 0x7f6dc80ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.770+0000 7f6dcf9e5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc80719a0 con 0x7f6dc806e590 2023-12-08T08:37:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.770+0000 7f6dcf9e5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc8071c80 con 0x7f6dc815d3e0 2023-12-08T08:37:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.770+0000 7f6dcf9e5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc8071f60 con 0x7f6dc806ac50 2023-12-08T08:37:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.770+0000 7f6dcd781640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f6dc815d3e0 0x7f6dc806a540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35946/0 (socket says 172.21.15.112:35946) 2023-12-08T08:37:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.770+0000 7f6dcd781640 1 -- 172.21.15.112:0/511752171 learned_addr learned my addr 172.21.15.112:0/511752171 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-12-08T08:37:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dccf80640 1 --2- 172.21.15.112:0/511752171 >> v2:172.21.15.16:3300/0 conn(0x7f6dc806ac50 0x7f6dc80ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 -- 172.21.15.112:0/511752171 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909682692 0 0) 0x7f6dc8071c80 con 0x7f6dc815d3e0 2023-12-08T08:37:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 -- 172.21.15.112:0/511752171 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6db4003520 con 0x7f6dc815d3e0 2023-12-08T08:37:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 -- 172.21.15.112:0/511752171 <== mon.1 v1:172.21.15.112:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 236448668 0 0) 0x7f6db4003520 con 0x7f6dc815d3e0 2023-12-08T08:37:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 -- 172.21.15.112:0/511752171 >> v2:172.21.15.16:3300/0 conn(0x7f6dc806ac50 msgr2=0x7f6dc80ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:37:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 --2- 172.21.15.112:0/511752171 >> v2:172.21.15.16:3300/0 conn(0x7f6dc806ac50 0x7f6dc80ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.776 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 -- 172.21.15.112:0/511752171 >> v1:172.21.15.16:6789/0 conn(0x7f6dc806e590 legacy=0x7f6dc806ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 -- 172.21.15.112:0/511752171 --> v1:172.21.15.112:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc8072240 con 0x7f6dc815d3e0 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 -- 172.21.15.112:0/511752171 <== mon.1 v1:172.21.15.112:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f6db8001670 con 0x7f6dc815d3e0 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 -- 172.21.15.112:0/511752171 >> v1:172.21.15.112:6789/0 conn(0x7f6dc815d3e0 legacy=0x7f6dc806a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 --2- 172.21.15.112:0/511752171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6db4003970 0x7f6db4003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 --2- 172.21.15.112:0/511752171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db4004370 0x7f6db4004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 --2- 172.21.15.112:0/511752171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6db40050e0 0x7f6db400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 -- 172.21.15.112:0/511752171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc8071f60 con 0x7f6db4004370 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 -- 172.21.15.112:0/511752171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db400c020 con 0x7f6db40050e0 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dbffff640 1 -- 172.21.15.112:0/511752171 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db400c1e0 con 0x7f6db4003970 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dcdf82640 1 --2- 172.21.15.112:0/511752171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6db40050e0 0x7f6db400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dcd781640 1 --2- 172.21.15.112:0/511752171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6db4003970 0x7f6db4003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dccf80640 1 --2- 172.21.15.112:0/511752171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db4004370 0x7f6db4004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dcdf82640 1 -- 172.21.15.112:0/511752171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6db4003970 msgr2=0x7f6db4003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dcdf82640 1 --2- 172.21.15.112:0/511752171 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6db4003970 0x7f6db4003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.777 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dcdf82640 1 -- 172.21.15.112:0/511752171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db4004370 msgr2=0x7f6db4004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:37:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dcdf82640 1 --2- 172.21.15.112:0/511752171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db4004370 0x7f6db4004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.771+0000 7f6dcdf82640 1 -- 172.21.15.112:0/511752171 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db4003520 con 0x7f6db40050e0 2023-12-08T08:37:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.772+0000 7f6dcdf82640 1 -- 172.21.15.112:0/511752171 reap_dead start 2023-12-08T08:37:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.772+0000 7f6dcdf82640 1 --2- 172.21.15.112:0/511752171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6db40050e0 0x7f6db400b980 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0001f60 tx=0x7f6db40057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4883239c114a714 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.772+0000 7f6dbffff640 1 -- 172.21.15.112:0/511752171 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0003b70 con 0x7f6db40050e0 2023-12-08T08:37:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.772+0000 7f6dbffff640 1 -- 172.21.15.112:0/511752171 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc0003cf0 con 0x7f6db40050e0 2023-12-08T08:37:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.772+0000 7f6dbffff640 1 -- 172.21.15.112:0/511752171 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc00052b0 con 0x7f6db40050e0 2023-12-08T08:37:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.773+0000 7f6dcf9e5640 1 -- 172.21.15.112:0/511752171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6db40050e0 msgr2=0x7f6db400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.773+0000 7f6dcf9e5640 1 --2- 172.21.15.112:0/511752171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6db40050e0 0x7f6db400b980 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0001f60 tx=0x7f6db40057f0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.773+0000 7f6dcf9e5640 1 -- 172.21.15.112:0/511752171 shutdown_connections 2023-12-08T08:37:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.773+0000 7f6dcf9e5640 1 --2- 172.21.15.112:0/511752171 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6db40050e0 0x7f6db400b980 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.773+0000 7f6dcf9e5640 1 -- 172.21.15.112:0/511752171 >> 172.21.15.112:0/511752171 conn(0x7f6dc80b1890 msgr2=0x7f6dc80b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.773+0000 7f6dcf9e5640 1 -- 172.21.15.112:0/511752171 shutdown_connections 2023-12-08T08:37:55.778 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.773+0000 7f6dcf9e5640 1 -- 172.21.15.112:0/511752171 wait complete. 2023-12-08T08:37:55.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.774+0000 7f6dcf9e5640 1 Processor -- start 2023-12-08T08:37:55.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.774+0000 7f6dcf9e5640 1 -- start start 2023-12-08T08:37:55.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dcf9e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db40050e0 0x7f6dc80b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dcf9e5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc80c6860 0x7f6dc80c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dcf9e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc814ab70 0x7f6dc80af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dcf9e5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc814d5d0 con 0x7f6db40050e0 2023-12-08T08:37:55.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dcf9e5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc80c8f20 con 0x7f6dc814ab70 2023-12-08T08:37:55.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dcf9e5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc814d7b0 con 0x7f6dc80c6860 2023-12-08T08:37:55.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dcdf82640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc814ab70 0x7f6dc80af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dccf80640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc80c6860 0x7f6dc80c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.781 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dcdf82640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc814ab70 0x7f6dc80af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51668/0 (socket says 172.21.15.112:51668) 2023-12-08T08:37:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dcdf82640 1 -- 172.21.15.112:0/1114180673 learned_addr learned my addr 172.21.15.112:0/1114180673 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-12-08T08:37:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dcd781640 1 --2- 172.21.15.112:0/1114180673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db40050e0 0x7f6dc80b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dccf80640 1 -- 172.21.15.112:0/1114180673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc814ab70 msgr2=0x7f6dc80af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dccf80640 1 --2- 172.21.15.112:0/1114180673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc814ab70 0x7f6dc80af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dccf80640 1 -- 172.21.15.112:0/1114180673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db40050e0 msgr2=0x7f6dc80b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dccf80640 1 --2- 172.21.15.112:0/1114180673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db40050e0 0x7f6dc80b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dccf80640 1 -- 172.21.15.112:0/1114180673 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc80c72f0 con 0x7f6dc80c6860 2023-12-08T08:37:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dcdf82640 1 --2- 172.21.15.112:0/1114180673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc814ab70 0x7f6dc80af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:37:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dccf80640 1 --2- 172.21.15.112:0/1114180673 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc80c6860 0x7f6dc80c6db0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4011960 tx=0x7f6dc4011d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dbdffb640 1 -- 172.21.15.112:0/1114180673 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400f9d0 con 0x7f6dc80c6860 2023-12-08T08:37:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dbdffb640 1 -- 172.21.15.112:0/1114180673 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc400fb30 con 0x7f6dc80c6860 2023-12-08T08:37:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dcf9e5640 1 -- 172.21.15.112:0/1114180673 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc814da50 con 0x7f6dc80c6860 2023-12-08T08:37:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.775+0000 7f6dbdffb640 1 -- 172.21.15.112:0/1114180673 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc400fea0 con 0x7f6dc80c6860 2023-12-08T08:37:55.782 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.776+0000 7f6dcf9e5640 1 -- 172.21.15.112:0/1114180673 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc80c8780 con 0x7f6dc80c6860 2023-12-08T08:37:55.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.776+0000 7f6dbdffb640 1 -- 172.21.15.112:0/1114180673 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6dc4014740 con 0x7f6dc80c6860 2023-12-08T08:37:55.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.776+0000 7f6dbdffb640 1 -- 172.21.15.112:0/1114180673 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f6dc401dad0 con 0x7f6dc80c6860 2023-12-08T08:37:55.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.776+0000 7f6dcf9e5640 1 -- 172.21.15.112:0/1114180673 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc8067ad0 con 0x7f6dc80c6860 2023-12-08T08:37:55.783 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.779+0000 7f6dbdffb640 1 -- 172.21.15.112:0/1114180673 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f6dc401dce0 con 0x7f6dc80c6860 2023-12-08T08:37:55.852 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.850+0000 7f6dcf9e5640 1 -- 172.21.15.112:0/1114180673 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "33af8d04-7310-43c1-ac4b-205f09214b04", "id": 1} v 0) v1 -- 0x7f6dc8074600 con 0x7f6dc80c6860 2023-12-08T08:37:55.855 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.852+0000 7f6dbdffb640 1 -- 172.21.15.112:0/1114180673 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "33af8d04-7310-43c1-ac4b-205f09214b04", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6dc4014b30 con 0x7f6dc80c6860 2023-12-08T08:37:55.857 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-12-08T08:37:55.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.853+0000 7f6dcf9e5640 1 -- 172.21.15.112:0/1114180673 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc80c6860 msgr2=0x7f6dc80c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.853+0000 7f6dcf9e5640 1 --2- 172.21.15.112:0/1114180673 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc80c6860 0x7f6dc80c6db0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4011960 tx=0x7f6dc4011d60 comp rx=0 tx=0).stop 2023-12-08T08:37:55.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.853+0000 7f6dcf9e5640 1 -- 172.21.15.112:0/1114180673 shutdown_connections 2023-12-08T08:37:55.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.853+0000 7f6dcf9e5640 1 --2- 172.21.15.112:0/1114180673 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6dc814ab70 0x7f6dc80af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.853+0000 7f6dcf9e5640 1 --2- 172.21.15.112:0/1114180673 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6dc80c6860 0x7f6dc80c6db0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.853+0000 7f6dcf9e5640 1 --2- 172.21.15.112:0/1114180673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6db40050e0 0x7f6dc80b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.853+0000 7f6dcf9e5640 1 -- 172.21.15.112:0/1114180673 >> 172.21.15.112:0/1114180673 conn(0x7f6dc80b1890 msgr2=0x7f6dc80cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:55.858 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.853+0000 7f6dcf9e5640 1 -- 172.21.15.112:0/1114180673 shutdown_connections 2023-12-08T08:37:55.859 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.853+0000 7f6dcf9e5640 1 -- 172.21.15.112:0/1114180673 wait complete. 2023-12-08T08:37:55.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 838af08b-fb26-4744-81ce-929407379290 2 2023-12-08T08:37:55.962 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:55.959+0000 7f898324c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-08T08:37:55.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.963+0000 7faa74c83640 1 Processor -- start 2023-12-08T08:37:55.966 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.963+0000 7faa74c83640 1 -- start start 2023-12-08T08:37:55.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.964+0000 7faa74c83640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa7015b000 0x7faa7006e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.964+0000 7faa74c83640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faa700acd50 0x7faa7006eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.964+0000 7faa74c83640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7006f0c0 con 0x7faa7006a4d0 2023-12-08T08:37:55.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.964+0000 7faa74c83640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7006aff0 con 0x7faa7015b000 2023-12-08T08:37:55.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.964+0000 7faa74c83640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa700cc770 con 0x7faa700acd50 2023-12-08T08:37:55.967 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.964+0000 7faa67fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faa700acd50 0x7faa7006eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.964+0000 7faa6ed9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7faa7006a4d0 0x7faa7006a8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35952/0 (socket says 172.21.15.112:35952) 2023-12-08T08:37:55.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.964+0000 7faa6ed9d640 1 -- 172.21.15.112:0/2733845289 learned_addr learned my addr 172.21.15.112:0/2733845289 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-12-08T08:37:55.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.964+0000 7faa6e59c640 1 --2- 172.21.15.112:0/2733845289 >> v2:172.21.15.16:3300/0 conn(0x7faa7015b000 0x7faa7006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.964+0000 7faa67fff640 1 -- 172.21.15.112:0/2733845289 >> v2:172.21.15.16:3300/0 conn(0x7faa7015b000 msgr2=0x7faa7006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:37:55.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.964+0000 7faa67fff640 1 --2- 172.21.15.112:0/2733845289 >> v2:172.21.15.16:3300/0 conn(0x7faa7015b000 0x7faa7006e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.964+0000 7faa67fff640 1 -- 172.21.15.112:0/2733845289 >> v1:172.21.15.112:6789/0 conn(0x7faa7006a4d0 legacy=0x7faa7006a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.964+0000 7faa67fff640 1 -- 172.21.15.112:0/2733845289 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa700cca50 con 0x7faa700acd50 2023-12-08T08:37:55.968 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.964+0000 7faa6e59c640 1 --2- 172.21.15.112:0/2733845289 >> v2:172.21.15.16:3300/0 conn(0x7faa7015b000 0x7faa7006e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:37:55.969 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.965+0000 7faa67fff640 1 --2- 172.21.15.112:0/2733845289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa700acd50 0x7faa7006eb80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faa7009e0a0 tx=0x7faa5802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=365fcb45bcb64f4f server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:55.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.965+0000 7faa6dd9b640 1 -- 172.21.15.112:0/2733845289 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58031830 con 0x7faa700acd50 2023-12-08T08:37:55.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.965+0000 7faa6dd9b640 1 -- 172.21.15.112:0/2733845289 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa58031990 con 0x7faa700acd50 2023-12-08T08:37:55.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.965+0000 7faa6dd9b640 1 -- 172.21.15.112:0/2733845289 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa58031d20 con 0x7faa700acd50 2023-12-08T08:37:55.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.966+0000 7faa74c83640 1 -- 172.21.15.112:0/2733845289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa700acd50 msgr2=0x7faa7006eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.966+0000 7faa74c83640 1 --2- 172.21.15.112:0/2733845289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa700acd50 0x7faa7006eb80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faa7009e0a0 tx=0x7faa5802b8d0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.966+0000 7faa74c83640 1 -- 172.21.15.112:0/2733845289 shutdown_connections 2023-12-08T08:37:55.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.966+0000 7faa74c83640 1 --2- 172.21.15.112:0/2733845289 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa700acd50 0x7faa7006eb80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.966+0000 7faa74c83640 1 --2- 172.21.15.112:0/2733845289 >> v2:172.21.15.16:3300/0 conn(0x7faa7015b000 0x7faa7006e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.966+0000 7faa74c83640 1 -- 172.21.15.112:0/2733845289 >> 172.21.15.112:0/2733845289 conn(0x7faa700b1890 msgr2=0x7faa700b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:55.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.966+0000 7faa74c83640 1 -- 172.21.15.112:0/2733845289 shutdown_connections 2023-12-08T08:37:55.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.966+0000 7faa74c83640 1 -- 172.21.15.112:0/2733845289 wait complete. 2023-12-08T08:37:55.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.967+0000 7faa74c83640 1 Processor -- start 2023-12-08T08:37:55.970 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.967+0000 7faa74c83640 1 -- start start 2023-12-08T08:37:55.972 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa74c83640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7006a4d0 0x7faa700b4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa74c83640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa700acd50 0x7faa7015c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa74c83640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faa7015b000 0x7faa700cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:55.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa74c83640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7015e3f0 con 0x7faa7006a4d0 2023-12-08T08:37:55.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa74c83640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7015c5a0 con 0x7faa700acd50 2023-12-08T08:37:55.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa74c83640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7015d3e0 con 0x7faa7015b000 2023-12-08T08:37:55.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa67fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa700acd50 0x7faa7015c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa6ed9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faa7015b000 0x7faa700cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa67fff640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa700acd50 0x7faa7015c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51684/0 (socket says 172.21.15.112:51684) 2023-12-08T08:37:55.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa67fff640 1 -- 172.21.15.112:0/3376835336 learned_addr learned my addr 172.21.15.112:0/3376835336 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-12-08T08:37:55.973 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa6e59c640 1 --2- 172.21.15.112:0/3376835336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7006a4d0 0x7faa700b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:55.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa67fff640 1 -- 172.21.15.112:0/3376835336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faa7015b000 msgr2=0x7faa700cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa67fff640 1 --2- 172.21.15.112:0/3376835336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faa7015b000 0x7faa700cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa67fff640 1 -- 172.21.15.112:0/3376835336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7006a4d0 msgr2=0x7faa700b4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:55.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa67fff640 1 --2- 172.21.15.112:0/3376835336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7006a4d0 0x7faa700b4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:55.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa67fff640 1 -- 172.21.15.112:0/3376835336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa58009400 con 0x7faa700acd50 2023-12-08T08:37:55.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.968+0000 7faa6ed9d640 1 --2- 172.21.15.112:0/3376835336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faa7015b000 0x7faa700cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:37:55.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.969+0000 7faa67fff640 1 --2- 172.21.15.112:0/3376835336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa700acd50 0x7faa7015c060 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faa58009860 tx=0x7faa580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:55.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.969+0000 7faa66ffd640 1 -- 172.21.15.112:0/3376835336 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5803baa0 con 0x7faa700acd50 2023-12-08T08:37:55.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.969+0000 7faa66ffd640 1 -- 172.21.15.112:0/3376835336 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5803bc20 con 0x7faa700acd50 2023-12-08T08:37:55.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.969+0000 7faa66ffd640 1 -- 172.21.15.112:0/3376835336 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa580433f0 con 0x7faa700acd50 2023-12-08T08:37:55.974 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.969+0000 7faa74c83640 1 -- 172.21.15.112:0/3376835336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7015c7d0 con 0x7faa700acd50 2023-12-08T08:37:55.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.969+0000 7faa74c83640 1 -- 172.21.15.112:0/3376835336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7015cd90 con 0x7faa700acd50 2023-12-08T08:37:55.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.969+0000 7faa66ffd640 1 -- 172.21.15.112:0/3376835336 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faa58043850 con 0x7faa700acd50 2023-12-08T08:37:55.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.969+0000 7faa66ffd640 1 -- 172.21.15.112:0/3376835336 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7faa5804cae0 con 0x7faa700acd50 2023-12-08T08:37:55.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.970+0000 7faa74c83640 1 -- 172.21.15.112:0/3376835336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa70067ad0 con 0x7faa700acd50 2023-12-08T08:37:55.975 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:55.971+0000 7f091f230040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-08T08:37:55.975 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:55.972+0000 7faa66ffd640 1 -- 172.21.15.112:0/3376835336 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7faa58051060 con 0x7faa700acd50 2023-12-08T08:37:56.016 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:56.013+0000 7f898324c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-08T08:37:56.026 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:56.024+0000 7f091f230040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-08T08:37:56.044 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.042+0000 7faa74c83640 1 -- 172.21.15.112:0/3376835336 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "838af08b-fb26-4744-81ce-929407379290", "id": 2} v 0) v1 -- 0x7faa700ad500 con 0x7faa700acd50 2023-12-08T08:37:56.047 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.045+0000 7faa66ffd640 1 -- 172.21.15.112:0/3376835336 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "838af08b-fb26-4744-81ce-929407379290", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7faa58035020 con 0x7faa700acd50 2023-12-08T08:37:56.047 INFO:teuthology.orchestra.run.smithi112.stdout:2 2023-12-08T08:37:56.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.045+0000 7faa74c83640 1 -- 172.21.15.112:0/3376835336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa700acd50 msgr2=0x7faa7015c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.045+0000 7faa74c83640 1 --2- 172.21.15.112:0/3376835336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa700acd50 0x7faa7015c060 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faa58009860 tx=0x7faa580046c0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.046+0000 7faa74c83640 1 -- 172.21.15.112:0/3376835336 shutdown_connections 2023-12-08T08:37:56.048 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.046+0000 7faa74c83640 1 --2- 172.21.15.112:0/3376835336 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faa7015b000 0x7faa700cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.046+0000 7faa74c83640 1 --2- 172.21.15.112:0/3376835336 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faa700acd50 0x7faa7015c060 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.046+0000 7faa74c83640 1 --2- 172.21.15.112:0/3376835336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa7006a4d0 0x7faa700b4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.046+0000 7faa74c83640 1 -- 172.21.15.112:0/3376835336 >> 172.21.15.112:0/3376835336 conn(0x7faa700b1890 msgr2=0x7faa700b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:56.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.046+0000 7faa74c83640 1 -- 172.21.15.112:0/3376835336 shutdown_connections 2023-12-08T08:37:56.049 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.046+0000 7faa74c83640 1 -- 172.21.15.112:0/3376835336 wait complete. 2023-12-08T08:37:56.058 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 137a9f13-952c-4bb3-b7fc-88a86457f7bd 3 2023-12-08T08:37:56.063 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:56.060+0000 7f898324c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-08T08:37:56.073 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:56.071+0000 7f091f230040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-08T08:37:56.119 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:56.116+0000 7f898324c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-08T08:37:56.129 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:56.127+0000 7f091f230040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-08T08:37:56.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.158+0000 7fbe8a223640 1 Processor -- start 2023-12-08T08:37:56.161 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.158+0000 7fbe8a223640 1 -- start start 2023-12-08T08:37:56.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.159+0000 7fbe8a223640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe8415e6a0 0x7fbe840ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.159+0000 7fbe8a223640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe840d1950 0x7fbe840cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.159+0000 7fbe8a223640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe840ce790 con 0x7fbe840cc9f0 2023-12-08T08:37:56.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.159+0000 7fbe8a223640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe840ceba0 con 0x7fbe840d1950 2023-12-08T08:37:56.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.159+0000 7fbe8a223640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840cee80 con 0x7fbe8415e6a0 2023-12-08T08:37:56.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.159+0000 7fbe837fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe8415e6a0 0x7fbe840ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.162 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.159+0000 7fbe83fff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fbe840cc9f0 0x7fbe840b3c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35954/0 (socket says 172.21.15.112:35954) 2023-12-08T08:37:56.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.159+0000 7fbe83fff640 1 -- 172.21.15.112:0/2415734368 learned_addr learned my addr 172.21.15.112:0/2415734368 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-12-08T08:37:56.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.159+0000 7fbe7bfff640 1 --2- 172.21.15.112:0/2415734368 >> v2:172.21.15.16:3300/0 conn(0x7fbe840d1950 0x7fbe840cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.159+0000 7fbe837fe640 1 -- 172.21.15.112:0/2415734368 >> v2:172.21.15.16:3300/0 conn(0x7fbe840d1950 msgr2=0x7fbe840cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:37:56.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.159+0000 7fbe837fe640 1 --2- 172.21.15.112:0/2415734368 >> v2:172.21.15.16:3300/0 conn(0x7fbe840d1950 0x7fbe840cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.159+0000 7fbe82ffd640 1 -- 172.21.15.112:0/2415734368 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3835974075 0 0) 0x7fbe840ce790 con 0x7fbe840cc9f0 2023-12-08T08:37:56.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.159+0000 7fbe837fe640 1 -- 172.21.15.112:0/2415734368 >> v1:172.21.15.112:6789/0 conn(0x7fbe840cc9f0 legacy=0x7fbe840b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.163 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.160+0000 7fbe837fe640 1 -- 172.21.15.112:0/2415734368 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe840cbcb0 con 0x7fbe8415e6a0 2023-12-08T08:37:56.163 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:56.160+0000 7f898324c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-08T08:37:56.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.160+0000 7fbe837fe640 1 --2- 172.21.15.112:0/2415734368 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe8415e6a0 0x7fbe840ce250 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe740097a0 tx=0x7fbe7402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c60d4bf40dce336 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:56.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.160+0000 7fbe82ffd640 1 -- 172.21.15.112:0/2415734368 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74031830 con 0x7fbe8415e6a0 2023-12-08T08:37:56.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.160+0000 7fbe82ffd640 1 -- 172.21.15.112:0/2415734368 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe74031990 con 0x7fbe8415e6a0 2023-12-08T08:37:56.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.160+0000 7fbe82ffd640 1 -- 172.21.15.112:0/2415734368 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74031d20 con 0x7fbe8415e6a0 2023-12-08T08:37:56.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.161+0000 7fbe8a223640 1 -- 172.21.15.112:0/2415734368 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe8415e6a0 msgr2=0x7fbe840ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.161+0000 7fbe8a223640 1 --2- 172.21.15.112:0/2415734368 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe8415e6a0 0x7fbe840ce250 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe740097a0 tx=0x7fbe7402b8d0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.161+0000 7fbe8a223640 1 -- 172.21.15.112:0/2415734368 shutdown_connections 2023-12-08T08:37:56.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.161+0000 7fbe8a223640 1 --2- 172.21.15.112:0/2415734368 >> v2:172.21.15.16:3300/0 conn(0x7fbe840d1950 0x7fbe840cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.161+0000 7fbe8a223640 1 --2- 172.21.15.112:0/2415734368 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe8415e6a0 0x7fbe840ce250 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.161+0000 7fbe8a223640 1 -- 172.21.15.112:0/2415734368 >> 172.21.15.112:0/2415734368 conn(0x7fbe840d0e30 msgr2=0x7fbe840d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:56.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.161+0000 7fbe8a223640 1 -- 172.21.15.112:0/2415734368 shutdown_connections 2023-12-08T08:37:56.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.161+0000 7fbe8a223640 1 -- 172.21.15.112:0/2415734368 wait complete. 2023-12-08T08:37:56.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.162+0000 7fbe8a223640 1 Processor -- start 2023-12-08T08:37:56.165 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.162+0000 7fbe8a223640 1 -- start start 2023-12-08T08:37:56.167 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe8a223640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe840cc9f0 0x7fbe841f3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe8a223640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe840d1950 0x7fbe8406bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe8a223640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe8415e6a0 0x7fbe8415c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe8a223640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe841f4350 con 0x7fbe840cc9f0 2023-12-08T08:37:56.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe8a223640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe841f6b10 con 0x7fbe8415e6a0 2023-12-08T08:37:56.168 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe8a223640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe841f3ab0 con 0x7fbe840d1950 2023-12-08T08:37:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe7bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe840d1950 0x7fbe8406bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe7bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe840d1950 0x7fbe8406bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:44360/0 (socket says 172.21.15.112:44360) 2023-12-08T08:37:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe7bfff640 1 -- 172.21.15.112:0/436881933 learned_addr learned my addr 172.21.15.112:0/436881933 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-12-08T08:37:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe83fff640 1 --2- 172.21.15.112:0/436881933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe8415e6a0 0x7fbe8415c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe837fe640 1 --2- 172.21.15.112:0/436881933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe840cc9f0 0x7fbe841f3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe7bfff640 1 -- 172.21.15.112:0/436881933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe8415e6a0 msgr2=0x7fbe8415c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe7bfff640 1 --2- 172.21.15.112:0/436881933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe8415e6a0 0x7fbe8415c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe7bfff640 1 -- 172.21.15.112:0/436881933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe840cc9f0 msgr2=0x7fbe841f3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe7bfff640 1 --2- 172.21.15.112:0/436881933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe840cc9f0 0x7fbe841f3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.163+0000 7fbe7bfff640 1 -- 172.21.15.112:0/436881933 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe74009400 con 0x7fbe840d1950 2023-12-08T08:37:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.164+0000 7fbe7bfff640 1 --2- 172.21.15.112:0/436881933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe840d1950 0x7fbe8406bff0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00e8a0 tx=0x7fbe6c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.164+0000 7fbe80ff9640 1 -- 172.21.15.112:0/436881933 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c0094f0 con 0x7fbe840d1950 2023-12-08T08:37:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.164+0000 7fbe80ff9640 1 -- 172.21.15.112:0/436881933 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6c00c940 con 0x7fbe840d1950 2023-12-08T08:37:56.169 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.164+0000 7fbe8a223640 1 -- 172.21.15.112:0/436881933 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe841f3d90 con 0x7fbe840d1950 2023-12-08T08:37:56.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.164+0000 7fbe80ff9640 1 -- 172.21.15.112:0/436881933 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c00ccf0 con 0x7fbe840d1950 2023-12-08T08:37:56.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.164+0000 7fbe8a223640 1 -- 172.21.15.112:0/436881933 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8415cde0 con 0x7fbe840d1950 2023-12-08T08:37:56.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.164+0000 7fbe80ff9640 1 -- 172.21.15.112:0/436881933 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbe6c005c90 con 0x7fbe840d1950 2023-12-08T08:37:56.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.164+0000 7fbe80ff9640 1 -- 172.21.15.112:0/436881933 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fbe6c0067a0 con 0x7fbe840d1950 2023-12-08T08:37:56.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.165+0000 7fbe8a223640 1 -- 172.21.15.112:0/436881933 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe84067ad0 con 0x7fbe840d1950 2023-12-08T08:37:56.170 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.167+0000 7fbe80ff9640 1 -- 172.21.15.112:0/436881933 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fbe6c021060 con 0x7fbe840d1950 2023-12-08T08:37:56.173 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:56.171+0000 7f091f230040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-08T08:37:56.239 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.237+0000 7fbe8a223640 1 -- 172.21.15.112:0/436881933 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd new", "uuid": "137a9f13-952c-4bb3-b7fc-88a86457f7bd", "id": 3} v 0) v1 -- 0x7fbe841f6c50 con 0x7fbe840d1950 2023-12-08T08:37:56.242 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.240+0000 7fbe80ff9640 1 -- 172.21.15.112:0/436881933 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "137a9f13-952c-4bb3-b7fc-88a86457f7bd", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbe6c029020 con 0x7fbe840d1950 2023-12-08T08:37:56.243 INFO:teuthology.orchestra.run.smithi112.stdout:3 2023-12-08T08:37:56.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.240+0000 7fbe8a223640 1 -- 172.21.15.112:0/436881933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe840d1950 msgr2=0x7fbe8406bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.240+0000 7fbe8a223640 1 --2- 172.21.15.112:0/436881933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe840d1950 0x7fbe8406bff0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c00e8a0 tx=0x7fbe6c00ed60 comp rx=0 tx=0).stop 2023-12-08T08:37:56.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.240+0000 7fbe8a223640 1 -- 172.21.15.112:0/436881933 shutdown_connections 2023-12-08T08:37:56.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.240+0000 7fbe8a223640 1 --2- 172.21.15.112:0/436881933 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbe8415e6a0 0x7fbe8415c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.240+0000 7fbe8a223640 1 --2- 172.21.15.112:0/436881933 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbe840d1950 0x7fbe8406bff0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.240+0000 7fbe8a223640 1 --2- 172.21.15.112:0/436881933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe840cc9f0 0x7fbe841f3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.240+0000 7fbe8a223640 1 -- 172.21.15.112:0/436881933 >> 172.21.15.112:0/436881933 conn(0x7fbe840d0e30 msgr2=0x7fbe840cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:56.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.241+0000 7fbe8a223640 1 -- 172.21.15.112:0/436881933 shutdown_connections 2023-12-08T08:37:56.244 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.241+0000 7fbe8a223640 1 -- 172.21.15.112:0/436881933 wait complete. 2023-12-08T08:37:56.253 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new f2d870f3-d72c-4943-bf09-a181ac9a70ec 4 2023-12-08T08:37:56.268 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:56.266+0000 7f898324c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-08T08:37:56.278 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:56.275+0000 7f091f230040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-08T08:37:56.341 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:56.338+0000 7f898324c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-08T08:37:56.350 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:56.348+0000 7f091f230040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-08T08:37:56.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.374+0000 7f9c62739640 1 Processor -- start 2023-12-08T08:37:56.376 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.374+0000 7f9c62739640 1 -- start start 2023-12-08T08:37:56.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.375+0000 7f9c62739640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c159c50 0x7f9c5c0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.375+0000 7f9c62739640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5c06e300 con 0x7f9c5c06f7d0 2023-12-08T08:37:56.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.375+0000 7f9c62739640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5c06e5e0 con 0x7f9c5c071130 2023-12-08T08:37:56.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.375+0000 7f9c62739640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c06e110 con 0x7f9c5c159c50 2023-12-08T08:37:56.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.375+0000 7f9c5bfff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c159c50 0x7f9c5c0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.375+0000 7f9c5b7fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f9c5c071130 0x7f9c5c06f0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35962/0 (socket says 172.21.15.112:35962) 2023-12-08T08:37:56.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.375+0000 7f9c5b7fe640 1 -- 172.21.15.112:0/2689908672 learned_addr learned my addr 172.21.15.112:0/2689908672 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-12-08T08:37:56.378 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.375+0000 7f9c5affd640 1 -- 172.21.15.112:0/2689908672 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871707955 0 0) 0x7f9c5c06e5e0 con 0x7f9c5c071130 2023-12-08T08:37:56.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.375+0000 7f9c5bfff640 1 -- 172.21.15.112:0/2689908672 >> v1:172.21.15.112:6789/0 conn(0x7f9c5c071130 legacy=0x7f9c5c06f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.375+0000 7f9c5bfff640 1 -- 172.21.15.112:0/2689908672 >> v1:172.21.15.16:6789/0 conn(0x7f9c5c06f7d0 legacy=0x7f9c5c0cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.379 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.375+0000 7f9c5bfff640 1 -- 172.21.15.112:0/2689908672 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c06e8c0 con 0x7f9c5c159c50 2023-12-08T08:37:56.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.377+0000 7f9c5bfff640 1 --2- 172.21.15.112:0/2689908672 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c159c50 0x7f9c5c0cd8a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0097d0 tx=0x7f9c4c02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=927b082d4cc20773 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:56.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.377+0000 7f9c5affd640 1 -- 172.21.15.112:0/2689908672 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c031890 con 0x7f9c5c159c50 2023-12-08T08:37:56.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.377+0000 7f9c5affd640 1 -- 172.21.15.112:0/2689908672 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c0319f0 con 0x7f9c5c159c50 2023-12-08T08:37:56.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.377+0000 7f9c5affd640 1 -- 172.21.15.112:0/2689908672 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c031d80 con 0x7f9c5c159c50 2023-12-08T08:37:56.380 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.377+0000 7f9c62739640 1 -- 172.21.15.112:0/2689908672 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c159c50 msgr2=0x7f9c5c0cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.377+0000 7f9c62739640 1 --2- 172.21.15.112:0/2689908672 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c159c50 0x7f9c5c0cd8a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0097d0 tx=0x7f9c4c02b930 comp rx=0 tx=0).stop 2023-12-08T08:37:56.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.378+0000 7f9c62739640 1 -- 172.21.15.112:0/2689908672 shutdown_connections 2023-12-08T08:37:56.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.378+0000 7f9c62739640 1 --2- 172.21.15.112:0/2689908672 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c159c50 0x7f9c5c0cd8a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.378+0000 7f9c62739640 1 -- 172.21.15.112:0/2689908672 >> 172.21.15.112:0/2689908672 conn(0x7f9c5c0cfcd0 msgr2=0x7f9c5c0d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:56.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.378+0000 7f9c62739640 1 -- 172.21.15.112:0/2689908672 shutdown_connections 2023-12-08T08:37:56.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.378+0000 7f9c62739640 1 -- 172.21.15.112:0/2689908672 wait complete. 2023-12-08T08:37:56.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.379+0000 7f9c62739640 1 Processor -- start 2023-12-08T08:37:56.381 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.379+0000 7f9c62739640 1 -- start start 2023-12-08T08:37:56.383 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c62739640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c06f7d0 0x7f9c5c14e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c62739640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c071130 0x7f9c5c0c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c62739640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c159c50 0x7f9c5c06a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c62739640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0c9200 con 0x7f9c5c06f7d0 2023-12-08T08:37:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c62739640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0c85f0 con 0x7f9c5c071130 2023-12-08T08:37:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c62739640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0c88d0 con 0x7f9c5c159c50 2023-12-08T08:37:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c5b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c071130 0x7f9c5c0c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c60cd6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c159c50 0x7f9c5c06a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c5b7fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c071130 0x7f9c5c0c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51706/0 (socket says 172.21.15.112:51706) 2023-12-08T08:37:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c5b7fe640 1 -- 172.21.15.112:0/3253347508 learned_addr learned my addr 172.21.15.112:0/3253347508 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-12-08T08:37:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c5bfff640 1 --2- 172.21.15.112:0/3253347508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c06f7d0 0x7f9c5c14e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c5b7fe640 1 -- 172.21.15.112:0/3253347508 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c159c50 msgr2=0x7f9c5c06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c5b7fe640 1 --2- 172.21.15.112:0/3253347508 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c159c50 0x7f9c5c06a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.384 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c5b7fe640 1 -- 172.21.15.112:0/3253347508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c06f7d0 msgr2=0x7f9c5c14e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c5b7fe640 1 --2- 172.21.15.112:0/3253347508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c06f7d0 0x7f9c5c14e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c5b7fe640 1 -- 172.21.15.112:0/3253347508 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c009400 con 0x7f9c5c071130 2023-12-08T08:37:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c60cd6640 1 --2- 172.21.15.112:0/3253347508 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c159c50 0x7f9c5c06a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:37:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c5b7fe640 1 --2- 172.21.15.112:0/3253347508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c071130 0x7f9c5c0c77b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000c4a0 tx=0x7f9c5000c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c58ff9640 1 -- 172.21.15.112:0/3253347508 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c500074e0 con 0x7f9c5c071130 2023-12-08T08:37:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c58ff9640 1 -- 172.21.15.112:0/3253347508 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c50007640 con 0x7f9c5c071130 2023-12-08T08:37:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c62739640 1 -- 172.21.15.112:0/3253347508 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c0c8bb0 con 0x7f9c5c071130 2023-12-08T08:37:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c58ff9640 1 -- 172.21.15.112:0/3253347508 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50004a00 con 0x7f9c5c071130 2023-12-08T08:37:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.380+0000 7f9c62739640 1 -- 172.21.15.112:0/3253347508 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c14e560 con 0x7f9c5c071130 2023-12-08T08:37:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.381+0000 7f9c58ff9640 1 -- 172.21.15.112:0/3253347508 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9c5001d440 con 0x7f9c5c071130 2023-12-08T08:37:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.381+0000 7f9c58ff9640 1 -- 172.21.15.112:0/3253347508 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f9c50012910 con 0x7f9c5c071130 2023-12-08T08:37:56.385 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.381+0000 7f9c62739640 1 -- 172.21.15.112:0/3253347508 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c067ad0 con 0x7f9c5c071130 2023-12-08T08:37:56.386 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.383+0000 7f9c58ff9640 1 -- 172.21.15.112:0/3253347508 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f9c50018070 con 0x7f9c5c071130 2023-12-08T08:37:56.389 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:56.386+0000 7f898324c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-08T08:37:56.400 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:56.398+0000 7f091f230040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-08T08:37:56.443 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:56.440+0000 7f898324c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-08T08:37:56.452 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:56.450+0000 7f091f230040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-08T08:37:56.455 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.453+0000 7f9c62739640 1 -- 172.21.15.112:0/3253347508 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f2d870f3-d72c-4943-bf09-a181ac9a70ec", "id": 4} v 0) v1 -- 0x7f9c5c0cef00 con 0x7f9c5c071130 2023-12-08T08:37:56.458 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.456+0000 7f9c58ff9640 1 -- 172.21.15.112:0/3253347508 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f2d870f3-d72c-4943-bf09-a181ac9a70ec", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9c50012ce0 con 0x7f9c5c071130 2023-12-08T08:37:56.459 INFO:teuthology.orchestra.run.smithi112.stdout:4 2023-12-08T08:37:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.457+0000 7f9c62739640 1 -- 172.21.15.112:0/3253347508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c071130 msgr2=0x7f9c5c0c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.457+0000 7f9c62739640 1 --2- 172.21.15.112:0/3253347508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c071130 0x7f9c5c0c77b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000c4a0 tx=0x7f9c5000c960 comp rx=0 tx=0).stop 2023-12-08T08:37:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.457+0000 7f9c62739640 1 -- 172.21.15.112:0/3253347508 shutdown_connections 2023-12-08T08:37:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.457+0000 7f9c62739640 1 --2- 172.21.15.112:0/3253347508 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c159c50 0x7f9c5c06a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.457+0000 7f9c62739640 1 --2- 172.21.15.112:0/3253347508 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c071130 0x7f9c5c0c77b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.457+0000 7f9c62739640 1 --2- 172.21.15.112:0/3253347508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c06f7d0 0x7f9c5c14e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.457+0000 7f9c62739640 1 -- 172.21.15.112:0/3253347508 >> 172.21.15.112:0/3253347508 conn(0x7f9c5c0cfcd0 msgr2=0x7f9c5c0ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.457+0000 7f9c62739640 1 -- 172.21.15.112:0/3253347508 shutdown_connections 2023-12-08T08:37:56.460 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.457+0000 7f9c62739640 1 -- 172.21.15.112:0/3253347508 wait complete. 2023-12-08T08:37:56.470 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 1229fda8-dfc6-48d3-99c1-3c8b8338aacf 5 2023-12-08T08:37:56.493 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:56.490+0000 7f898324c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-08T08:37:56.502 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:56.500+0000 7f091f230040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-08T08:37:56.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.566+0000 7fba89c2b640 1 Processor -- start 2023-12-08T08:37:56.568 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.566+0000 7fba89c2b640 1 -- start start 2023-12-08T08:37:56.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba89c2b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba8415d3e0 0x7fba840ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.570 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba89c2b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fba8406e590 0x7fba8406ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba89c2b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba840719a0 con 0x7fba8406a540 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba89c2b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba84071c80 con 0x7fba8406e590 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba89c2b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba84071f60 con 0x7fba8415d3e0 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba837fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba8415d3e0 0x7fba840ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba82ffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fba8406a540 0x7fba8406a9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35970/0 (socket says 172.21.15.112:35970) 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba837fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba8415d3e0 0x7fba840ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:44384/0 (socket says 172.21.15.112:44384) 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba82ffd640 1 -- 172.21.15.112:0/1659344254 learned_addr learned my addr 172.21.15.112:0/1659344254 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba83fff640 1 --2- 172.21.15.112:0/1659344254 >> v2:172.21.15.16:3300/0 conn(0x7fba8406e590 0x7fba8406ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba827fc640 1 -- 172.21.15.112:0/1659344254 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196611425 0 0) 0x7fba840719a0 con 0x7fba8406a540 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba837fe640 1 -- 172.21.15.112:0/1659344254 >> v2:172.21.15.16:3300/0 conn(0x7fba8406e590 msgr2=0x7fba8406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba837fe640 1 --2- 172.21.15.112:0/1659344254 >> v2:172.21.15.16:3300/0 conn(0x7fba8406e590 0x7fba8406ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba837fe640 1 -- 172.21.15.112:0/1659344254 >> v1:172.21.15.112:6789/0 conn(0x7fba8406a540 legacy=0x7fba8406a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.567+0000 7fba837fe640 1 -- 172.21.15.112:0/1659344254 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba84072240 con 0x7fba8415d3e0 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.568+0000 7fba837fe640 1 --2- 172.21.15.112:0/1659344254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba8415d3e0 0x7fba840ca330 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fba740097a0 tx=0x7fba7402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6352d2549793b80d server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:56.571 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.568+0000 7fba827fc640 1 -- 172.21.15.112:0/1659344254 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74031830 con 0x7fba8415d3e0 2023-12-08T08:37:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.568+0000 7fba827fc640 1 -- 172.21.15.112:0/1659344254 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba74031990 con 0x7fba8415d3e0 2023-12-08T08:37:56.572 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.568+0000 7fba827fc640 1 -- 172.21.15.112:0/1659344254 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74031d20 con 0x7fba8415d3e0 2023-12-08T08:37:56.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.569+0000 7fba89c2b640 1 -- 172.21.15.112:0/1659344254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba8415d3e0 msgr2=0x7fba840ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.569+0000 7fba89c2b640 1 --2- 172.21.15.112:0/1659344254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba8415d3e0 0x7fba840ca330 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fba740097a0 tx=0x7fba7402b8d0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.569+0000 7fba89c2b640 1 -- 172.21.15.112:0/1659344254 shutdown_connections 2023-12-08T08:37:56.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.569+0000 7fba89c2b640 1 --2- 172.21.15.112:0/1659344254 >> v2:172.21.15.16:3300/0 conn(0x7fba8406e590 0x7fba8406ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.569+0000 7fba89c2b640 1 --2- 172.21.15.112:0/1659344254 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba8415d3e0 0x7fba840ca330 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.569+0000 7fba89c2b640 1 -- 172.21.15.112:0/1659344254 >> 172.21.15.112:0/1659344254 conn(0x7fba840b1890 msgr2=0x7fba840b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:56.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.569+0000 7fba89c2b640 1 -- 172.21.15.112:0/1659344254 shutdown_connections 2023-12-08T08:37:56.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.569+0000 7fba89c2b640 1 -- 172.21.15.112:0/1659344254 wait complete. 2023-12-08T08:37:56.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.570+0000 7fba89c2b640 1 Processor -- start 2023-12-08T08:37:56.573 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.570+0000 7fba89c2b640 1 -- start start 2023-12-08T08:37:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba89c2b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba8406a540 0x7fba84151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba89c2b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8406e590 0x7fba840b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba89c2b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba8415d3e0 0x7fba840c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba89c2b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba840b4030 con 0x7fba8415d3e0 2023-12-08T08:37:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba89c2b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba840b4390 con 0x7fba8406e590 2023-12-08T08:37:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba89c2b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba840ae200 con 0x7fba8406a540 2023-12-08T08:37:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba837fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba8406a540 0x7fba84151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.576 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba837fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba8406a540 0x7fba84151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:44396/0 (socket says 172.21.15.112:44396) 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba82ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8406e590 0x7fba840b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba837fe640 1 -- 172.21.15.112:0/2405383191 learned_addr learned my addr 172.21.15.112:0/2405383191 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba83fff640 1 --2- 172.21.15.112:0/2405383191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba8415d3e0 0x7fba840c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba82ffd640 1 -- 172.21.15.112:0/2405383191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba8406a540 msgr2=0x7fba84151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba82ffd640 1 --2- 172.21.15.112:0/2405383191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba8406a540 0x7fba84151390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba82ffd640 1 -- 172.21.15.112:0/2405383191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba8415d3e0 msgr2=0x7fba840c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba82ffd640 1 --2- 172.21.15.112:0/2405383191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba8415d3e0 0x7fba840c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba82ffd640 1 -- 172.21.15.112:0/2405383191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba74009400 con 0x7fba8406e590 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba837fe640 1 --2- 172.21.15.112:0/2405383191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba8406a540 0x7fba84151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.571+0000 7fba82ffd640 1 --2- 172.21.15.112:0/2405383191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8406e590 0x7fba840b3af0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00c450 tx=0x7fba6c00c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.572+0000 7fba7bfff640 1 -- 172.21.15.112:0/2405383191 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0074e0 con 0x7fba8406e590 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.572+0000 7fba7bfff640 1 -- 172.21.15.112:0/2405383191 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c007640 con 0x7fba8406e590 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.572+0000 7fba7bfff640 1 -- 172.21.15.112:0/2405383191 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c004a20 con 0x7fba8406e590 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.572+0000 7fba89c2b640 1 -- 172.21.15.112:0/2405383191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba840ad960 con 0x7fba8406e590 2023-12-08T08:37:56.577 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.572+0000 7fba89c2b640 1 -- 172.21.15.112:0/2405383191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba840adf20 con 0x7fba8406e590 2023-12-08T08:37:56.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.572+0000 7fba7bfff640 1 -- 172.21.15.112:0/2405383191 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fba6c012400 con 0x7fba8406e590 2023-12-08T08:37:56.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.572+0000 7fba7bfff640 1 -- 172.21.15.112:0/2405383191 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fba6c01b8e0 con 0x7fba8406e590 2023-12-08T08:37:56.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.573+0000 7fba89c2b640 1 -- 172.21.15.112:0/2405383191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba84067ad0 con 0x7fba8406e590 2023-12-08T08:37:56.578 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.575+0000 7fba7bfff640 1 -- 172.21.15.112:0/2405383191 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fba6c018070 con 0x7fba8406e590 2023-12-08T08:37:56.647 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.644+0000 7fba89c2b640 1 -- 172.21.15.112:0/2405383191 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1229fda8-dfc6-48d3-99c1-3c8b8338aacf", "id": 5} v 0) v1 -- 0x7fba840b44d0 con 0x7fba8406e590 2023-12-08T08:37:56.649 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.647+0000 7fba7bfff640 1 -- 172.21.15.112:0/2405383191 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1229fda8-dfc6-48d3-99c1-3c8b8338aacf", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fba6c01bc70 con 0x7fba8406e590 2023-12-08T08:37:56.650 INFO:teuthology.orchestra.run.smithi112.stdout:5 2023-12-08T08:37:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.648+0000 7fba89c2b640 1 -- 172.21.15.112:0/2405383191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8406e590 msgr2=0x7fba840b3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.648+0000 7fba89c2b640 1 --2- 172.21.15.112:0/2405383191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8406e590 0x7fba840b3af0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00c450 tx=0x7fba6c00c910 comp rx=0 tx=0).stop 2023-12-08T08:37:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.648+0000 7fba89c2b640 1 -- 172.21.15.112:0/2405383191 shutdown_connections 2023-12-08T08:37:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.648+0000 7fba89c2b640 1 --2- 172.21.15.112:0/2405383191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba8415d3e0 0x7fba840c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.648+0000 7fba89c2b640 1 --2- 172.21.15.112:0/2405383191 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba8406e590 0x7fba840b3af0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.648+0000 7fba89c2b640 1 --2- 172.21.15.112:0/2405383191 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba8406a540 0x7fba84151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.648+0000 7fba89c2b640 1 -- 172.21.15.112:0/2405383191 >> 172.21.15.112:0/2405383191 conn(0x7fba840b1890 msgr2=0x7fba840af800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.648+0000 7fba89c2b640 1 -- 172.21.15.112:0/2405383191 shutdown_connections 2023-12-08T08:37:56.651 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.648+0000 7fba89c2b640 1 -- 172.21.15.112:0/2405383191 wait complete. 2023-12-08T08:37:56.661 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new 916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf 6 2023-12-08T08:37:56.801 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.799+0000 7f2489a1e640 1 Processor -- start 2023-12-08T08:37:56.802 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.799+0000 7f2489a1e640 1 -- start start 2023-12-08T08:37:56.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.800+0000 7f2489a1e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24840cfa80 0x7f24840d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.800+0000 7f2489a1e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24840ce690 0x7f24840cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.800+0000 7f2489a1e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24840b5ec0 con 0x7f24840d10f0 2023-12-08T08:37:56.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.800+0000 7f2489a1e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24840d03a0 con 0x7f24840cfa80 2023-12-08T08:37:56.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.800+0000 7f2489a1e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24840cede0 con 0x7f24840ce690 2023-12-08T08:37:56.804 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.800+0000 7f24827fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24840ce690 0x7f24840cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.800+0000 7f24827fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24840ce690 0x7f24840cfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:44410/0 (socket says 172.21.15.112:44410) 2023-12-08T08:37:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.800+0000 7f24827fc640 1 -- 172.21.15.112:0/2043172351 learned_addr learned my addr 172.21.15.112:0/2043172351 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-12-08T08:37:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.800+0000 7f2482ffd640 1 --2- 172.21.15.112:0/2043172351 >> v2:172.21.15.16:3300/0 conn(0x7f24840cfa80 0x7f24840d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.800+0000 7f24827fc640 1 -- 172.21.15.112:0/2043172351 >> v2:172.21.15.16:3300/0 conn(0x7f24840cfa80 msgr2=0x7f24840d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:37:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.800+0000 7f24827fc640 1 --2- 172.21.15.112:0/2043172351 >> v2:172.21.15.16:3300/0 conn(0x7f24840cfa80 0x7f24840d3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.800+0000 7f24827fc640 1 -- 172.21.15.112:0/2043172351 >> v1:172.21.15.16:6789/0 conn(0x7f24840d10f0 legacy=0x7f24840d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.800+0000 7f24827fc640 1 -- 172.21.15.112:0/2043172351 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24840b6110 con 0x7f24840ce690 2023-12-08T08:37:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.801+0000 7f24827fc640 1 --2- 172.21.15.112:0/2043172351 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24840ce690 0x7f24840cfe60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2474009870 tx=0x7f247402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=712c2e9cd2a18cb7 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.801+0000 7f2481ffb640 1 -- 172.21.15.112:0/2043172351 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474031640 con 0x7f24840ce690 2023-12-08T08:37:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.801+0000 7f2481ffb640 1 -- 172.21.15.112:0/2043172351 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24740317a0 con 0x7f24840ce690 2023-12-08T08:37:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.801+0000 7f2481ffb640 1 -- 172.21.15.112:0/2043172351 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474031b30 con 0x7f24840ce690 2023-12-08T08:37:56.805 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.802+0000 7f2489a1e640 1 -- 172.21.15.112:0/2043172351 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24840ce690 msgr2=0x7f24840cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.802+0000 7f2489a1e640 1 --2- 172.21.15.112:0/2043172351 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24840ce690 0x7f24840cfe60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2474009870 tx=0x7f247402b8f0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.802+0000 7f2489a1e640 1 -- 172.21.15.112:0/2043172351 shutdown_connections 2023-12-08T08:37:56.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.802+0000 7f2489a1e640 1 --2- 172.21.15.112:0/2043172351 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24840ce690 0x7f24840cfe60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.802+0000 7f2489a1e640 1 --2- 172.21.15.112:0/2043172351 >> v2:172.21.15.16:3300/0 conn(0x7f24840cfa80 0x7f24840d3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.802+0000 7f2489a1e640 1 -- 172.21.15.112:0/2043172351 >> 172.21.15.112:0/2043172351 conn(0x7f248409c340 msgr2=0x7f24840d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:56.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.802+0000 7f2489a1e640 1 -- 172.21.15.112:0/2043172351 shutdown_connections 2023-12-08T08:37:56.806 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.802+0000 7f2489a1e640 1 -- 172.21.15.112:0/2043172351 wait complete. 2023-12-08T08:37:56.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.803+0000 7f2489a1e640 1 Processor -- start 2023-12-08T08:37:56.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.803+0000 7f2489a1e640 1 -- start start 2023-12-08T08:37:56.808 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f2489a1e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24840ce690 0x7f2484148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f2489a1e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24840cfa80 0x7f24840c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f2489a1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24840d10f0 0x7f24840c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f2489a1e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248415c770 con 0x7f24840d10f0 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f2489a1e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24840c9600 con 0x7f24840cfa80 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f2489a1e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24840c98e0 con 0x7f24840ce690 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f2482ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24840ce690 0x7f2484148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f2482ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24840ce690 0x7f2484148590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:44414/0 (socket says 172.21.15.112:44414) 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f2482ffd640 1 -- 172.21.15.112:0/4292527778 learned_addr learned my addr 172.21.15.112:0/4292527778 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f24827fc640 1 --2- 172.21.15.112:0/4292527778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24840cfa80 0x7f24840c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f24837fe640 1 --2- 172.21.15.112:0/4292527778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24840d10f0 0x7f24840c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f24827fc640 1 -- 172.21.15.112:0/4292527778 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24840ce690 msgr2=0x7f2484148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f24827fc640 1 --2- 172.21.15.112:0/4292527778 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24840ce690 0x7f2484148590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f24827fc640 1 -- 172.21.15.112:0/4292527778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24840d10f0 msgr2=0x7f24840c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f24827fc640 1 --2- 172.21.15.112:0/4292527778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24840d10f0 0x7f24840c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.809 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f24827fc640 1 -- 172.21.15.112:0/4292527778 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24740094d0 con 0x7f24840cfa80 2023-12-08T08:37:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f2482ffd640 1 --2- 172.21.15.112:0/4292527778 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24840ce690 0x7f2484148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:37:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f24827fc640 1 --2- 172.21.15.112:0/4292527778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24840cfa80 0x7f24840c6ad0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2474033000 tx=0x7f24740046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.804+0000 7f247bfff640 1 -- 172.21.15.112:0/4292527778 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247403ac60 con 0x7f24840cfa80 2023-12-08T08:37:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.805+0000 7f247bfff640 1 -- 172.21.15.112:0/4292527778 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247403ade0 con 0x7f24840cfa80 2023-12-08T08:37:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.805+0000 7f2489a1e640 1 -- 172.21.15.112:0/4292527778 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24840c9bc0 con 0x7f24840cfa80 2023-12-08T08:37:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.805+0000 7f247bfff640 1 -- 172.21.15.112:0/4292527778 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474044630 con 0x7f24840cfa80 2023-12-08T08:37:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.805+0000 7f2489a1e640 1 -- 172.21.15.112:0/4292527778 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24840ca4f0 con 0x7f24840cfa80 2023-12-08T08:37:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.805+0000 7f247bfff640 1 -- 172.21.15.112:0/4292527778 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f247402f050 con 0x7f24840cfa80 2023-12-08T08:37:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.805+0000 7f247bfff640 1 -- 172.21.15.112:0/4292527778 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f2474030990 con 0x7f24840cfa80 2023-12-08T08:37:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.805+0000 7f2489a1e640 1 -- 172.21.15.112:0/4292527778 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2484067430 con 0x7f24840cfa80 2023-12-08T08:37:56.810 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.808+0000 7f247bfff640 1 -- 172.21.15.112:0/4292527778 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f2474035020 con 0x7f24840cfa80 2023-12-08T08:37:56.881 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.878+0000 7f2489a1e640 1 -- 172.21.15.112:0/4292527778 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf", "id": 6} v 0) v1 -- 0x7f24840d2e50 con 0x7f24840cfa80 2023-12-08T08:37:56.883 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.881+0000 7f247bfff640 1 -- 172.21.15.112:0/4292527778 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f247402d040 con 0x7f24840cfa80 2023-12-08T08:37:56.883 INFO:teuthology.orchestra.run.smithi112.stdout:6 2023-12-08T08:37:56.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.882+0000 7f2489a1e640 1 -- 172.21.15.112:0/4292527778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24840cfa80 msgr2=0x7f24840c6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.884 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.882+0000 7f2489a1e640 1 --2- 172.21.15.112:0/4292527778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24840cfa80 0x7f24840c6ad0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2474033000 tx=0x7f24740046c0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.882+0000 7f2489a1e640 1 -- 172.21.15.112:0/4292527778 shutdown_connections 2023-12-08T08:37:56.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.882+0000 7f2489a1e640 1 --2- 172.21.15.112:0/4292527778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24840d10f0 0x7f24840c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.882+0000 7f2489a1e640 1 --2- 172.21.15.112:0/4292527778 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f24840cfa80 0x7f24840c6ad0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.882+0000 7f2489a1e640 1 --2- 172.21.15.112:0/4292527778 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f24840ce690 0x7f2484148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.882+0000 7f2489a1e640 1 -- 172.21.15.112:0/4292527778 >> 172.21.15.112:0/4292527778 conn(0x7f248409c340 msgr2=0x7f248406c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:56.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.882+0000 7f2489a1e640 1 -- 172.21.15.112:0/4292527778 shutdown_connections 2023-12-08T08:37:56.885 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.882+0000 7f2489a1e640 1 -- 172.21.15.112:0/4292527778 wait complete. 2023-12-08T08:37:56.891 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:56.889+0000 7f898324c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-08T08:37:56.894 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --cluster ceph osd new a4052663-452c-4808-8fb5-11069b32cf37 7 2023-12-08T08:37:56.895 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:56.893+0000 7f091f230040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-08T08:37:56.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.991+0000 7f94c7a4f640 1 Processor -- start 2023-12-08T08:37:56.994 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.992+0000 7f94c7a4f640 1 -- start start 2023-12-08T08:37:56.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.992+0000 7f94c7a4f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f94c0146870 0x7f94c00aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.992+0000 7f94c7a4f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f94c015d640 0x7f94c006a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:56.996 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.992+0000 7f94c7a4f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c015dc00 con 0x7f94c00ad810 2023-12-08T08:37:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.993+0000 7f94c7a4f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c0160c80 con 0x7f94c015d640 2023-12-08T08:37:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.993+0000 7f94c7a4f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c015dee0 con 0x7f94c0146870 2023-12-08T08:37:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.993+0000 7f94c57eb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f94c0146870 0x7f94c00aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.993+0000 7f94c4fea640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f94c00ad810 0x7f94c00b2c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.112:35974/0 (socket says 172.21.15.112:35974) 2023-12-08T08:37:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.993+0000 7f94c4fea640 1 -- 172.21.15.112:0/39342739 learned_addr learned my addr 172.21.15.112:0/39342739 (peer_addr_for_me v1:172.21.15.112:0/0) 2023-12-08T08:37:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.993+0000 7f94c57eb640 1 --2- 172.21.15.112:0/39342739 >> v2:172.21.15.112:3300/0 conn(0x7f94c0146870 0x7f94c00aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.112:51724/0 (socket says 172.21.15.112:51724) 2023-12-08T08:37:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.993+0000 7f94c5fec640 1 --2- 172.21.15.112:0/39342739 >> v2:172.21.15.16:3300/0 conn(0x7f94c015d640 0x7f94c006a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.993+0000 7f94c57eb640 1 -- 172.21.15.112:0/39342739 >> v2:172.21.15.16:3300/0 conn(0x7f94c015d640 msgr2=0x7f94c006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:37:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.993+0000 7f94c57eb640 1 --2- 172.21.15.112:0/39342739 >> v2:172.21.15.16:3300/0 conn(0x7f94c015d640 0x7f94c006a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.993+0000 7f94c57eb640 1 -- 172.21.15.112:0/39342739 >> v1:172.21.15.112:6789/0 conn(0x7f94c00ad810 legacy=0x7f94c00b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.993+0000 7f94b3fff640 1 -- 172.21.15.112:0/39342739 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2172240551 0 0) 0x7f94c015dc00 con 0x7f94c00ad810 2023-12-08T08:37:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.993+0000 7f94c57eb640 1 -- 172.21.15.112:0/39342739 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c015e1c0 con 0x7f94c0146870 2023-12-08T08:37:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.993+0000 7f94c5fec640 1 --2- 172.21.15.112:0/39342739 >> v2:172.21.15.16:3300/0 conn(0x7f94c015d640 0x7f94c006a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:37:56.997 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.994+0000 7f94c57eb640 1 --2- 172.21.15.112:0/39342739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94c0146870 0x7f94c00aeec0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f94c006d090 tx=0x7f94b402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30242de86df748e server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:56.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.994+0000 7f94b3fff640 1 -- 172.21.15.112:0/39342739 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4031830 con 0x7f94c0146870 2023-12-08T08:37:56.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.994+0000 7f94b3fff640 1 -- 172.21.15.112:0/39342739 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b4031990 con 0x7f94c0146870 2023-12-08T08:37:56.998 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.994+0000 7f94b3fff640 1 -- 172.21.15.112:0/39342739 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4031d20 con 0x7f94c0146870 2023-12-08T08:37:57.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.994+0000 7f94c7a4f640 1 -- 172.21.15.112:0/39342739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94c0146870 msgr2=0x7f94c00aeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:57.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.994+0000 7f94c7a4f640 1 --2- 172.21.15.112:0/39342739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94c0146870 0x7f94c00aeec0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f94c006d090 tx=0x7f94b402b8d0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.995+0000 7f94c7a4f640 1 -- 172.21.15.112:0/39342739 shutdown_connections 2023-12-08T08:37:57.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.995+0000 7f94c7a4f640 1 --2- 172.21.15.112:0/39342739 >> v2:172.21.15.16:3300/0 conn(0x7f94c015d640 0x7f94c006a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.995+0000 7f94c7a4f640 1 --2- 172.21.15.112:0/39342739 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94c0146870 0x7f94c00aeec0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.001 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.995+0000 7f94c7a4f640 1 -- 172.21.15.112:0/39342739 >> 172.21.15.112:0/39342739 conn(0x7f94c00b1970 msgr2=0x7f94c00b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:57.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.995+0000 7f94c7a4f640 1 -- 172.21.15.112:0/39342739 shutdown_connections 2023-12-08T08:37:57.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.995+0000 7f94c7a4f640 1 -- 172.21.15.112:0/39342739 wait complete. 2023-12-08T08:37:57.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.996+0000 7f94c7a4f640 1 Processor -- start 2023-12-08T08:37:57.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.996+0000 7f94c7a4f640 1 -- start start 2023-12-08T08:37:57.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.996+0000 7f94c7a4f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94c00ad810 0x7f94c0143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:57.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.996+0000 7f94c7a4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c0146870 0x7f94c01f6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:57.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.996+0000 7f94c7a4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94c015d640 0x7f94c01f6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:57.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.996+0000 7f94c7a4f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c01f3020 con 0x7f94c0146870 2023-12-08T08:37:57.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.996+0000 7f94c7a4f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c00cd740 con 0x7f94c015d640 2023-12-08T08:37:57.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.996+0000 7f94c7a4f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c01f7010 con 0x7f94c00ad810 2023-12-08T08:37:57.002 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94c57eb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94c00ad810 0x7f94c0143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:57.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94c5fec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94c015d640 0x7f94c01f6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:57.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94c57eb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94c00ad810 0x7f94c0143ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.112:44418/0 (socket says 172.21.15.112:44418) 2023-12-08T08:37:57.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94c57eb640 1 -- 172.21.15.112:0/3102036197 learned_addr learned my addr 172.21.15.112:0/3102036197 (peer_addr_for_me v2:172.21.15.112:0/0) 2023-12-08T08:37:57.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94c4fea640 1 --2- 172.21.15.112:0/3102036197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c0146870 0x7f94c01f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:57.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94c5fec640 1 -- 172.21.15.112:0/3102036197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94c00ad810 msgr2=0x7f94c0143ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:57.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94c5fec640 1 --2- 172.21.15.112:0/3102036197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94c00ad810 0x7f94c0143ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94c5fec640 1 -- 172.21.15.112:0/3102036197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c0146870 msgr2=0x7f94c01f6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:57.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94c5fec640 1 --2- 172.21.15.112:0/3102036197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c0146870 0x7f94c01f6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94c5fec640 1 -- 172.21.15.112:0/3102036197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b4009400 con 0x7f94c015d640 2023-12-08T08:37:57.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94c57eb640 1 --2- 172.21.15.112:0/3102036197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94c00ad810 0x7f94c0143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:37:57.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94c4fea640 1 --2- 172.21.15.112:0/3102036197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c0146870 0x7f94c01f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:37:57.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94c5fec640 1 --2- 172.21.15.112:0/3102036197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94c015d640 0x7f94c01f6ad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f94b8007d40 tx=0x7f94b800d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:57.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94b1ffb640 1 -- 172.21.15.112:0/3102036197 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8003990 con 0x7f94c015d640 2023-12-08T08:37:57.003 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94b1ffb640 1 -- 172.21.15.112:0/3102036197 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b800f040 con 0x7f94c015d640 2023-12-08T08:37:57.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94b1ffb640 1 -- 172.21.15.112:0/3102036197 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8003d30 con 0x7f94c015d640 2023-12-08T08:37:57.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94c7a4f640 1 -- 172.21.15.112:0/3102036197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c01f72a0 con 0x7f94c015d640 2023-12-08T08:37:57.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.997+0000 7f94b1ffb640 1 -- 172.21.15.112:0/3102036197 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f94b8004330 con 0x7f94c015d640 2023-12-08T08:37:57.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.998+0000 7f94c7a4f640 1 -- 172.21.15.112:0/3102036197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c00cdc30 con 0x7f94c015d640 2023-12-08T08:37:57.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.998+0000 7f94b1ffb640 1 -- 172.21.15.112:0/3102036197 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f94b8015440 con 0x7f94c015d640 2023-12-08T08:37:57.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:56.998+0000 7f94c7a4f640 1 -- 172.21.15.112:0/3102036197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c0067ad0 con 0x7f94c015d640 2023-12-08T08:37:57.004 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:57.000+0000 7f94b1ffb640 1 -- 172.21.15.112:0/3102036197 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f94b801a070 con 0x7f94c015d640 2023-12-08T08:37:57.033 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:57.031+0000 7f898324c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-08T08:37:57.036 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:57.034+0000 7f091f230040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-08T08:37:57.073 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:57.070+0000 7f94c7a4f640 1 -- 172.21.15.112:0/3102036197 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a4052663-452c-4808-8fb5-11069b32cf37", "id": 7} v 0) v1 -- 0x7f94c006b240 con 0x7f94c015d640 2023-12-08T08:37:57.078 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:57.076+0000 7f94b1ffb640 1 -- 172.21.15.112:0/3102036197 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a4052663-452c-4808-8fb5-11069b32cf37", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f94b8015940 con 0x7f94c015d640 2023-12-08T08:37:57.078 INFO:teuthology.orchestra.run.smithi112.stdout:7 2023-12-08T08:37:57.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:57.077+0000 7f94c7a4f640 1 -- 172.21.15.112:0/3102036197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94c015d640 msgr2=0x7f94c01f6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:57.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:57.077+0000 7f94c7a4f640 1 --2- 172.21.15.112:0/3102036197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94c015d640 0x7f94c01f6ad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f94b8007d40 tx=0x7f94b800d4a0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:57.077+0000 7f94c7a4f640 1 -- 172.21.15.112:0/3102036197 shutdown_connections 2023-12-08T08:37:57.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:57.077+0000 7f94c7a4f640 1 --2- 172.21.15.112:0/3102036197 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94c015d640 0x7f94c01f6ad0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:57.077+0000 7f94c7a4f640 1 --2- 172.21.15.112:0/3102036197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94c0146870 0x7f94c01f6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:57.077+0000 7f94c7a4f640 1 --2- 172.21.15.112:0/3102036197 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94c00ad810 0x7f94c0143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:57.077+0000 7f94c7a4f640 1 -- 172.21.15.112:0/3102036197 >> 172.21.15.112:0/3102036197 conn(0x7f94c00b1970 msgr2=0x7f94c01458a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:57.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:57.077+0000 7f94c7a4f640 1 -- 172.21.15.112:0/3102036197 shutdown_connections 2023-12-08T08:37:57.080 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:37:57.077+0000 7f94c7a4f640 1 -- 172.21.15.112:0/3102036197 wait complete. 2023-12-08T08:37:57.090 INFO:tasks.ceph.osd.0:Restarting daemon 2023-12-08T08:37:57.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-12-08T08:37:57.092 INFO:tasks.ceph.osd.0:Started 2023-12-08T08:37:57.092 INFO:tasks.ceph.osd.1:Restarting daemon 2023-12-08T08:37:57.092 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-12-08T08:37:57.094 INFO:tasks.ceph.osd.1:Started 2023-12-08T08:37:57.094 INFO:tasks.ceph.osd.2:Restarting daemon 2023-12-08T08:37:57.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-12-08T08:37:57.096 INFO:tasks.ceph.osd.2:Started 2023-12-08T08:37:57.096 INFO:tasks.ceph.osd.3:Restarting daemon 2023-12-08T08:37:57.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-12-08T08:37:57.099 INFO:tasks.ceph.osd.3:Started 2023-12-08T08:37:57.099 INFO:tasks.ceph.osd.4:Restarting daemon 2023-12-08T08:37:57.099 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-12-08T08:37:57.101 INFO:tasks.ceph.osd.4:Started 2023-12-08T08:37:57.101 INFO:tasks.ceph.osd.5:Restarting daemon 2023-12-08T08:37:57.101 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-12-08T08:37:57.103 INFO:tasks.ceph.osd.5:Started 2023-12-08T08:37:57.103 INFO:tasks.ceph.osd.6:Restarting daemon 2023-12-08T08:37:57.104 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-12-08T08:37:57.105 INFO:tasks.ceph.osd.6:Started 2023-12-08T08:37:57.106 INFO:tasks.ceph.osd.7:Restarting daemon 2023-12-08T08:37:57.106 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-12-08T08:37:57.108 INFO:tasks.ceph.osd.7:Started 2023-12-08T08:37:57.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:37:57.162 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:57.159+0000 7f091f230040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-08T08:37:57.184 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:57.181+0000 7f898324c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-08T08:37:57.222 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:57.220+0000 7f091f230040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-08T08:37:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.236+0000 7faf7e090640 1 Processor -- start 2023-12-08T08:37:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.236+0000 7faf7e090640 1 -- start start 2023-12-08T08:37:57.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.237+0000 7faf7e090640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faf7806ccd0 0x7faf780cd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:57.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.237+0000 7faf7e090640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf780cec60 0x7faf780b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:57.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.237+0000 7faf7e090640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf78159b00 con 0x7faf7806e900 2023-12-08T08:37:57.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.237+0000 7faf7e090640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf780ce840 con 0x7faf7806ccd0 2023-12-08T08:37:57.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.237+0000 7faf7e090640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf780cdbc0 con 0x7faf780cec60 2023-12-08T08:37:57.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.238+0000 7faf76ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faf7806ccd0 0x7faf780cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:57.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.238+0000 7faf77fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf780cec60 0x7faf780b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:57.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.238+0000 7faf777fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7faf7806e900 0x7faf7806ece0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:52474/0 (socket says 172.21.15.16:52474) 2023-12-08T08:37:57.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.238+0000 7faf777fe640 1 -- 172.21.15.16:0/2763687058 learned_addr learned my addr 172.21.15.16:0/2763687058 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:37:57.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.238+0000 7faf767fc640 1 -- 172.21.15.16:0/2763687058 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769416597 0 0) 0x7faf78159b00 con 0x7faf7806e900 2023-12-08T08:37:57.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.238+0000 7faf76ffd640 1 -- 172.21.15.16:0/2763687058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf780cec60 msgr2=0x7faf780b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:37:57.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.238+0000 7faf76ffd640 1 --2- 172.21.15.16:0/2763687058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf780cec60 0x7faf780b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.238+0000 7faf76ffd640 1 -- 172.21.15.16:0/2763687058 >> v1:172.21.15.112:6789/0 conn(0x7faf7806e900 legacy=0x7faf7806ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:57.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.238+0000 7faf76ffd640 1 -- 172.21.15.16:0/2763687058 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf780cdea0 con 0x7faf7806ccd0 2023-12-08T08:37:57.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.238+0000 7faf77fff640 1 --2- 172.21.15.16:0/2763687058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf780cec60 0x7faf780b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:37:57.244 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:57.238+0000 7f898324c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-08T08:37:57.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.239+0000 7faf76ffd640 1 --2- 172.21.15.16:0/2763687058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7806ccd0 0x7faf780cd500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0097a0 tx=0x7faf6c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22b2a0deeff502b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:57.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.239+0000 7faf767fc640 1 -- 172.21.15.16:0/2763687058 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c02e070 con 0x7faf7806ccd0 2023-12-08T08:37:57.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.239+0000 7faf767fc640 1 -- 172.21.15.16:0/2763687058 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6c0315a0 con 0x7faf7806ccd0 2023-12-08T08:37:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.239+0000 7faf767fc640 1 -- 172.21.15.16:0/2763687058 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c031930 con 0x7faf7806ccd0 2023-12-08T08:37:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.240+0000 7faf7e090640 1 -- 172.21.15.16:0/2763687058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7806ccd0 msgr2=0x7faf780cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.240+0000 7faf7e090640 1 --2- 172.21.15.16:0/2763687058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7806ccd0 0x7faf780cd500 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7faf6c0097a0 tx=0x7faf6c02b840 comp rx=0 tx=0).stop 2023-12-08T08:37:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.240+0000 7faf7e090640 1 -- 172.21.15.16:0/2763687058 shutdown_connections 2023-12-08T08:37:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.240+0000 7faf7e090640 1 --2- 172.21.15.16:0/2763687058 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf780cec60 0x7faf780b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.240+0000 7faf7e090640 1 --2- 172.21.15.16:0/2763687058 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf7806ccd0 0x7faf780cd500 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.240+0000 7faf7e090640 1 -- 172.21.15.16:0/2763687058 >> 172.21.15.16:0/2763687058 conn(0x7faf78071850 msgr2=0x7faf7806a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.240+0000 7faf7e090640 1 -- 172.21.15.16:0/2763687058 shutdown_connections 2023-12-08T08:37:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.240+0000 7faf7e090640 1 -- 172.21.15.16:0/2763687058 wait complete. 2023-12-08T08:37:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.241+0000 7faf7e090640 1 Processor -- start 2023-12-08T08:37:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.241+0000 7faf7e090640 1 -- start start 2023-12-08T08:37:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.242+0000 7faf7e090640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7806ccd0 0x7faf78149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.242+0000 7faf7e090640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf7806e900 0x7faf780adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.242+0000 7faf7e090640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf780cec60 0x7faf780af9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.242+0000 7faf7e090640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf780a94c0 con 0x7faf7806ccd0 2023-12-08T08:37:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.242+0000 7faf7e090640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf780a9760 con 0x7faf780cec60 2023-12-08T08:37:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.242+0000 7faf7e090640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf780a9a40 con 0x7faf7806e900 2023-12-08T08:37:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.242+0000 7faf777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7806ccd0 0x7faf78149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.242+0000 7faf777fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7806ccd0 0x7faf78149320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34090/0 (socket says 172.21.15.16:34090) 2023-12-08T08:37:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.242+0000 7faf777fe640 1 -- 172.21.15.16:0/1373353416 learned_addr learned my addr 172.21.15.16:0/1373353416 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:37:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.242+0000 7faf76ffd640 1 --2- 172.21.15.16:0/1373353416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf7806e900 0x7faf780adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.243+0000 7faf77fff640 1 --2- 172.21.15.16:0/1373353416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf780cec60 0x7faf780af9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.243+0000 7faf777fe640 1 -- 172.21.15.16:0/1373353416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf7806e900 msgr2=0x7faf780adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.243+0000 7faf777fe640 1 --2- 172.21.15.16:0/1373353416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf7806e900 0x7faf780adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.243+0000 7faf777fe640 1 -- 172.21.15.16:0/1373353416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf780cec60 msgr2=0x7faf780af9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:57.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.243+0000 7faf777fe640 1 --2- 172.21.15.16:0/1373353416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf780cec60 0x7faf780af9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.243+0000 7faf777fe640 1 -- 172.21.15.16:0/1373353416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7000bc80 con 0x7faf7806ccd0 2023-12-08T08:37:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.243+0000 7faf777fe640 1 --2- 172.21.15.16:0/1373353416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7806ccd0 0x7faf78149320 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faf6800c430 tx=0x7faf6800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.243+0000 7faf5ffff640 1 -- 172.21.15.16:0/1373353416 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf680074e0 con 0x7faf7806ccd0 2023-12-08T08:37:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.243+0000 7faf5ffff640 1 -- 172.21.15.16:0/1373353416 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf68007660 con 0x7faf7806ccd0 2023-12-08T08:37:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.243+0000 7faf5ffff640 1 -- 172.21.15.16:0/1373353416 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf68004a20 con 0x7faf7806ccd0 2023-12-08T08:37:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.243+0000 7faf7e090640 1 -- 172.21.15.16:0/1373353416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf6c009400 con 0x7faf7806ccd0 2023-12-08T08:37:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.243+0000 7faf7e090640 1 -- 172.21.15.16:0/1373353416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf780aa000 con 0x7faf7806ccd0 2023-12-08T08:37:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.243+0000 7faf5ffff640 1 -- 172.21.15.16:0/1373353416 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faf680133c0 con 0x7faf7806ccd0 2023-12-08T08:37:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.244+0000 7faf5ffff640 1 -- 172.21.15.16:0/1373353416 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7faf68013d40 con 0x7faf7806ccd0 2023-12-08T08:37:57.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.244+0000 7faf7e090640 1 -- 172.21.15.16:0/1373353416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf780aefc0 con 0x7faf7806ccd0 2023-12-08T08:37:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.247+0000 7faf5ffff640 1 -- 172.21.15.16:0/1373353416 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7faf68013670 con 0x7faf7806ccd0 2023-12-08T08:37:57.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.308+0000 7faf7e090640 1 -- 172.21.15.16:0/1373353416 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf78067ad0 con 0x7faf7806ccd0 2023-12-08T08:37:57.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.308+0000 7faf5ffff640 1 -- 172.21.15.16:0/1373353416 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7faf68013810 con 0x7faf7806ccd0 2023-12-08T08:37:57.311 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:37:57.311 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:37:57.072680+0000","last_up_change":"0.000000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:37:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.309+0000 7faf7e090640 1 -- 172.21.15.16:0/1373353416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7806ccd0 msgr2=0x7faf78149320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.309+0000 7faf7e090640 1 --2- 172.21.15.16:0/1373353416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7806ccd0 0x7faf78149320 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7faf6800c430 tx=0x7faf6800c8f0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.309+0000 7faf7e090640 1 -- 172.21.15.16:0/1373353416 shutdown_connections 2023-12-08T08:37:57.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.309+0000 7faf7e090640 1 --2- 172.21.15.16:0/1373353416 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf780cec60 0x7faf780af9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.309+0000 7faf7e090640 1 --2- 172.21.15.16:0/1373353416 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf7806e900 0x7faf780adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.309+0000 7faf7e090640 1 --2- 172.21.15.16:0/1373353416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7806ccd0 0x7faf78149320 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:57.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.309+0000 7faf7e090640 1 -- 172.21.15.16:0/1373353416 >> 172.21.15.16:0/1373353416 conn(0x7faf78071850 msgr2=0x7faf78158bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:57.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.309+0000 7faf7e090640 1 -- 172.21.15.16:0/1373353416 shutdown_connections 2023-12-08T08:37:57.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:37:57.309+0000 7faf7e090640 1 -- 172.21.15.16:0/1373353416 wait complete. 2023-12-08T08:37:57.319 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:57.317+0000 7f091f230040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-08T08:37:57.323 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-12-08T08:37:57.324 INFO:tasks.ceph:Waiting for OSDs to come up 2023-12-08T08:37:57.345 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:57.342+0000 7f898324c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-08T08:37:57.411 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:57.409+0000 7f091f230040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-08T08:37:57.443 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:57.441+0000 7f898324c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-08T08:37:57.551 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:57.548+0000 7f091f230040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-08T08:37:57.589 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:57.586+0000 7f898324c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-08T08:37:57.592 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:57.590+0000 7f091f230040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-08T08:37:57.633 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:57.630+0000 7f898324c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-08T08:37:57.704 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T08:37:57.701+0000 7fb33079b500 -1 Falling back to public interface 2023-12-08T08:37:57.706 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T08:37:57.703+0000 7eff92222500 -1 Falling back to public interface 2023-12-08T08:37:57.707 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T08:37:57.705+0000 7f7fa0412500 -1 Falling back to public interface 2023-12-08T08:37:57.720 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T08:37:57.718+0000 7f147da11500 -1 Falling back to public interface 2023-12-08T08:37:57.721 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T08:37:57.718+0000 7fdb11010500 -1 Falling back to public interface 2023-12-08T08:37:57.725 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T08:37:57.723+0000 7ff2542fa500 -1 Falling back to public interface 2023-12-08T08:37:57.726 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T08:37:57.723+0000 7f6d12820500 -1 Falling back to public interface 2023-12-08T08:37:57.726 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T08:37:57.723+0000 7ff2532a3640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2023-12-08T08:37:57.730 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T08:37:57.728+0000 7f0ae6012500 -1 Falling back to public interface 2023-12-08T08:37:57.733 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:57.730+0000 7f091f230040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-08T08:37:57.784 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:57.781+0000 7f898324c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-08T08:37:58.131 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-08T08:37:58.224 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.221+0000 7f8247bd7640 1 Processor -- start 2023-12-08T08:37:58.225 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.221+0000 7f8247bd7640 1 -- start start 2023-12-08T08:37:58.226 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.222+0000 7f8247bd7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82400b4750 0x7f8240143130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:58.227 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.222+0000 7f8247bd7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8240140210 0x7f82400d13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:58.227 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.222+0000 7f8247bd7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82400b74b0 con 0x7f82400b4b30 2023-12-08T08:37:58.227 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.222+0000 7f8247bd7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82400b7750 con 0x7f8240140210 2023-12-08T08:37:58.227 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.222+0000 7f8247bd7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82400b7a30 con 0x7f82400b4750 2023-12-08T08:37:58.227 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.222+0000 7f8246174640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8240140210 0x7f82400d13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:58.227 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.222+0000 7f8245973640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82400b4750 0x7f8240143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:58.227 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.222+0000 7f8245172640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f82400b4b30 0x7f824013fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:52476/0 (socket says 172.21.15.16:52476) 2023-12-08T08:37:58.227 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.222+0000 7f8245172640 1 -- 172.21.15.16:0/2818528350 learned_addr learned my addr 172.21.15.16:0/2818528350 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:37:58.227 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.223+0000 7f8246174640 1 -- 172.21.15.16:0/2818528350 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82400b4750 msgr2=0x7f8240143130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:37:58.227 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.223+0000 7f8246174640 1 --2- 172.21.15.16:0/2818528350 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82400b4750 0x7f8240143130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:58.227 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.223+0000 7f8246174640 1 -- 172.21.15.16:0/2818528350 >> v1:172.21.15.112:6789/0 conn(0x7f82400b4b30 legacy=0x7f824013fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:58.227 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.223+0000 7f8246174640 1 -- 172.21.15.16:0/2818528350 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82400b7d10 con 0x7f8240140210 2023-12-08T08:37:58.227 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.223+0000 7f8244971640 1 -- 172.21.15.16:0/2818528350 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260138683 0 0) 0x7f82400b74b0 con 0x7f82400b4b30 2023-12-08T08:37:58.228 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.223+0000 7f8245973640 1 --2- 172.21.15.16:0/2818528350 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82400b4750 0x7f8240143130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:37:58.228 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.224+0000 7f8246174640 1 --2- 172.21.15.16:0/2818528350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8240140210 0x7f82400d13c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8238009d90 tx=0x7f823802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb37d689b10c27c server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:58.228 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.224+0000 7f8244971640 1 -- 172.21.15.16:0/2818528350 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823802cc50 con 0x7f8240140210 2023-12-08T08:37:58.228 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.224+0000 7f8244971640 1 -- 172.21.15.16:0/2818528350 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823802cdb0 con 0x7f8240140210 2023-12-08T08:37:58.228 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.224+0000 7f8244971640 1 -- 172.21.15.16:0/2818528350 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238034660 con 0x7f8240140210 2023-12-08T08:37:58.233 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.224+0000 7f8247bd7640 1 -- 172.21.15.16:0/2818528350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8240140210 msgr2=0x7f82400d13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:58.233 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.224+0000 7f8247bd7640 1 --2- 172.21.15.16:0/2818528350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8240140210 0x7f82400d13c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8238009d90 tx=0x7f823802bea0 comp rx=0 tx=0).stop 2023-12-08T08:37:58.233 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.224+0000 7f8247bd7640 1 -- 172.21.15.16:0/2818528350 shutdown_connections 2023-12-08T08:37:58.233 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.224+0000 7f8247bd7640 1 --2- 172.21.15.16:0/2818528350 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8240140210 0x7f82400d13c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:58.233 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.224+0000 7f8247bd7640 1 --2- 172.21.15.16:0/2818528350 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f82400b4750 0x7f8240143130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:58.234 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.224+0000 7f8247bd7640 1 -- 172.21.15.16:0/2818528350 >> 172.21.15.16:0/2818528350 conn(0x7f8240071f40 msgr2=0x7f8240072320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:58.234 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.225+0000 7f8247bd7640 1 -- 172.21.15.16:0/2818528350 shutdown_connections 2023-12-08T08:37:58.234 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.225+0000 7f8247bd7640 1 -- 172.21.15.16:0/2818528350 wait complete. 2023-12-08T08:37:58.234 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.226+0000 7f8247bd7640 1 Processor -- start 2023-12-08T08:37:58.235 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.226+0000 7f8247bd7640 1 -- start start 2023-12-08T08:37:58.235 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8247bd7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82400b4750 0x7f8240157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:58.235 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8247bd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82400b4b30 0x7f82400c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:58.235 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8247bd7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8240140210 0x7f82400ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:37:58.235 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8247bd7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82401f7040 con 0x7f82400b4b30 2023-12-08T08:37:58.235 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8247bd7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82401f62c0 con 0x7f82400b4750 2023-12-08T08:37:58.235 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8247bd7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82400ca760 con 0x7f8240140210 2023-12-08T08:37:58.236 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8245172640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82400b4b30 0x7f82400c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:58.236 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8245172640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82400b4b30 0x7f82400c9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34092/0 (socket says 172.21.15.16:34092) 2023-12-08T08:37:58.236 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8245172640 1 -- 172.21.15.16:0/1515966351 learned_addr learned my addr 172.21.15.16:0/1515966351 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:37:58.236 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8245973640 1 --2- 172.21.15.16:0/1515966351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82400b4750 0x7f8240157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:58.236 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8246174640 1 --2- 172.21.15.16:0/1515966351 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8240140210 0x7f82400ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:37:58.236 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8245172640 1 -- 172.21.15.16:0/1515966351 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8240140210 msgr2=0x7f82400ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:58.236 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8245172640 1 --2- 172.21.15.16:0/1515966351 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8240140210 0x7f82400ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:58.236 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8245172640 1 -- 172.21.15.16:0/1515966351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82400b4750 msgr2=0x7f8240157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:58.236 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8245172640 1 --2- 172.21.15.16:0/1515966351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82400b4750 0x7f8240157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:58.237 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8245172640 1 -- 172.21.15.16:0/1515966351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823c009400 con 0x7f82400b4b30 2023-12-08T08:37:58.237 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8245172640 1 --2- 172.21.15.16:0/1515966351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82400b4b30 0x7f82400c9c10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f823000c4d0 tx=0x7f823000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:37:58.237 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f82367fc640 1 -- 172.21.15.16:0/1515966351 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82300074e0 con 0x7f82400b4b30 2023-12-08T08:37:58.237 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f82367fc640 1 -- 172.21.15.16:0/1515966351 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8230007660 con 0x7f82400b4b30 2023-12-08T08:37:58.237 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f82367fc640 1 -- 172.21.15.16:0/1515966351 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230004a20 con 0x7f82400b4b30 2023-12-08T08:37:58.237 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.227+0000 7f8247bd7640 1 -- 172.21.15.16:0/1515966351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8238009a70 con 0x7f82400b4b30 2023-12-08T08:37:58.237 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.228+0000 7f8247bd7640 1 -- 172.21.15.16:0/1515966351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82400b8690 con 0x7f82400b4b30 2023-12-08T08:37:58.238 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.228+0000 7f82367fc640 1 -- 172.21.15.16:0/1515966351 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8230012400 con 0x7f82400b4b30 2023-12-08T08:37:58.238 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.228+0000 7f82367fc640 1 -- 172.21.15.16:0/1515966351 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f823001ba30 con 0x7f82400b4b30 2023-12-08T08:37:58.238 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.228+0000 7f8247bd7640 1 -- 172.21.15.16:0/1515966351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82400678e0 con 0x7f82400b4b30 2023-12-08T08:37:58.239 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.230+0000 7f82367fc640 1 -- 172.21.15.16:0/1515966351 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f8230018070 con 0x7f82400b4b30 2023-12-08T08:37:58.295 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.292+0000 7f8247bd7640 1 -- 172.21.15.16:0/1515966351 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f824004f090 con 0x7f82400b4b30 2023-12-08T08:37:58.296 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.293+0000 7f82367fc640 1 -- 172.21.15.16:0/1515966351 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f823001bcf0 con 0x7f82400b4b30 2023-12-08T08:37:58.296 INFO:teuthology.misc.health.smithi016.stdout: 2023-12-08T08:37:58.296 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":10,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:37:57.072680+0000","last_up_change":"0.000000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:37:58.297 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.293+0000 7f8247bd7640 1 -- 172.21.15.16:0/1515966351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82400b4b30 msgr2=0x7f82400c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:37:58.297 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.293+0000 7f8247bd7640 1 --2- 172.21.15.16:0/1515966351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82400b4b30 0x7f82400c9c10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f823000c4d0 tx=0x7f823000c990 comp rx=0 tx=0).stop 2023-12-08T08:37:58.297 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.293+0000 7f8247bd7640 1 -- 172.21.15.16:0/1515966351 shutdown_connections 2023-12-08T08:37:58.297 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.293+0000 7f8247bd7640 1 --2- 172.21.15.16:0/1515966351 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8240140210 0x7f82400ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:58.297 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.293+0000 7f8247bd7640 1 --2- 172.21.15.16:0/1515966351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82400b4b30 0x7f82400c9c10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:58.297 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.293+0000 7f8247bd7640 1 --2- 172.21.15.16:0/1515966351 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f82400b4750 0x7f8240157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:37:58.297 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.293+0000 7f8247bd7640 1 -- 172.21.15.16:0/1515966351 >> 172.21.15.16:0/1515966351 conn(0x7f8240071f40 msgr2=0x7f82400708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:37:58.297 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.294+0000 7f8247bd7640 1 -- 172.21.15.16:0/1515966351 shutdown_connections 2023-12-08T08:37:58.298 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:37:58.294+0000 7f8247bd7640 1 -- 172.21.15.16:0/1515966351 wait complete. 2023-12-08T08:37:58.305 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-08T08:37:58.532 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:58.529+0000 7f091f230040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-08T08:37:58.632 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:58.629+0000 7f898324c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-08T08:37:58.799 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:58.796+0000 7f091f230040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-08T08:37:58.850 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:58.847+0000 7f091f230040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-08T08:37:58.916 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:58.913+0000 7f898324c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-08T08:37:58.950 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:58.947+0000 7f091f230040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-08T08:37:58.970 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:58.967+0000 7f898324c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-08T08:37:59.077 INFO:tasks.ceph.mgr.y.smithi016.stderr:2023-12-08T08:37:59.075+0000 7f898324c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-08T08:37:59.980 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:59.978+0000 7f08bfed7640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-12-08T08:37:59.982 INFO:tasks.ceph.mgr.x.smithi112.stderr:2023-12-08T08:37:59.980+0000 7f08bfed7640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-12-08T08:38:01.148 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T08:38:01.146+0000 7f7fa0412500 -1 osd.7 0 log_to_monitors true 2023-12-08T08:38:01.171 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T08:38:01.169+0000 7fdb11010500 -1 osd.3 0 log_to_monitors true 2023-12-08T08:38:01.196 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T08:38:01.194+0000 7eff92222500 -1 osd.2 0 log_to_monitors true 2023-12-08T08:38:01.199 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T08:38:01.197+0000 7fb33079b500 -1 osd.6 0 log_to_monitors true 2023-12-08T08:38:01.221 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T08:38:01.219+0000 7f147da11500 -1 osd.5 0 log_to_monitors true 2023-12-08T08:38:01.223 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T08:38:01.221+0000 7ff2542fa500 -1 osd.1 0 log_to_monitors true 2023-12-08T08:38:01.243 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T08:38:01.242+0000 7f6d12820500 -1 osd.0 0 log_to_monitors true 2023-12-08T08:38:01.253 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T08:38:01.250+0000 7f0ae6012500 -1 osd.4 0 log_to_monitors true 2023-12-08T08:38:04.642 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T08:38:04.639+0000 7f6d0e979640 -1 osd.0 0 waiting for initial osdmap 2023-12-08T08:38:04.646 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T08:38:04.644+0000 7ff25028a640 -1 osd.1 0 waiting for initial osdmap 2023-12-08T08:38:04.652 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T08:38:04.651+0000 7f6d05f8c640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-08T08:38:04.656 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T08:38:04.655+0000 7ff2480b1640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-08T08:38:04.707 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T08:38:04.705+0000 7fdb0d0f3640 -1 osd.3 0 waiting for initial osdmap 2023-12-08T08:38:04.716 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T08:38:04.714+0000 7eff8e404640 -1 osd.2 0 waiting for initial osdmap 2023-12-08T08:38:04.725 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T08:38:04.724+0000 7fdb04706640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-08T08:38:04.731 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T08:38:04.729+0000 7eff85a17640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-08T08:38:05.102 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T08:38:05.099+0000 7f0ae1977640 -1 osd.4 0 waiting for initial osdmap 2023-12-08T08:38:05.112 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:05.117 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T08:38:05.115+0000 7f0ad979e640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-08T08:38:05.119 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T08:38:05.117+0000 7f14793de640 -1 osd.5 0 waiting for initial osdmap 2023-12-08T08:38:05.131 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T08:38:05.129+0000 7f1471205640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-08T08:38:05.154 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T08:38:05.152+0000 7f7f9bd09640 -1 osd.7 0 waiting for initial osdmap 2023-12-08T08:38:05.168 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T08:38:05.166+0000 7f7f93b30640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-08T08:38:05.172 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T08:38:05.170+0000 7fb32c72b640 -1 osd.6 0 waiting for initial osdmap 2023-12-08T08:38:05.183 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T08:38:05.181+0000 7fb324552640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-08T08:38:05.209 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.207+0000 7fd792ad4640 1 Processor -- start 2023-12-08T08:38:05.210 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.207+0000 7fd792ad4640 1 -- start start 2023-12-08T08:38:05.211 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.208+0000 7fd792ad4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd78c140100 0x7fd78c0b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:05.211 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.208+0000 7fd792ad4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd78c161790 0x7fd78c0b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:05.211 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.208+0000 7fd792ad4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd78c161ba0 con 0x7fd78c0b78e0 2023-12-08T08:38:05.211 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.208+0000 7fd792ad4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78c0d2170 con 0x7fd78c140100 2023-12-08T08:38:05.211 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.208+0000 7fd792ad4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c0b82d0 con 0x7fd78c161790 2023-12-08T08:38:05.211 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.208+0000 7fd78bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd78c0b78e0 0x7fd78c161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:55864/0 (socket says 172.21.15.16:55864) 2023-12-08T08:38:05.211 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.208+0000 7fd78bfff640 1 -- 172.21.15.16:0/2723956619 learned_addr learned my addr 172.21.15.16:0/2723956619 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:05.211 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.209+0000 7fd791071640 1 --2- 172.21.15.16:0/2723956619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd78c161790 0x7fd78c0b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:05.212 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.209+0000 7fd790870640 1 --2- 172.21.15.16:0/2723956619 >> v2:172.21.15.112:3300/0 conn(0x7fd78c140100 0x7fd78c0b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:05.212 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.209+0000 7fd78b7fe640 1 -- 172.21.15.16:0/2723956619 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214004789 0 0) 0x7fd78c161ba0 con 0x7fd78c0b78e0 2023-12-08T08:38:05.212 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.209+0000 7fd78b7fe640 1 -- 172.21.15.16:0/2723956619 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd778003520 con 0x7fd78c0b78e0 2023-12-08T08:38:05.212 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.209+0000 7fd791071640 1 -- 172.21.15.16:0/2723956619 >> v2:172.21.15.112:3300/0 conn(0x7fd78c140100 msgr2=0x7fd78c0b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:05.212 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.209+0000 7fd791071640 1 --2- 172.21.15.16:0/2723956619 >> v2:172.21.15.112:3300/0 conn(0x7fd78c140100 0x7fd78c0b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:05.212 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.209+0000 7fd791071640 1 -- 172.21.15.16:0/2723956619 >> v1:172.21.15.16:6789/0 conn(0x7fd78c0b78e0 legacy=0x7fd78c161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:05.212 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.209+0000 7fd791071640 1 -- 172.21.15.16:0/2723956619 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c0b39e0 con 0x7fd78c161790 2023-12-08T08:38:05.212 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.209+0000 7fd790870640 1 --2- 172.21.15.16:0/2723956619 >> v2:172.21.15.112:3300/0 conn(0x7fd78c140100 0x7fd78c0b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:05.213 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.210+0000 7fd791071640 1 --2- 172.21.15.16:0/2723956619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd78c161790 0x7fd78c0b7d90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd784009d90 tx=0x7fd78402be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71c59fd90421881 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:05.213 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.210+0000 7fd78b7fe640 1 -- 172.21.15.16:0/2723956619 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78402cbc0 con 0x7fd78c161790 2023-12-08T08:38:05.214 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.210+0000 7fd78b7fe640 1 -- 172.21.15.16:0/2723956619 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78402cd20 con 0x7fd78c161790 2023-12-08T08:38:05.214 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.210+0000 7fd78b7fe640 1 -- 172.21.15.16:0/2723956619 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784034660 con 0x7fd78c161790 2023-12-08T08:38:05.214 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.210+0000 7fd792ad4640 1 -- 172.21.15.16:0/2723956619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd78c161790 msgr2=0x7fd78c0b7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:05.214 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.210+0000 7fd792ad4640 1 --2- 172.21.15.16:0/2723956619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd78c161790 0x7fd78c0b7d90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd784009d90 tx=0x7fd78402be10 comp rx=0 tx=0).stop 2023-12-08T08:38:05.214 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.210+0000 7fd792ad4640 1 -- 172.21.15.16:0/2723956619 shutdown_connections 2023-12-08T08:38:05.214 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.210+0000 7fd792ad4640 1 --2- 172.21.15.16:0/2723956619 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd78c161790 0x7fd78c0b7d90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:05.214 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.210+0000 7fd792ad4640 1 --2- 172.21.15.16:0/2723956619 >> v2:172.21.15.112:3300/0 conn(0x7fd78c140100 0x7fd78c0b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:05.214 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.210+0000 7fd792ad4640 1 -- 172.21.15.16:0/2723956619 >> 172.21.15.16:0/2723956619 conn(0x7fd78c072120 msgr2=0x7fd78c072500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:05.214 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.210+0000 7fd792ad4640 1 -- 172.21.15.16:0/2723956619 shutdown_connections 2023-12-08T08:38:05.214 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.210+0000 7fd792ad4640 1 -- 172.21.15.16:0/2723956619 wait complete. 2023-12-08T08:38:05.214 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.212+0000 7fd792ad4640 1 Processor -- start 2023-12-08T08:38:05.214 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.212+0000 7fd792ad4640 1 -- start start 2023-12-08T08:38:05.217 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.212+0000 7fd792ad4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd78c0b78e0 0x7fd78c12e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:05.217 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.212+0000 7fd792ad4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd78c140100 0x7fd78c1f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:05.217 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.212+0000 7fd792ad4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd78c161790 0x7fd78c12b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:05.217 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.212+0000 7fd792ad4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c1f3a60 con 0x7fd78c140100 2023-12-08T08:38:05.217 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.212+0000 7fd792ad4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c1f34a0 con 0x7fd78c0b78e0 2023-12-08T08:38:05.217 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.212+0000 7fd792ad4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c1f3780 con 0x7fd78c161790 2023-12-08T08:38:05.217 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd78bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd78c140100 0x7fd78c1f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:05.217 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd78bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd78c140100 0x7fd78c1f29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34144/0 (socket says 172.21.15.16:34144) 2023-12-08T08:38:05.217 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd78bfff640 1 -- 172.21.15.16:0/1878582554 learned_addr learned my addr 172.21.15.16:0/1878582554 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:05.217 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd790870640 1 --2- 172.21.15.16:0/1878582554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd78c0b78e0 0x7fd78c12e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:05.217 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd791071640 1 --2- 172.21.15.16:0/1878582554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd78c161790 0x7fd78c12b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:05.218 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd78bfff640 1 -- 172.21.15.16:0/1878582554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd78c161790 msgr2=0x7fd78c12b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:05.218 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd78bfff640 1 --2- 172.21.15.16:0/1878582554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd78c161790 0x7fd78c12b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:05.218 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd78bfff640 1 -- 172.21.15.16:0/1878582554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd78c0b78e0 msgr2=0x7fd78c12e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:05.218 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd78bfff640 1 --2- 172.21.15.16:0/1878582554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd78c0b78e0 0x7fd78c12e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:05.218 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd78bfff640 1 -- 172.21.15.16:0/1878582554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd774009400 con 0x7fd78c140100 2023-12-08T08:38:05.218 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd78bfff640 1 --2- 172.21.15.16:0/1878582554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd78c140100 0x7fd78c1f29e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00c500 tx=0x7fd77c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:05.218 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd7897fa640 1 -- 172.21.15.16:0/1878582554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c005ce0 con 0x7fd78c140100 2023-12-08T08:38:05.218 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd7897fa640 1 -- 172.21.15.16:0/1878582554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77c005e40 con 0x7fd78c140100 2023-12-08T08:38:05.218 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd792ad4640 1 -- 172.21.15.16:0/1878582554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd784009a70 con 0x7fd78c140100 2023-12-08T08:38:05.218 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd7897fa640 1 -- 172.21.15.16:0/1878582554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c007330 con 0x7fd78c140100 2023-12-08T08:38:05.218 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.213+0000 7fd792ad4640 1 -- 172.21.15.16:0/1878582554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78c15e050 con 0x7fd78c140100 2023-12-08T08:38:05.218 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.214+0000 7fd792ad4640 1 -- 172.21.15.16:0/1878582554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78c0678e0 con 0x7fd78c140100 2023-12-08T08:38:05.218 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.215+0000 7fd7897fa640 1 -- 172.21.15.16:0/1878582554 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7fd77c007550 con 0x7fd78c140100 2023-12-08T08:38:05.218 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.215+0000 7fd7897fa640 1 --2- 172.21.15.16:0/1878582554 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd76c06adb0 0x7fd76c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:05.219 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.215+0000 7fd7897fa640 1 -- 172.21.15.16:0/1878582554 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fd77c087d80 con 0x7fd78c140100 2023-12-08T08:38:05.219 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.215+0000 7fd790870640 1 --2- 172.21.15.16:0/1878582554 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd76c06adb0 0x7fd76c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:05.219 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.215+0000 7fd790870640 1 --2- 172.21.15.16:0/1878582554 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd76c06adb0 0x7fd76c06d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd774009c60 tx=0x7fd774008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:05.221 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.219+0000 7fd7897fa640 1 -- 172.21.15.16:0/1878582554 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd77c053df0 con 0x7fd78c140100 2023-12-08T08:38:05.400 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.398+0000 7fd792ad4640 1 -- 172.21.15.16:0/1878582554 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd78c15dbb0 con 0x7fd78c140100 2023-12-08T08:38:05.400 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.398+0000 7fd7897fa640 1 -- 172.21.15.16:0/1878582554 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd77c017070 con 0x7fd78c140100 2023-12-08T08:38:05.400 INFO:teuthology.misc.health.smithi016.stdout: 2023-12-08T08:38:05.400 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":12,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:02.984565+0000","last_up_change":"0.000000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:05.402 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.400+0000 7fd792ad4640 1 -- 172.21.15.16:0/1878582554 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd76c06adb0 msgr2=0x7fd76c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:05.402 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.400+0000 7fd792ad4640 1 --2- 172.21.15.16:0/1878582554 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd76c06adb0 0x7fd76c06d270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd774009c60 tx=0x7fd774008040 comp rx=0 tx=0).stop 2023-12-08T08:38:05.402 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.400+0000 7fd792ad4640 1 -- 172.21.15.16:0/1878582554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd78c140100 msgr2=0x7fd78c1f29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:05.402 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.400+0000 7fd792ad4640 1 --2- 172.21.15.16:0/1878582554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd78c140100 0x7fd78c1f29e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00c500 tx=0x7fd77c00c9c0 comp rx=0 tx=0).stop 2023-12-08T08:38:05.402 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.400+0000 7fd792ad4640 1 -- 172.21.15.16:0/1878582554 shutdown_connections 2023-12-08T08:38:05.403 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.400+0000 7fd792ad4640 1 --2- 172.21.15.16:0/1878582554 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd76c06adb0 0x7fd76c06d270 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:05.403 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.400+0000 7fd792ad4640 1 --2- 172.21.15.16:0/1878582554 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd78c161790 0x7fd78c12b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:05.403 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.400+0000 7fd792ad4640 1 --2- 172.21.15.16:0/1878582554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd78c140100 0x7fd78c1f29e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:05.403 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.400+0000 7fd792ad4640 1 --2- 172.21.15.16:0/1878582554 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd78c0b78e0 0x7fd78c12e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:05.403 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.400+0000 7fd792ad4640 1 -- 172.21.15.16:0/1878582554 >> 172.21.15.16:0/1878582554 conn(0x7fd78c072120 msgr2=0x7fd78c072500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:05.403 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.400+0000 7fd792ad4640 1 -- 172.21.15.16:0/1878582554 shutdown_connections 2023-12-08T08:38:05.403 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:05.400+0000 7fd792ad4640 1 -- 172.21.15.16:0/1878582554 wait complete. 2023-12-08T08:38:05.412 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-08T08:38:10.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:38:10.232 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:38:12.219 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:12.323 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.321+0000 7f132dbb0640 1 Processor -- start 2023-12-08T08:38:12.324 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.321+0000 7f132dbb0640 1 -- start start 2023-12-08T08:38:12.327 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.322+0000 7f132dbb0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f13280d1b90 0x7f1328160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.327 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.322+0000 7f132dbb0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1328161c10 con 0x7f13280d0910 2023-12-08T08:38:12.327 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.322+0000 7f132dbb0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1328161370 con 0x7f1328142e20 2023-12-08T08:38:12.327 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.322+0000 7f132dbb0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1328161650 con 0x7f13280d1b90 2023-12-08T08:38:12.327 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.322+0000 7f1326ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f13280d0910 0x7f132815d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46452/0 (socket says 172.21.15.16:46452) 2023-12-08T08:38:12.328 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.322+0000 7f1326ffd640 1 -- 172.21.15.16:0/2640899150 learned_addr learned my addr 172.21.15.16:0/2640899150 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:12.328 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13277fe640 1 --2- 172.21.15.16:0/2640899150 >> v2:172.21.15.112:3300/0 conn(0x7f13280d1b90 0x7f1328160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.328 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842023432 0 0) 0x7f1328161c10 con 0x7f13280d0910 2023-12-08T08:38:12.328 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1310003520 con 0x7f13280d0910 2023-12-08T08:38:12.328 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255765768 0 0) 0x7f1328161370 con 0x7f1328142e20 2023-12-08T08:38:12.328 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1328161c10 con 0x7f1328142e20 2023-12-08T08:38:12.328 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3586336708 0 0) 0x7f1310003520 con 0x7f13280d0910 2023-12-08T08:38:12.328 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 >> v2:172.21.15.112:3300/0 conn(0x7f13280d1b90 msgr2=0x7f1328160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:12.328 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 --2- 172.21.15.16:0/2640899150 >> v2:172.21.15.112:3300/0 conn(0x7f13280d1b90 0x7f1328160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.328 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 >> v1:172.21.15.112:6789/0 conn(0x7f1328142e20 legacy=0x7f1328070db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.328 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1328161930 con 0x7f13280d0910 2023-12-08T08:38:12.328 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f131c001670 con 0x7f13280d0910 2023-12-08T08:38:12.328 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13277fe640 1 --2- 172.21.15.16:0/2640899150 >> v2:172.21.15.112:3300/0 conn(0x7f13280d1b90 0x7f1328160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:12.328 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 >> v1:172.21.15.16:6789/0 conn(0x7f13280d0910 legacy=0x7f132815d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.329 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 --2- 172.21.15.16:0/2640899150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310003b00 0x7f1310003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.329 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 --2- 172.21.15.16:0/2640899150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13100047c0 0x7f1310004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.329 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 --2- 172.21.15.16:0/2640899150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1310005180 0x7f131000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.329 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1328161650 con 0x7f1310003b00 2023-12-08T08:38:12.329 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131000c0b0 con 0x7f1310005180 2023-12-08T08:38:12.329 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131000c270 con 0x7f13100047c0 2023-12-08T08:38:12.329 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13277fe640 1 --2- 172.21.15.16:0/2640899150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310003b00 0x7f1310003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.329 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f1326ffd640 1 --2- 172.21.15.16:0/2640899150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13100047c0 0x7f1310004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.329 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f1327fff640 1 --2- 172.21.15.16:0/2640899150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1310005180 0x7f131000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.329 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13277fe640 1 -- 172.21.15.16:0/2640899150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13100047c0 msgr2=0x7f1310004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:12.329 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13277fe640 1 --2- 172.21.15.16:0/2640899150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f13100047c0 0x7f1310004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.329 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13277fe640 1 -- 172.21.15.16:0/2640899150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1310005180 msgr2=0x7f131000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:12.329 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13277fe640 1 --2- 172.21.15.16:0/2640899150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1310005180 0x7f131000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.329 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f13277fe640 1 -- 172.21.15.16:0/2640899150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1310003520 con 0x7f1310003b00 2023-12-08T08:38:12.330 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.323+0000 7f1327fff640 1 -- 172.21.15.16:0/2640899150 reap_dead start 2023-12-08T08:38:12.330 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.324+0000 7f13277fe640 1 --2- 172.21.15.16:0/2640899150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310003b00 0x7f1310003f00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1318009c90 tx=0x7f13100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47b5ecc2d2e80e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:12.330 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.324+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131802c8f0 con 0x7f1310003b00 2023-12-08T08:38:12.330 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.324+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131802ca50 con 0x7f1310003b00 2023-12-08T08:38:12.330 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.324+0000 7f13267fc640 1 -- 172.21.15.16:0/2640899150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131802ce40 con 0x7f1310003b00 2023-12-08T08:38:12.330 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.325+0000 7f132dbb0640 1 -- 172.21.15.16:0/2640899150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310003b00 msgr2=0x7f1310003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.330 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.325+0000 7f132dbb0640 1 --2- 172.21.15.16:0/2640899150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310003b00 0x7f1310003f00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1318009c90 tx=0x7f13100057f0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.330 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.325+0000 7f132dbb0640 1 -- 172.21.15.16:0/2640899150 shutdown_connections 2023-12-08T08:38:12.330 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.325+0000 7f132dbb0640 1 --2- 172.21.15.16:0/2640899150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1310003b00 0x7f1310003f00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.330 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.325+0000 7f132dbb0640 1 -- 172.21.15.16:0/2640899150 >> 172.21.15.16:0/2640899150 conn(0x7f132806f990 msgr2=0x7f13280b54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:12.330 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.325+0000 7f132dbb0640 1 -- 172.21.15.16:0/2640899150 shutdown_connections 2023-12-08T08:38:12.330 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.325+0000 7f132dbb0640 1 -- 172.21.15.16:0/2640899150 wait complete. 2023-12-08T08:38:12.333 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.326+0000 7f132dbb0640 1 Processor -- start 2023-12-08T08:38:12.333 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.327+0000 7f132dbb0640 1 -- start start 2023-12-08T08:38:12.333 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.327+0000 7f132dbb0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1310003b00 0x7f1328158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.333 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.327+0000 7f132dbb0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13281456f0 0x7f13281590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.333 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.327+0000 7f132dbb0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f132813d0d0 0x7f1328154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.334 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.327+0000 7f132dbb0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132815a4c0 con 0x7f13281456f0 2023-12-08T08:38:12.334 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.327+0000 7f132dbb0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1328144a20 con 0x7f132813d0d0 2023-12-08T08:38:12.334 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.327+0000 7f132dbb0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13281599e0 con 0x7f1310003b00 2023-12-08T08:38:12.334 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.327+0000 7f1326ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13281456f0 0x7f13281590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.334 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.327+0000 7f1326ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13281456f0 0x7f13281590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34354/0 (socket says 172.21.15.16:34354) 2023-12-08T08:38:12.334 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.327+0000 7f1326ffd640 1 -- 172.21.15.16:0/3651313272 learned_addr learned my addr 172.21.15.16:0/3651313272 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:12.334 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.328+0000 7f13277fe640 1 --2- 172.21.15.16:0/3651313272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1310003b00 0x7f1328158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.334 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.328+0000 7f1327fff640 1 --2- 172.21.15.16:0/3651313272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f132813d0d0 0x7f1328154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.334 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.328+0000 7f1326ffd640 1 -- 172.21.15.16:0/3651313272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1310003b00 msgr2=0x7f1328158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.334 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.328+0000 7f1326ffd640 1 --2- 172.21.15.16:0/3651313272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1310003b00 0x7f1328158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.335 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.328+0000 7f1326ffd640 1 -- 172.21.15.16:0/3651313272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f132813d0d0 msgr2=0x7f1328154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.335 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.328+0000 7f1326ffd640 1 --2- 172.21.15.16:0/3651313272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f132813d0d0 0x7f1328154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.335 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.328+0000 7f1326ffd640 1 -- 172.21.15.16:0/3651313272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1328159cc0 con 0x7f13281456f0 2023-12-08T08:38:12.335 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.328+0000 7f1326ffd640 1 --2- 172.21.15.16:0/3651313272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13281456f0 0x7f13281590a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f131c001400 tx=0x7f131c00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:12.335 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.328+0000 7f130ffff640 1 -- 172.21.15.16:0/3651313272 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c00c660 con 0x7f13281456f0 2023-12-08T08:38:12.335 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.328+0000 7f130ffff640 1 -- 172.21.15.16:0/3651313272 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f131c00c7c0 con 0x7f13281456f0 2023-12-08T08:38:12.335 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.328+0000 7f130ffff640 1 -- 172.21.15.16:0/3651313272 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c00cb70 con 0x7f13281456f0 2023-12-08T08:38:12.335 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.328+0000 7f132dbb0640 1 -- 172.21.15.16:0/3651313272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1328159fa0 con 0x7f13281456f0 2023-12-08T08:38:12.335 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.328+0000 7f132dbb0640 1 -- 172.21.15.16:0/3651313272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1328154d50 con 0x7f13281456f0 2023-12-08T08:38:12.335 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.329+0000 7f132dbb0640 1 -- 172.21.15.16:0/3651313272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13280678e0 con 0x7f13281456f0 2023-12-08T08:38:12.335 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.330+0000 7f130ffff640 1 -- 172.21.15.16:0/3651313272 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f131c00ccd0 con 0x7f13281456f0 2023-12-08T08:38:12.335 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.330+0000 7f130ffff640 1 --2- 172.21.15.16:0/3651313272 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f131406ae40 0x7f131406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.335 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.330+0000 7f130ffff640 1 -- 172.21.15.16:0/3651313272 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f131c0890b0 con 0x7f13281456f0 2023-12-08T08:38:12.335 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.330+0000 7f13277fe640 1 --2- 172.21.15.16:0/3651313272 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f131406ae40 0x7f131406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.336 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.331+0000 7f13277fe640 1 --2- 172.21.15.16:0/3651313272 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f131406ae40 0x7f131406d300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1318036000 tx=0x7f1318030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:12.336 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.335+0000 7f130ffff640 1 -- 172.21.15.16:0/3651313272 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f131c0544b0 con 0x7f13281456f0 2023-12-08T08:38:12.513 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.511+0000 7f132dbb0640 1 -- 172.21.15.16:0/3651313272 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f132804f090 con 0x7f13281456f0 2023-12-08T08:38:12.514 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.512+0000 7f130ffff640 1 -- 172.21.15.16:0/3651313272 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f131c057ad0 con 0x7f13281456f0 2023-12-08T08:38:12.514 INFO:teuthology.misc.health.smithi016.stdout: 2023-12-08T08:38:12.515 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":16,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:09.114424+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:12.517 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.514+0000 7f132dbb0640 1 -- 172.21.15.16:0/3651313272 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f131406ae40 msgr2=0x7f131406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.517 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.514+0000 7f132dbb0640 1 --2- 172.21.15.16:0/3651313272 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f131406ae40 0x7f131406d300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1318036000 tx=0x7f1318030000 comp rx=0 tx=0).stop 2023-12-08T08:38:12.517 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.514+0000 7f132dbb0640 1 -- 172.21.15.16:0/3651313272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13281456f0 msgr2=0x7f13281590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.517 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.514+0000 7f132dbb0640 1 --2- 172.21.15.16:0/3651313272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13281456f0 0x7f13281590a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f131c001400 tx=0x7f131c00eb50 comp rx=0 tx=0).stop 2023-12-08T08:38:12.517 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.514+0000 7f132dbb0640 1 -- 172.21.15.16:0/3651313272 shutdown_connections 2023-12-08T08:38:12.517 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.514+0000 7f132dbb0640 1 --2- 172.21.15.16:0/3651313272 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f131406ae40 0x7f131406d300 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.517 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.514+0000 7f132dbb0640 1 --2- 172.21.15.16:0/3651313272 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f132813d0d0 0x7f1328154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.517 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.514+0000 7f132dbb0640 1 --2- 172.21.15.16:0/3651313272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13281456f0 0x7f13281590a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.517 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.514+0000 7f132dbb0640 1 --2- 172.21.15.16:0/3651313272 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1310003b00 0x7f1328158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.517 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.514+0000 7f132dbb0640 1 -- 172.21.15.16:0/3651313272 >> 172.21.15.16:0/3651313272 conn(0x7f132806f990 msgr2=0x7f1328140640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:12.517 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.514+0000 7f132dbb0640 1 -- 172.21.15.16:0/3651313272 shutdown_connections 2023-12-08T08:38:12.517 INFO:teuthology.misc.health.smithi016.stderr:2023-12-08T08:38:12.514+0000 7f132dbb0640 1 -- 172.21.15.16:0/3651313272 wait complete. 2023-12-08T08:38:12.526 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-12-08T08:38:12.526 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-12-08T08:38:12.526 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-08T08:38:12.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-08T08:38:12.528 INFO:tasks.ceph.mds.a:Started 2023-12-08T08:38:12.529 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-08T08:38:12.529 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-08T08:38:12.531 INFO:tasks.ceph.mds.c:Started 2023-12-08T08:38:12.531 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-08T08:38:12.531 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-08T08:38:12.533 INFO:tasks.ceph.mds.b:Started 2023-12-08T08:38:12.534 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-08T08:38:12.534 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-08T08:38:12.536 INFO:tasks.ceph.mds.d:Started 2023-12-08T08:38:12.536 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-12-08T08:38:12.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:12.609 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-12-08T08:38:12.609 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-12-08T08:38:12.610 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-12-08T08:38:12.608+0000 7fec7093a640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2023-12-08T08:38:12.618 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2023-12-08T08:38:12.618 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2023-12-08T08:38:12.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.645+0000 7f25ed12c640 1 Processor -- start 2023-12-08T08:38:12.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.645+0000 7f25ed12c640 1 -- start start 2023-12-08T08:38:12.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25ed12c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f25e8157ba0 0x7f25e815a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25ed12c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25e806d9b0 0x7f25e80b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25ed12c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e815b410 con 0x7f25e815a960 2023-12-08T08:38:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25ed12c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e815ae50 con 0x7f25e8157ba0 2023-12-08T08:38:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25ed12c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e815b130 con 0x7f25e806d9b0 2023-12-08T08:38:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25e659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f25e815a960 0x7f25e806d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46472/0 (socket says 172.21.15.16:46472) 2023-12-08T08:38:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25e659c640 1 -- 172.21.15.16:0/319975236 learned_addr learned my addr 172.21.15.16:0/319975236 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25e759e640 1 --2- 172.21.15.16:0/319975236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25e806d9b0 0x7f25e80b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25e6d9d640 1 --2- 172.21.15.16:0/319975236 >> v2:172.21.15.112:3300/0 conn(0x7f25e8157ba0 0x7f25e815a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25e5d9b640 1 -- 172.21.15.16:0/319975236 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167887912 0 0) 0x7f25e815b410 con 0x7f25e815a960 2023-12-08T08:38:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25e5d9b640 1 -- 172.21.15.16:0/319975236 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25d0003520 con 0x7f25e815a960 2023-12-08T08:38:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25e759e640 1 -- 172.21.15.16:0/319975236 >> v2:172.21.15.112:3300/0 conn(0x7f25e8157ba0 msgr2=0x7f25e815a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25e759e640 1 --2- 172.21.15.16:0/319975236 >> v2:172.21.15.112:3300/0 conn(0x7f25e8157ba0 0x7f25e815a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25e759e640 1 -- 172.21.15.16:0/319975236 >> v1:172.21.15.16:6789/0 conn(0x7f25e815a960 legacy=0x7f25e806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25e759e640 1 -- 172.21.15.16:0/319975236 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e80b30b0 con 0x7f25e806d9b0 2023-12-08T08:38:12.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.646+0000 7f25e6d9d640 1 --2- 172.21.15.16:0/319975236 >> v2:172.21.15.112:3300/0 conn(0x7f25e8157ba0 0x7f25e815a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.647+0000 7f25e759e640 1 --2- 172.21.15.16:0/319975236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25e806d9b0 0x7f25e80b38b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009d90 tx=0x7f25e002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43b27b31e6a33cd8 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.647+0000 7f25e5d9b640 1 -- 172.21.15.16:0/319975236 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e002cbc0 con 0x7f25e806d9b0 2023-12-08T08:38:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.647+0000 7f25e5d9b640 1 -- 172.21.15.16:0/319975236 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e002cd20 con 0x7f25e806d9b0 2023-12-08T08:38:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.647+0000 7f25e5d9b640 1 -- 172.21.15.16:0/319975236 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0034660 con 0x7f25e806d9b0 2023-12-08T08:38:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.648+0000 7f25ed12c640 1 -- 172.21.15.16:0/319975236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25e806d9b0 msgr2=0x7f25e80b38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.648+0000 7f25ed12c640 1 --2- 172.21.15.16:0/319975236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25e806d9b0 0x7f25e80b38b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009d90 tx=0x7f25e002be10 comp rx=0 tx=0).stop 2023-12-08T08:38:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.648+0000 7f25ed12c640 1 -- 172.21.15.16:0/319975236 shutdown_connections 2023-12-08T08:38:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.648+0000 7f25ed12c640 1 --2- 172.21.15.16:0/319975236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25e806d9b0 0x7f25e80b38b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.648+0000 7f25ed12c640 1 --2- 172.21.15.16:0/319975236 >> v2:172.21.15.112:3300/0 conn(0x7f25e8157ba0 0x7f25e815a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.648+0000 7f25ed12c640 1 -- 172.21.15.16:0/319975236 >> 172.21.15.16:0/319975236 conn(0x7f25e80cff90 msgr2=0x7f25e80d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.648+0000 7f25ed12c640 1 -- 172.21.15.16:0/319975236 shutdown_connections 2023-12-08T08:38:12.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.648+0000 7f25ed12c640 1 -- 172.21.15.16:0/319975236 wait complete. 2023-12-08T08:38:12.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.649+0000 7f25ed12c640 1 Processor -- start 2023-12-08T08:38:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.649+0000 7f25ed12c640 1 -- start start 2023-12-08T08:38:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25ed12c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e806d9b0 0x7f25e81558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25ed12c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e8157ba0 0x7f25e80c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25ed12c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25e815a960 0x7f25e81565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25ed12c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e80c4ef0 con 0x7f25e806d9b0 2023-12-08T08:38:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25ed12c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e80c5140 con 0x7f25e8157ba0 2023-12-08T08:38:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25ed12c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e80c5420 con 0x7f25e815a960 2023-12-08T08:38:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25e6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e806d9b0 0x7f25e81558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25e6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e806d9b0 0x7f25e81558e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34398/0 (socket says 172.21.15.16:34398) 2023-12-08T08:38:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25e6d9d640 1 -- 172.21.15.16:0/2153871433 learned_addr learned my addr 172.21.15.16:0/2153871433 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25e659c640 1 --2- 172.21.15.16:0/2153871433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e8157ba0 0x7f25e80c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25e759e640 1 --2- 172.21.15.16:0/2153871433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25e815a960 0x7f25e81565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25e6d9d640 1 -- 172.21.15.16:0/2153871433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25e815a960 msgr2=0x7f25e81565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25e6d9d640 1 --2- 172.21.15.16:0/2153871433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25e815a960 0x7f25e81565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25e6d9d640 1 -- 172.21.15.16:0/2153871433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e8157ba0 msgr2=0x7f25e80c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.650+0000 7f25e6d9d640 1 --2- 172.21.15.16:0/2153871433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e8157ba0 0x7f25e80c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.651+0000 7f25e6d9d640 1 -- 172.21.15.16:0/2153871433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d8009400 con 0x7f25e806d9b0 2023-12-08T08:38:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.651+0000 7f25e6d9d640 1 --2- 172.21.15.16:0/2153871433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e806d9b0 0x7f25e81558e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f25d800ebe0 tx=0x7f25d800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.651+0000 7f25cf7fe640 1 -- 172.21.15.16:0/2153871433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8009760 con 0x7f25e806d9b0 2023-12-08T08:38:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.651+0000 7f25cf7fe640 1 -- 172.21.15.16:0/2153871433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d800cc30 con 0x7f25e806d9b0 2023-12-08T08:38:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.651+0000 7f25ed12c640 1 -- 172.21.15.16:0/2153871433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e0009a70 con 0x7f25e806d9b0 2023-12-08T08:38:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.651+0000 7f25cf7fe640 1 -- 172.21.15.16:0/2153871433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d800eee0 con 0x7f25e806d9b0 2023-12-08T08:38:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.651+0000 7f25ed12c640 1 -- 172.21.15.16:0/2153871433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e81548b0 con 0x7f25e806d9b0 2023-12-08T08:38:12.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.652+0000 7f25ed12c640 1 -- 172.21.15.16:0/2153871433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e8067ad0 con 0x7f25e806d9b0 2023-12-08T08:38:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.652+0000 7f25cf7fe640 1 -- 172.21.15.16:0/2153871433 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f25d8013470 con 0x7f25e806d9b0 2023-12-08T08:38:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.653+0000 7f25cf7fe640 1 --2- 172.21.15.16:0/2153871433 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f25d406ae40 0x7f25d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.653+0000 7f25cf7fe640 1 -- 172.21.15.16:0/2153871433 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f25d8089540 con 0x7f25e806d9b0 2023-12-08T08:38:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.653+0000 7f25e659c640 1 --2- 172.21.15.16:0/2153871433 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f25d406ae40 0x7f25d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.653+0000 7f25e659c640 1 --2- 172.21.15.16:0/2153871433 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f25d406ae40 0x7f25d406d300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f25e8154250 tx=0x7f25dc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:12.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.657+0000 7f25cf7fe640 1 -- 172.21.15.16:0/2153871433 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f25d80549d0 con 0x7f25e806d9b0 2023-12-08T08:38:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.835+0000 7f25ed12c640 1 -- 172.21.15.16:0/2153871433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25e806e950 con 0x7f25e806d9b0 2023-12-08T08:38:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.835+0000 7f25cf7fe640 1 -- 172.21.15.16:0/2153871433 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f25d8057ff0 con 0x7f25e806d9b0 2023-12-08T08:38:12.838 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:12.838 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:09.114424+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:12.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.837+0000 7f25ed12c640 1 -- 172.21.15.16:0/2153871433 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f25d406ae40 msgr2=0x7f25d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.837+0000 7f25ed12c640 1 --2- 172.21.15.16:0/2153871433 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f25d406ae40 0x7f25d406d300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f25e8154250 tx=0x7f25dc009040 comp rx=0 tx=0).stop 2023-12-08T08:38:12.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.837+0000 7f25ed12c640 1 -- 172.21.15.16:0/2153871433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e806d9b0 msgr2=0x7f25e81558e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.837+0000 7f25ed12c640 1 --2- 172.21.15.16:0/2153871433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e806d9b0 0x7f25e81558e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f25d800ebe0 tx=0x7f25d800c520 comp rx=0 tx=0).stop 2023-12-08T08:38:12.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.837+0000 7f25ed12c640 1 -- 172.21.15.16:0/2153871433 shutdown_connections 2023-12-08T08:38:12.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.838+0000 7f25ed12c640 1 --2- 172.21.15.16:0/2153871433 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f25d406ae40 0x7f25d406d300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.838+0000 7f25ed12c640 1 --2- 172.21.15.16:0/2153871433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25e815a960 0x7f25e81565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.838+0000 7f25ed12c640 1 --2- 172.21.15.16:0/2153871433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25e8157ba0 0x7f25e80c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.838+0000 7f25ed12c640 1 --2- 172.21.15.16:0/2153871433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e806d9b0 0x7f25e81558e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.838+0000 7f25ed12c640 1 -- 172.21.15.16:0/2153871433 >> 172.21.15.16:0/2153871433 conn(0x7f25e80cff90 msgr2=0x7f25e80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:12.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.838+0000 7f25ed12c640 1 -- 172.21.15.16:0/2153871433 shutdown_connections 2023-12-08T08:38:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.838+0000 7f25ed12c640 1 -- 172.21.15.16:0/2153871433 wait complete. 2023-12-08T08:38:12.850 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-08T08:38:12.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:38:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.958+0000 7ff46fae7640 1 Processor -- start 2023-12-08T08:38:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.958+0000 7ff46fae7640 1 -- start start 2023-12-08T08:38:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.959+0000 7ff46fae7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4680cf9b0 0x7ff4680b7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.959+0000 7ff46fae7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4680b5f00 con 0x7ff4680b8950 2023-12-08T08:38:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.959+0000 7ff46fae7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4680b6170 con 0x7ff4680cf570 2023-12-08T08:38:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.959+0000 7ff46fae7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4680b6450 con 0x7ff4680cf9b0 2023-12-08T08:38:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.959+0000 7ff46e084640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4680cf9b0 0x7ff4680b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.959+0000 7ff46d883640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff4680b8950 0x7ff4680ce680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46476/0 (socket says 172.21.15.16:46476) 2023-12-08T08:38:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.959+0000 7ff46d883640 1 -- 172.21.15.16:0/1655490958 learned_addr learned my addr 172.21.15.16:0/1655490958 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.959+0000 7ff46e084640 1 -- 172.21.15.16:0/1655490958 >> v1:172.21.15.112:6789/0 conn(0x7ff4680cf570 legacy=0x7ff4680b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.959+0000 7ff46e084640 1 -- 172.21.15.16:0/1655490958 >> v1:172.21.15.16:6789/0 conn(0x7ff4680b8950 legacy=0x7ff4680ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.959+0000 7ff46e084640 1 -- 172.21.15.16:0/1655490958 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4680b6730 con 0x7ff4680cf9b0 2023-12-08T08:38:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.960+0000 7ff46e084640 1 --2- 172.21.15.16:0/1655490958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4680cf9b0 0x7ff4680b7360 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff460009d90 tx=0x7ff46002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1835fc8ca1dab2 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.960+0000 7ff46c881640 1 -- 172.21.15.16:0/1655490958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46002cc60 con 0x7ff4680cf9b0 2023-12-08T08:38:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.960+0000 7ff46c881640 1 -- 172.21.15.16:0/1655490958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46002cdc0 con 0x7ff4680cf9b0 2023-12-08T08:38:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.960+0000 7ff46c881640 1 -- 172.21.15.16:0/1655490958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460034660 con 0x7ff4680cf9b0 2023-12-08T08:38:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.960+0000 7ff46fae7640 1 -- 172.21.15.16:0/1655490958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4680cf9b0 msgr2=0x7ff4680b7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.960+0000 7ff46fae7640 1 --2- 172.21.15.16:0/1655490958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4680cf9b0 0x7ff4680b7360 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff460009d90 tx=0x7ff46002beb0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.960+0000 7ff46fae7640 1 -- 172.21.15.16:0/1655490958 shutdown_connections 2023-12-08T08:38:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.960+0000 7ff46fae7640 1 --2- 172.21.15.16:0/1655490958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4680cf9b0 0x7ff4680b7360 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.960+0000 7ff46fae7640 1 -- 172.21.15.16:0/1655490958 >> 172.21.15.16:0/1655490958 conn(0x7ff4680d3010 msgr2=0x7ff4680d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:12.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.961+0000 7ff46fae7640 1 -- 172.21.15.16:0/1655490958 shutdown_connections 2023-12-08T08:38:12.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.961+0000 7ff46fae7640 1 -- 172.21.15.16:0/1655490958 wait complete. 2023-12-08T08:38:12.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.962+0000 7ff46fae7640 1 Processor -- start 2023-12-08T08:38:12.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.962+0000 7ff46fae7640 1 -- start start 2023-12-08T08:38:12.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46fae7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4680b8950 0x7ff468150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46fae7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4680cf570 0x7ff46809d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46fae7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4680cf9b0 0x7ff46809dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46fae7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680c7470 con 0x7ff4680cf570 2023-12-08T08:38:12.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46fae7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680c6860 con 0x7ff4680b8950 2023-12-08T08:38:12.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46fae7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4680c6b40 con 0x7ff4680cf9b0 2023-12-08T08:38:12.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46d082640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4680cf570 0x7ff46809d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46d082640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4680cf570 0x7ff46809d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34420/0 (socket says 172.21.15.16:34420) 2023-12-08T08:38:12.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46d082640 1 -- 172.21.15.16:0/2527421380 learned_addr learned my addr 172.21.15.16:0/2527421380 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:12.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46e084640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4680cf9b0 0x7ff46809dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46d883640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4680b8950 0x7ff468150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46d082640 1 -- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4680cf9b0 msgr2=0x7ff46809dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46d082640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4680cf9b0 0x7ff46809dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46d082640 1 -- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4680b8950 msgr2=0x7ff468150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:12.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46d082640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4680b8950 0x7ff468150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:12.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46d082640 1 -- 172.21.15.16:0/2527421380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff460009a70 con 0x7ff4680cf570 2023-12-08T08:38:12.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46d883640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4680b8950 0x7ff468150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:12.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46d082640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4680cf570 0x7ff46809d860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff45800c450 tx=0x7ff45800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:12.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff45e7fc640 1 -- 172.21.15.16:0/2527421380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4580052d0 con 0x7ff4680cf570 2023-12-08T08:38:12.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff45e7fc640 1 -- 172.21.15.16:0/2527421380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff458005430 con 0x7ff4680cf570 2023-12-08T08:38:12.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46fae7640 1 -- 172.21.15.16:0/2527421380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4680c6e20 con 0x7ff4680cf570 2023-12-08T08:38:12.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff45e7fc640 1 -- 172.21.15.16:0/2527421380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458012870 con 0x7ff4680cf570 2023-12-08T08:38:12.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.963+0000 7ff46fae7640 1 -- 172.21.15.16:0/2527421380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46809ce70 con 0x7ff4680cf570 2023-12-08T08:38:12.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.964+0000 7ff46fae7640 1 -- 172.21.15.16:0/2527421380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4680d0250 con 0x7ff4680cf570 2023-12-08T08:38:12.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.965+0000 7ff45e7fc640 1 -- 172.21.15.16:0/2527421380 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff4580129d0 con 0x7ff4680cf570 2023-12-08T08:38:12.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.965+0000 7ff45e7fc640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff44c06ae40 0x7ff44c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:12.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.965+0000 7ff45e7fc640 1 -- 172.21.15.16:0/2527421380 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff458088e10 con 0x7ff4680cf570 2023-12-08T08:38:12.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.965+0000 7ff46d883640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff44c06ae40 0x7ff44c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:12.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.966+0000 7ff46d883640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff44c06ae40 0x7ff44c06d300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff464005880 tx=0x7ff464009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:12.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:12.969+0000 7ff45e7fc640 1 -- 172.21.15.16:0/2527421380 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff458054210 con 0x7ff4680cf570 2023-12-08T08:38:13.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.143+0000 7ff46fae7640 1 -- 172.21.15.16:0/2527421380 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff468067430 con 0x7ff4680cf570 2023-12-08T08:38:13.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.144+0000 7ff45e7fc640 1 -- 172.21.15.16:0/2527421380 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff458057830 con 0x7ff4680cf570 2023-12-08T08:38:13.146 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:38:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.145+0000 7ff46fae7640 1 -- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff44c06ae40 msgr2=0x7ff44c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.145+0000 7ff46fae7640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff44c06ae40 0x7ff44c06d300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff464005880 tx=0x7ff464009040 comp rx=0 tx=0).stop 2023-12-08T08:38:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.145+0000 7ff46fae7640 1 -- 172.21.15.16:0/2527421380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4680cf570 msgr2=0x7ff46809d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.145+0000 7ff46fae7640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4680cf570 0x7ff46809d860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff45800c450 tx=0x7ff45800c910 comp rx=0 tx=0).stop 2023-12-08T08:38:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.146+0000 7ff46fae7640 1 -- 172.21.15.16:0/2527421380 shutdown_connections 2023-12-08T08:38:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.146+0000 7ff46fae7640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff44c06ae40 0x7ff44c06d300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.146+0000 7ff46fae7640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4680cf9b0 0x7ff46809dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.146+0000 7ff46fae7640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4680cf570 0x7ff46809d860 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.146+0000 7ff46fae7640 1 --2- 172.21.15.16:0/2527421380 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4680b8950 0x7ff468150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.146+0000 7ff46fae7640 1 -- 172.21.15.16:0/2527421380 >> 172.21.15.16:0/2527421380 conn(0x7ff4680d3010 msgr2=0x7ff4680d33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.146+0000 7ff46fae7640 1 -- 172.21.15.16:0/2527421380 shutdown_connections 2023-12-08T08:38:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.146+0000 7ff46fae7640 1 -- 172.21.15.16:0/2527421380 wait complete. 2023-12-08T08:38:13.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-08T08:38:13.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.263+0000 7f6124b0f640 1 Processor -- start 2023-12-08T08:38:13.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.263+0000 7f6124b0f640 1 -- start start 2023-12-08T08:38:13.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.264+0000 7f6124b0f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f61201608a0 0x7f61200d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:13.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.264+0000 7f6124b0f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61200b1d70 0x7f61200d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:13.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.264+0000 7f6124b0f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61200d20b0 con 0x7f61200afce0 2023-12-08T08:38:13.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.264+0000 7f6124b0f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61200d0620 con 0x7f61201608a0 2023-12-08T08:38:13.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.264+0000 7f6124b0f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61200d1dd0 con 0x7f61200b1d70 2023-12-08T08:38:13.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.264+0000 7f6117fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61200b1d70 0x7f61200d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:13.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.264+0000 7f611e59c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f61201608a0 0x7f61200d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:13.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.264+0000 7f611ed9d640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f61200afce0 0x7f61200b0170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:51908/0 (socket says 172.21.15.16:51908) 2023-12-08T08:38:13.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.264+0000 7f611ed9d640 1 -- 172.21.15.16:0/4206219715 learned_addr learned my addr 172.21.15.16:0/4206219715 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:13.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.264+0000 7f6117fff640 1 -- 172.21.15.16:0/4206219715 >> v2:172.21.15.112:3300/0 conn(0x7f61201608a0 msgr2=0x7f61200d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:13.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.264+0000 7f6117fff640 1 --2- 172.21.15.16:0/4206219715 >> v2:172.21.15.112:3300/0 conn(0x7f61201608a0 0x7f61200d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:13.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.264+0000 7f6117fff640 1 -- 172.21.15.16:0/4206219715 >> v1:172.21.15.112:6789/0 conn(0x7f61200afce0 legacy=0x7f61200b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:13.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.264+0000 7f6117fff640 1 -- 172.21.15.16:0/4206219715 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61200cff60 con 0x7f61200b1d70 2023-12-08T08:38:13.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.264+0000 7f611e59c640 1 --2- 172.21.15.16:0/4206219715 >> v2:172.21.15.112:3300/0 conn(0x7f61201608a0 0x7f61200d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:13.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.265+0000 7f6117fff640 1 --2- 172.21.15.16:0/4206219715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61200b1d70 0x7f61200d1470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f61080097a0 tx=0x7f610802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4f5c982952dd4a0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:13.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.265+0000 7f611dd9b640 1 -- 172.21.15.16:0/4206219715 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610802e070 con 0x7f61200b1d70 2023-12-08T08:38:13.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.265+0000 7f611dd9b640 1 -- 172.21.15.16:0/4206219715 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61080315a0 con 0x7f61200b1d70 2023-12-08T08:38:13.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.265+0000 7f611dd9b640 1 -- 172.21.15.16:0/4206219715 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6108031930 con 0x7f61200b1d70 2023-12-08T08:38:13.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.266+0000 7f6124b0f640 1 -- 172.21.15.16:0/4206219715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61200b1d70 msgr2=0x7f61200d1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:13.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.266+0000 7f6124b0f640 1 --2- 172.21.15.16:0/4206219715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61200b1d70 0x7f61200d1470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f61080097a0 tx=0x7f610802b840 comp rx=0 tx=0).stop 2023-12-08T08:38:13.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.266+0000 7f6124b0f640 1 -- 172.21.15.16:0/4206219715 shutdown_connections 2023-12-08T08:38:13.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.266+0000 7f6124b0f640 1 --2- 172.21.15.16:0/4206219715 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61200b1d70 0x7f61200d1470 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:13.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.266+0000 7f6124b0f640 1 --2- 172.21.15.16:0/4206219715 >> v2:172.21.15.112:3300/0 conn(0x7f61201608a0 0x7f61200d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:13.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.266+0000 7f6124b0f640 1 -- 172.21.15.16:0/4206219715 >> 172.21.15.16:0/4206219715 conn(0x7f61200d3030 msgr2=0x7f61200d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:13.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.266+0000 7f6124b0f640 1 -- 172.21.15.16:0/4206219715 shutdown_connections 2023-12-08T08:38:13.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.266+0000 7f6124b0f640 1 -- 172.21.15.16:0/4206219715 wait complete. 2023-12-08T08:38:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.267+0000 7f6124b0f640 1 Processor -- start 2023-12-08T08:38:13.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.267+0000 7f6124b0f640 1 -- start start 2023-12-08T08:38:13.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f6124b0f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61200afce0 0x7f61200c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:13.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f6124b0f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61200b1d70 0x7f61201f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:13.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f6124b0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61201608a0 0x7f61201f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:13.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f6124b0f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612014cba0 con 0x7f61201608a0 2023-12-08T08:38:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f6124b0f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61200c7d70 con 0x7f61200afce0 2023-12-08T08:38:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f6124b0f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61200c8050 con 0x7f61200b1d70 2023-12-08T08:38:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f611ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61201608a0 0x7f61201f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f611ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61201608a0 0x7f61201f5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34424/0 (socket says 172.21.15.16:34424) 2023-12-08T08:38:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f611ed9d640 1 -- 172.21.15.16:0/1474115636 learned_addr learned my addr 172.21.15.16:0/1474115636 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f611e59c640 1 --2- 172.21.15.16:0/1474115636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61200afce0 0x7f61200c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f6117fff640 1 --2- 172.21.15.16:0/1474115636 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61200b1d70 0x7f61201f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f611ed9d640 1 -- 172.21.15.16:0/1474115636 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61200b1d70 msgr2=0x7f61201f6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f611ed9d640 1 --2- 172.21.15.16:0/1474115636 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61200b1d70 0x7f61201f6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f611ed9d640 1 -- 172.21.15.16:0/1474115636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61200afce0 msgr2=0x7f61200c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f611ed9d640 1 --2- 172.21.15.16:0/1474115636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61200afce0 0x7f61200c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.268+0000 7f611ed9d640 1 -- 172.21.15.16:0/1474115636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6110009400 con 0x7f61201608a0 2023-12-08T08:38:13.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.269+0000 7f611ed9d640 1 --2- 172.21.15.16:0/1474115636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61201608a0 0x7f61201f5ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f611800ca80 tx=0x7f611800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.269+0000 7f6116ffd640 1 -- 172.21.15.16:0/1474115636 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611800ac30 con 0x7f61201608a0 2023-12-08T08:38:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.269+0000 7f6116ffd640 1 -- 172.21.15.16:0/1474115636 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611800ad90 con 0x7f61201608a0 2023-12-08T08:38:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.269+0000 7f6124b0f640 1 -- 172.21.15.16:0/1474115636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6108009400 con 0x7f61201608a0 2023-12-08T08:38:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.269+0000 7f6116ffd640 1 -- 172.21.15.16:0/1474115636 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6118012600 con 0x7f61201608a0 2023-12-08T08:38:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.269+0000 7f6124b0f640 1 -- 172.21.15.16:0/1474115636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612014ce40 con 0x7f61201608a0 2023-12-08T08:38:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.270+0000 7f6124b0f640 1 -- 172.21.15.16:0/1474115636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61200d0f30 con 0x7f61201608a0 2023-12-08T08:38:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.270+0000 7f6116ffd640 1 -- 172.21.15.16:0/1474115636 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6118007070 con 0x7f61201608a0 2023-12-08T08:38:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.271+0000 7f6116ffd640 1 --2- 172.21.15.16:0/1474115636 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f610006ae70 0x7f610006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.271+0000 7f6116ffd640 1 -- 172.21.15.16:0/1474115636 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6118088fa0 con 0x7f61201608a0 2023-12-08T08:38:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.271+0000 7f611e59c640 1 --2- 172.21.15.16:0/1474115636 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f610006ae70 0x7f610006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.271+0000 7f611e59c640 1 --2- 172.21.15.16:0/1474115636 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f610006ae70 0x7f610006d330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6110009c30 tx=0x7f6110009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.275+0000 7f6116ffd640 1 -- 172.21.15.16:0/1474115636 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f61180bbe90 con 0x7f61201608a0 2023-12-08T08:38:13.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.489+0000 7f6124b0f640 1 -- 172.21.15.16:0/1474115636 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f612014d030 con 0x7f61201608a0 2023-12-08T08:38:13.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.489+0000 7f6116ffd640 1 -- 172.21.15.16:0/1474115636 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f6118017090 con 0x7f61201608a0 2023-12-08T08:38:13.492 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:13.493 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1424152341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1424152341},{"type":"v1","addr":"172.21.15.16:6833","nonce":1424152341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-08T08:38:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2023-12-08T08:38:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.491+0000 7f6124b0f640 1 -- 172.21.15.16:0/1474115636 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f610006ae70 msgr2=0x7f610006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.491+0000 7f6124b0f640 1 --2- 172.21.15.16:0/1474115636 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f610006ae70 0x7f610006d330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6110009c30 tx=0x7f6110009190 comp rx=0 tx=0).stop 2023-12-08T08:38:13.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.491+0000 7f6124b0f640 1 -- 172.21.15.16:0/1474115636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61201608a0 msgr2=0x7f61201f5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.491+0000 7f6124b0f640 1 --2- 172.21.15.16:0/1474115636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61201608a0 0x7f61201f5ed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f611800ca80 tx=0x7f611800cf40 comp rx=0 tx=0).stop 2023-12-08T08:38:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.491+0000 7f6124b0f640 1 -- 172.21.15.16:0/1474115636 shutdown_connections 2023-12-08T08:38:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.491+0000 7f6124b0f640 1 --2- 172.21.15.16:0/1474115636 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f610006ae70 0x7f610006d330 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.491+0000 7f6124b0f640 1 --2- 172.21.15.16:0/1474115636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61201608a0 0x7f61201f5ed0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.491+0000 7f6124b0f640 1 --2- 172.21.15.16:0/1474115636 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f61200b1d70 0x7f61201f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.491+0000 7f6124b0f640 1 --2- 172.21.15.16:0/1474115636 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f61200afce0 0x7f61200c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.491+0000 7f6124b0f640 1 -- 172.21.15.16:0/1474115636 >> 172.21.15.16:0/1474115636 conn(0x7f61200d3030 msgr2=0x7f61200b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:13.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.491+0000 7f6124b0f640 1 -- 172.21.15.16:0/1474115636 shutdown_connections 2023-12-08T08:38:13.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:13.492+0000 7f6124b0f640 1 -- 172.21.15.16:0/1474115636 wait complete. 2023-12-08T08:38:15.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-08T08:38:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.623+0000 7fdff31fc640 1 Processor -- start 2023-12-08T08:38:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.623+0000 7fdff31fc640 1 -- start start 2023-12-08T08:38:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.624+0000 7fdff31fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdfec0b3340 0x7fdfec0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.624+0000 7fdff31fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdfec0d1d00 0x7fdfec0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.624+0000 7fdff31fc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfec0cdc20 con 0x7fdfec0b3bf0 2023-12-08T08:38:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.624+0000 7fdff31fc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfec0d11d0 con 0x7fdfec0b3340 2023-12-08T08:38:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.624+0000 7fdff31fc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfec0d3a60 con 0x7fdfec0d1d00 2023-12-08T08:38:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.624+0000 7fdff0f98640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdfec0b3340 0x7fdfec0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.624+0000 7fdff0f98640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdfec0b3340 0x7fdfec0d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34436/0 (socket says 172.21.15.16:34436) 2023-12-08T08:38:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.624+0000 7fdff0f98640 1 -- 172.21.15.16:0/1016634417 learned_addr learned my addr 172.21.15.16:0/1016634417 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.624+0000 7fdff1799640 1 --2- 172.21.15.16:0/1016634417 >> v2:172.21.15.112:3300/0 conn(0x7fdfec0d1d00 0x7fdfec0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.624+0000 7fdff0f98640 1 -- 172.21.15.16:0/1016634417 >> v2:172.21.15.112:3300/0 conn(0x7fdfec0d1d00 msgr2=0x7fdfec0b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.624+0000 7fdff0f98640 1 --2- 172.21.15.16:0/1016634417 >> v2:172.21.15.112:3300/0 conn(0x7fdfec0d1d00 0x7fdfec0b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.624+0000 7fdff0f98640 1 -- 172.21.15.16:0/1016634417 >> v1:172.21.15.112:6789/0 conn(0x7fdfec0b3bf0 legacy=0x7fdfec0d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.624+0000 7fdff0f98640 1 -- 172.21.15.16:0/1016634417 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfec0d3d40 con 0x7fdfec0b3340 2023-12-08T08:38:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.625+0000 7fdff0f98640 1 --2- 172.21.15.16:0/1016634417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec0b3340 0x7fdfec0d00c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0d0600 tx=0x7fdfe002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d12180b0989e9d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.625+0000 7fdfeb7fe640 1 -- 172.21.15.16:0/1016634417 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe00317e0 con 0x7fdfec0b3340 2023-12-08T08:38:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.625+0000 7fdfeb7fe640 1 -- 172.21.15.16:0/1016634417 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe0031940 con 0x7fdfec0b3340 2023-12-08T08:38:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.625+0000 7fdfeb7fe640 1 -- 172.21.15.16:0/1016634417 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0031cd0 con 0x7fdfec0b3340 2023-12-08T08:38:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.626+0000 7fdff31fc640 1 -- 172.21.15.16:0/1016634417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec0b3340 msgr2=0x7fdfec0d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.626+0000 7fdff31fc640 1 --2- 172.21.15.16:0/1016634417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec0b3340 0x7fdfec0d00c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdfec0d0600 tx=0x7fdfe002b880 comp rx=0 tx=0).stop 2023-12-08T08:38:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.626+0000 7fdff31fc640 1 -- 172.21.15.16:0/1016634417 shutdown_connections 2023-12-08T08:38:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.626+0000 7fdff31fc640 1 --2- 172.21.15.16:0/1016634417 >> v2:172.21.15.112:3300/0 conn(0x7fdfec0d1d00 0x7fdfec0b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.626+0000 7fdff31fc640 1 --2- 172.21.15.16:0/1016634417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec0b3340 0x7fdfec0d00c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.626+0000 7fdff31fc640 1 -- 172.21.15.16:0/1016634417 >> 172.21.15.16:0/1016634417 conn(0x7fdfec09c2f0 msgr2=0x7fdfec0d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:15.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.626+0000 7fdff31fc640 1 -- 172.21.15.16:0/1016634417 shutdown_connections 2023-12-08T08:38:15.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.626+0000 7fdff31fc640 1 -- 172.21.15.16:0/1016634417 wait complete. 2023-12-08T08:38:15.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.627+0000 7fdff31fc640 1 Processor -- start 2023-12-08T08:38:15.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.627+0000 7fdff31fc640 1 -- start start 2023-12-08T08:38:15.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.628+0000 7fdff31fc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfec0b3340 0x7fdfec0c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.628+0000 7fdff31fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfec0b3bf0 0x7fdfec1562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.628+0000 7fdff31fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec0d1d00 0x7fdfec149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.628+0000 7fdff31fc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec147430 con 0x7fdfec0d1d00 2023-12-08T08:38:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.628+0000 7fdff31fc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec149ca0 con 0x7fdfec0b3340 2023-12-08T08:38:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.628+0000 7fdff31fc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfec149f80 con 0x7fdfec0b3bf0 2023-12-08T08:38:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.628+0000 7fdff1799640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec0d1d00 0x7fdfec149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.628+0000 7fdff1799640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec0d1d00 0x7fdfec149470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34440/0 (socket says 172.21.15.16:34440) 2023-12-08T08:38:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.628+0000 7fdff1799640 1 -- 172.21.15.16:0/1456450977 learned_addr learned my addr 172.21.15.16:0/1456450977 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.628+0000 7fdff0f98640 1 --2- 172.21.15.16:0/1456450977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfec0b3340 0x7fdfec0c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.628+0000 7fdfebfff640 1 --2- 172.21.15.16:0/1456450977 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfec0b3bf0 0x7fdfec1562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.629+0000 7fdff1799640 1 -- 172.21.15.16:0/1456450977 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfec0b3bf0 msgr2=0x7fdfec1562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.629+0000 7fdff1799640 1 --2- 172.21.15.16:0/1456450977 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfec0b3bf0 0x7fdfec1562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.629+0000 7fdff1799640 1 -- 172.21.15.16:0/1456450977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfec0b3340 msgr2=0x7fdfec0c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.629+0000 7fdff1799640 1 --2- 172.21.15.16:0/1456450977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfec0b3340 0x7fdfec0c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.629+0000 7fdff1799640 1 -- 172.21.15.16:0/1456450977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe00094d0 con 0x7fdfec0d1d00 2023-12-08T08:38:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.629+0000 7fdff1799640 1 --2- 172.21.15.16:0/1456450977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec0d1d00 0x7fdfec149470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4007e60 tx=0x7fdfe400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.629+0000 7fdfe97fa640 1 -- 172.21.15.16:0/1456450977 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4014420 con 0x7fdfec0d1d00 2023-12-08T08:38:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.629+0000 7fdfe97fa640 1 -- 172.21.15.16:0/1456450977 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe40145a0 con 0x7fdfec0d1d00 2023-12-08T08:38:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.629+0000 7fdff31fc640 1 -- 172.21.15.16:0/1456450977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfec14a260 con 0x7fdfec0d1d00 2023-12-08T08:38:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.629+0000 7fdfe97fa640 1 -- 172.21.15.16:0/1456450977 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfe4014950 con 0x7fdfec0d1d00 2023-12-08T08:38:15.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.629+0000 7fdff31fc640 1 -- 172.21.15.16:0/1456450977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfec1567e0 con 0x7fdfec0d1d00 2023-12-08T08:38:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.630+0000 7fdff31fc640 1 -- 172.21.15.16:0/1456450977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfec1478a0 con 0x7fdfec0d1d00 2023-12-08T08:38:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.631+0000 7fdfe97fa640 1 -- 172.21.15.16:0/1456450977 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdfe4014ab0 con 0x7fdfec0d1d00 2023-12-08T08:38:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.631+0000 7fdfe97fa640 1 --2- 172.21.15.16:0/1456450977 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdfd006ae40 0x7fdfd006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.631+0000 7fdfe97fa640 1 -- 172.21.15.16:0/1456450977 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdfe4089aa0 con 0x7fdfec0d1d00 2023-12-08T08:38:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.631+0000 7fdff0f98640 1 --2- 172.21.15.16:0/1456450977 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdfd006ae40 0x7fdfd006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:15.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.632+0000 7fdff0f98640 1 --2- 172.21.15.16:0/1456450977 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdfd006ae40 0x7fdfd006d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0036000 tx=0x7fdfe003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:15.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.636+0000 7fdfe97fa640 1 -- 172.21.15.16:0/1456450977 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdfec1478a0 con 0x7fdfec0d1d00 2023-12-08T08:38:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.850+0000 7fdff31fc640 1 -- 172.21.15.16:0/1456450977 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfec1478a0 con 0x7fdfec0d1d00 2023-12-08T08:38:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.851+0000 7fdfe97fa640 1 -- 172.21.15.16:0/1456450977 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7fdfec1478a0 con 0x7fdfec0d1d00 2023-12-08T08:38:15.853 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:15.853 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/307039451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":307039451},{"type":"v1","addr":"172.21.15.112:6836","nonce":307039451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1424152341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1424152341},{"type":"v1","addr":"172.21.15.16:6833","nonce":1424152341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3597811670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3597811670},{"type":"v1","addr":"172.21.15.16:6835","nonce":3597811670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4297,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1082241707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1082241707},{"type":"v1","addr":"172.21.15.112:6837","nonce":1082241707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-12-08T08:38:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 3 2023-12-08T08:38:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.853+0000 7fdff31fc640 1 -- 172.21.15.16:0/1456450977 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdfd006ae40 msgr2=0x7fdfd006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.853+0000 7fdff31fc640 1 --2- 172.21.15.16:0/1456450977 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdfd006ae40 0x7fdfd006d300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0036000 tx=0x7fdfe003d040 comp rx=0 tx=0).stop 2023-12-08T08:38:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.853+0000 7fdff31fc640 1 -- 172.21.15.16:0/1456450977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec0d1d00 msgr2=0x7fdfec149470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.853+0000 7fdff31fc640 1 --2- 172.21.15.16:0/1456450977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec0d1d00 0x7fdfec149470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4007e60 tx=0x7fdfe400e4b0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.853+0000 7fdff31fc640 1 -- 172.21.15.16:0/1456450977 shutdown_connections 2023-12-08T08:38:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.853+0000 7fdff31fc640 1 --2- 172.21.15.16:0/1456450977 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdfd006ae40 0x7fdfd006d300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.853+0000 7fdff31fc640 1 --2- 172.21.15.16:0/1456450977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfec0d1d00 0x7fdfec149470 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.853+0000 7fdff31fc640 1 --2- 172.21.15.16:0/1456450977 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdfec0b3bf0 0x7fdfec1562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.853+0000 7fdff31fc640 1 --2- 172.21.15.16:0/1456450977 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdfec0b3340 0x7fdfec0c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.853+0000 7fdff31fc640 1 -- 172.21.15.16:0/1456450977 >> 172.21.15.16:0/1456450977 conn(0x7fdfec09c2f0 msgr2=0x7fdfec0b6040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.853+0000 7fdff31fc640 1 -- 172.21.15.16:0/1456450977 shutdown_connections 2023-12-08T08:38:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.853+0000 7fdff31fc640 1 -- 172.21.15.16:0/1456450977 wait complete. 2023-12-08T08:38:15.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.970+0000 7f0fdf0e5640 1 Processor -- start 2023-12-08T08:38:15.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.970+0000 7f0fdf0e5640 1 -- start start 2023-12-08T08:38:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.970+0000 7f0fdf0e5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fd80cf380 0x7f0fd806a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.970+0000 7f0fdf0e5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0fd815e150 0x7f0fd8161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.970+0000 7f0fdf0e5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd806b070 con 0x7f0fd80cf760 2023-12-08T08:38:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.970+0000 7f0fdf0e5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd8072290 con 0x7f0fd815e150 2023-12-08T08:38:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.970+0000 7f0fdf0e5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80d0d00 con 0x7f0fd80cf380 2023-12-08T08:38:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.971+0000 7f0fdce81640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fd80cf380 0x7f0fd806a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.971+0000 7f0fdd682640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0fd815e150 0x7f0fd8161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.971+0000 7f0fdce81640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fd80cf380 0x7f0fd806a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:42068/0 (socket says 172.21.15.16:42068) 2023-12-08T08:38:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.971+0000 7f0fdce81640 1 -- 172.21.15.16:0/2045915289 learned_addr learned my addr 172.21.15.16:0/2045915289 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.971+0000 7f0fdce81640 1 -- 172.21.15.16:0/2045915289 >> v2:172.21.15.112:3300/0 conn(0x7f0fd815e150 msgr2=0x7f0fd8161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:15.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.971+0000 7f0fdce81640 1 --2- 172.21.15.16:0/2045915289 >> v2:172.21.15.112:3300/0 conn(0x7f0fd815e150 0x7f0fd8161690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.971+0000 7f0fdce81640 1 -- 172.21.15.16:0/2045915289 >> v1:172.21.15.112:6789/0 conn(0x7f0fd80cf760 legacy=0x7f0fd8071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:15.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.971+0000 7f0fdce81640 1 -- 172.21.15.16:0/2045915289 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd806aab0 con 0x7f0fd80cf380 2023-12-08T08:38:15.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.971+0000 7f0fdd682640 1 --2- 172.21.15.16:0/2045915289 >> v2:172.21.15.112:3300/0 conn(0x7f0fd815e150 0x7f0fd8161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:15.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.972+0000 7f0fdce81640 1 --2- 172.21.15.16:0/2045915289 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fd80cf380 0x7f0fd806a4d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0009e50 tx=0x7f0fd002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b48dcd2cc128af server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:15.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.972+0000 7f0fd77fe640 1 -- 172.21.15.16:0/2045915289 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd002ca70 con 0x7f0fd80cf380 2023-12-08T08:38:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.972+0000 7f0fd77fe640 1 -- 172.21.15.16:0/2045915289 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd002cbd0 con 0x7f0fd80cf380 2023-12-08T08:38:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.972+0000 7f0fd77fe640 1 -- 172.21.15.16:0/2045915289 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00343f0 con 0x7f0fd80cf380 2023-12-08T08:38:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.973+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/2045915289 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fd80cf380 msgr2=0x7f0fd806a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.973+0000 7f0fdf0e5640 1 --2- 172.21.15.16:0/2045915289 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fd80cf380 0x7f0fd806a4d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0009e50 tx=0x7f0fd002bed0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.973+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/2045915289 shutdown_connections 2023-12-08T08:38:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.973+0000 7f0fdf0e5640 1 --2- 172.21.15.16:0/2045915289 >> v2:172.21.15.112:3300/0 conn(0x7f0fd815e150 0x7f0fd8161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.973+0000 7f0fdf0e5640 1 --2- 172.21.15.16:0/2045915289 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fd80cf380 0x7f0fd806a4d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.973+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/2045915289 >> 172.21.15.16:0/2045915289 conn(0x7f0fd80cfda0 msgr2=0x7f0fd80d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.973+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/2045915289 shutdown_connections 2023-12-08T08:38:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.973+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/2045915289 wait complete. 2023-12-08T08:38:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.974+0000 7f0fdf0e5640 1 Processor -- start 2023-12-08T08:38:15.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.974+0000 7f0fdf0e5640 1 -- start start 2023-12-08T08:38:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdf0e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd80cf380 0x7f0fd8155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdf0e5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fd80cf760 0x7f0fd80b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdf0e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fd815e150 0x7f0fd8159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdf0e5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd8155870 con 0x7f0fd80cf380 2023-12-08T08:38:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdf0e5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80b32e0 con 0x7f0fd815e150 2023-12-08T08:38:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdf0e5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80b35c0 con 0x7f0fd80cf760 2023-12-08T08:38:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdce81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd80cf380 0x7f0fd8155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdce81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd80cf380 0x7f0fd8155330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34444/0 (socket says 172.21.15.16:34444) 2023-12-08T08:38:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdce81640 1 -- 172.21.15.16:0/1277661696 learned_addr learned my addr 172.21.15.16:0/1277661696 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdd682640 1 --2- 172.21.15.16:0/1277661696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fd815e150 0x7f0fd8159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:15.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fd7fff640 1 --2- 172.21.15.16:0/1277661696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fd80cf760 0x7f0fd80b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:15.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdce81640 1 -- 172.21.15.16:0/1277661696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fd80cf760 msgr2=0x7f0fd80b29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:15.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdce81640 1 --2- 172.21.15.16:0/1277661696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fd80cf760 0x7f0fd80b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdce81640 1 -- 172.21.15.16:0/1277661696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fd815e150 msgr2=0x7f0fd8159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:15.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdce81640 1 --2- 172.21.15.16:0/1277661696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fd815e150 0x7f0fd8159bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:15.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.975+0000 7f0fdce81640 1 -- 172.21.15.16:0/1277661696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc0009400 con 0x7f0fd80cf380 2023-12-08T08:38:15.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.976+0000 7f0fdce81640 1 --2- 172.21.15.16:0/1277661696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd80cf380 0x7f0fd8155330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0000f80 tx=0x7f0fd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:15.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.976+0000 7f0fd57fa640 1 -- 172.21.15.16:0/1277661696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0009510 con 0x7f0fd80cf380 2023-12-08T08:38:15.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.976+0000 7f0fd57fa640 1 -- 172.21.15.16:0/1277661696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd003e450 con 0x7f0fd80cf380 2023-12-08T08:38:15.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.976+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/1277661696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd0009b30 con 0x7f0fd80cf380 2023-12-08T08:38:15.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.976+0000 7f0fd57fa640 1 -- 172.21.15.16:0/1277661696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd003e800 con 0x7f0fd80cf380 2023-12-08T08:38:15.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.976+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/1277661696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd80b3b80 con 0x7f0fd80cf380 2023-12-08T08:38:15.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.977+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/1277661696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd8067430 con 0x7f0fd80cf380 2023-12-08T08:38:15.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.977+0000 7f0fd57fa640 1 -- 172.21.15.16:0/1277661696 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0fd003e960 con 0x7f0fd80cf380 2023-12-08T08:38:15.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.977+0000 7f0fd57fa640 1 --2- 172.21.15.16:0/1277661696 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0fb806ae70 0x7f0fb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:15.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.978+0000 7f0fd57fa640 1 -- 172.21.15.16:0/1277661696 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0fd00abdb0 con 0x7f0fd80cf380 2023-12-08T08:38:15.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.978+0000 7f0fd7fff640 1 --2- 172.21.15.16:0/1277661696 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0fb806ae70 0x7f0fb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:15.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.978+0000 7f0fd7fff640 1 --2- 172.21.15.16:0/1277661696 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0fb806ae70 0x7f0fb806d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8160f40 tx=0x7f0fc8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:15.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:15.982+0000 7f0fd57fa640 1 -- 172.21.15.16:0/1277661696 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0fd00771c0 con 0x7f0fd80cf380 2023-12-08T08:38:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.156+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/1277661696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fd806a600 con 0x7f0fd80cf380 2023-12-08T08:38:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.157+0000 7f0fd57fa640 1 -- 172.21.15.16:0/1277661696 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f0fd007a7e0 con 0x7f0fd80cf380 2023-12-08T08:38:16.159 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:16.159 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:09.114424+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.159+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/1277661696 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0fb806ae70 msgr2=0x7f0fb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.159+0000 7f0fdf0e5640 1 --2- 172.21.15.16:0/1277661696 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0fb806ae70 0x7f0fb806d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0fd8160f40 tx=0x7f0fc8009040 comp rx=0 tx=0).stop 2023-12-08T08:38:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.159+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/1277661696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd80cf380 msgr2=0x7f0fd8155330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.159+0000 7f0fdf0e5640 1 --2- 172.21.15.16:0/1277661696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd80cf380 0x7f0fd8155330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0000f80 tx=0x7f0fd0004a40 comp rx=0 tx=0).stop 2023-12-08T08:38:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.159+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/1277661696 shutdown_connections 2023-12-08T08:38:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.159+0000 7f0fdf0e5640 1 --2- 172.21.15.16:0/1277661696 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0fb806ae70 0x7f0fb806d330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.159+0000 7f0fdf0e5640 1 --2- 172.21.15.16:0/1277661696 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0fd815e150 0x7f0fd8159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.159+0000 7f0fdf0e5640 1 --2- 172.21.15.16:0/1277661696 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0fd80cf760 0x7f0fd80b29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.159+0000 7f0fdf0e5640 1 --2- 172.21.15.16:0/1277661696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0fd80cf380 0x7f0fd8155330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.159+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/1277661696 >> 172.21.15.16:0/1277661696 conn(0x7f0fd80cfda0 msgr2=0x7f0fd80d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:16.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.159+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/1277661696 shutdown_connections 2023-12-08T08:38:16.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.159+0000 7f0fdf0e5640 1 -- 172.21.15.16:0/1277661696 wait complete. 2023-12-08T08:38:16.171 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-08T08:38:16.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:38:16.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.279+0000 7f2528141640 1 Processor -- start 2023-12-08T08:38:16.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.279+0000 7f2528141640 1 -- start start 2023-12-08T08:38:16.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.280+0000 7f2528141640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f25200cfe10 0x7f25200b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.280+0000 7f2528141640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25200c6be0 con 0x7f25200b4150 2023-12-08T08:38:16.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.280+0000 7f2528141640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25200d3ff0 con 0x7f25200cfa30 2023-12-08T08:38:16.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.280+0000 7f2528141640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25200b4860 con 0x7f25200cfe10 2023-12-08T08:38:16.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.280+0000 7f25256dc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f25200b4150 0x7f25200cdf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:46492/0 (socket says 172.21.15.16:46492) 2023-12-08T08:38:16.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.280+0000 7f25256dc640 1 -- 172.21.15.16:0/1921484798 learned_addr learned my addr 172.21.15.16:0/1921484798 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:16.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.280+0000 7f25266de640 1 --2- 172.21.15.16:0/1921484798 >> v2:172.21.15.112:3300/0 conn(0x7f25200cfe10 0x7f25200b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:16.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.280+0000 7f2524edb640 1 -- 172.21.15.16:0/1921484798 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637113489 0 0) 0x7f25200c6be0 con 0x7f25200b4150 2023-12-08T08:38:16.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.281+0000 7f2524edb640 1 -- 172.21.15.16:0/1921484798 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2508003520 con 0x7f25200b4150 2023-12-08T08:38:16.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.281+0000 7f25266de640 1 -- 172.21.15.16:0/1921484798 >> v1:172.21.15.112:6789/0 conn(0x7f25200cfa30 legacy=0x7f25200b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.281+0000 7f25266de640 1 -- 172.21.15.16:0/1921484798 >> v1:172.21.15.16:6789/0 conn(0x7f25200b4150 legacy=0x7f25200cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.281+0000 7f25266de640 1 -- 172.21.15.16:0/1921484798 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25200d3d50 con 0x7f25200cfe10 2023-12-08T08:38:16.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.281+0000 7f25266de640 1 --2- 172.21.15.16:0/1921484798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25200cfe10 0x7f25200b3630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25180097a0 tx=0x7f251802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=157dd94c1c930baa server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:16.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.282+0000 7f2524edb640 1 -- 172.21.15.16:0/1921484798 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518031830 con 0x7f25200cfe10 2023-12-08T08:38:16.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.282+0000 7f2524edb640 1 -- 172.21.15.16:0/1921484798 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2518031990 con 0x7f25200cfe10 2023-12-08T08:38:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.282+0000 7f2524edb640 1 -- 172.21.15.16:0/1921484798 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518031d20 con 0x7f25200cfe10 2023-12-08T08:38:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.282+0000 7f2528141640 1 -- 172.21.15.16:0/1921484798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25200cfe10 msgr2=0x7f25200b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.282+0000 7f2528141640 1 --2- 172.21.15.16:0/1921484798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25200cfe10 0x7f25200b3630 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25180097a0 tx=0x7f251802b8d0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.282+0000 7f2528141640 1 -- 172.21.15.16:0/1921484798 shutdown_connections 2023-12-08T08:38:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.282+0000 7f2528141640 1 --2- 172.21.15.16:0/1921484798 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25200cfe10 0x7f25200b3630 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.282+0000 7f2528141640 1 -- 172.21.15.16:0/1921484798 >> 172.21.15.16:0/1921484798 conn(0x7f25200d3090 msgr2=0x7f25200d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.282+0000 7f2528141640 1 -- 172.21.15.16:0/1921484798 shutdown_connections 2023-12-08T08:38:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.282+0000 7f2528141640 1 -- 172.21.15.16:0/1921484798 wait complete. 2023-12-08T08:38:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.283+0000 7f2528141640 1 Processor -- start 2023-12-08T08:38:16.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.284+0000 7f2528141640 1 -- start start 2023-12-08T08:38:16.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.284+0000 7f2528141640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25200b4150 0x7f25200b66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.284+0000 7f2528141640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25200cfa30 0x7f25200b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.284+0000 7f2528141640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25200cfe10 0x7f2520158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.284+0000 7f2528141640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25200b8860 con 0x7f25200b4150 2023-12-08T08:38:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.284+0000 7f2528141640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25200b8b40 con 0x7f25200cfa30 2023-12-08T08:38:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.284+0000 7f2528141640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252015ef50 con 0x7f25200cfe10 2023-12-08T08:38:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.284+0000 7f2525edd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25200b4150 0x7f25200b66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.284+0000 7f2525edd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25200b4150 0x7f25200b66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34448/0 (socket says 172.21.15.16:34448) 2023-12-08T08:38:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.284+0000 7f2525edd640 1 -- 172.21.15.16:0/2855283774 learned_addr learned my addr 172.21.15.16:0/2855283774 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.285+0000 7f25256dc640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25200cfa30 0x7f25200b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.285+0000 7f25266de640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25200cfe10 0x7f2520158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.285+0000 7f2525edd640 1 -- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25200cfe10 msgr2=0x7f2520158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.285+0000 7f2525edd640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25200cfe10 0x7f2520158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.285+0000 7f2525edd640 1 -- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25200cfa30 msgr2=0x7f25200b7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.285+0000 7f2525edd640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25200cfa30 0x7f25200b7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.285+0000 7f2525edd640 1 -- 172.21.15.16:0/2855283774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2518009400 con 0x7f25200b4150 2023-12-08T08:38:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.285+0000 7f25266de640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25200cfe10 0x7f2520158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.285+0000 7f2525edd640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25200b4150 0x7f25200b66b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f251000c790 tx=0x7f251000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.285+0000 7f251e7fc640 1 -- 172.21.15.16:0/2855283774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251000cea0 con 0x7f25200b4150 2023-12-08T08:38:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.285+0000 7f251e7fc640 1 -- 172.21.15.16:0/2855283774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2510006d50 con 0x7f25200b4150 2023-12-08T08:38:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.285+0000 7f251e7fc640 1 -- 172.21.15.16:0/2855283774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510003bb0 con 0x7f25200b4150 2023-12-08T08:38:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.285+0000 7f2528141640 1 -- 172.21.15.16:0/2855283774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252015f230 con 0x7f25200b4150 2023-12-08T08:38:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.285+0000 7f2528141640 1 -- 172.21.15.16:0/2855283774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25200caf10 con 0x7f25200b4150 2023-12-08T08:38:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.286+0000 7f2528141640 1 -- 172.21.15.16:0/2855283774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252015edb0 con 0x7f25200b4150 2023-12-08T08:38:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.287+0000 7f251e7fc640 1 -- 172.21.15.16:0/2855283774 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2510003d10 con 0x7f25200b4150 2023-12-08T08:38:16.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.287+0000 7f251e7fc640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f250c06ae40 0x7f250c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.287+0000 7f251e7fc640 1 -- 172.21.15.16:0/2855283774 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2510088da0 con 0x7f25200b4150 2023-12-08T08:38:16.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.287+0000 7f25256dc640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f250c06ae40 0x7f250c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:16.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.288+0000 7f25256dc640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f250c06ae40 0x7f250c06d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2520158710 tx=0x7f2514007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:16.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.292+0000 7f251e7fc640 1 -- 172.21.15.16:0/2855283774 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f252015edb0 con 0x7f25200b4150 2023-12-08T08:38:16.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.465+0000 7f2528141640 1 -- 172.21.15.16:0/2855283774 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f252015edb0 con 0x7f25200b4150 2023-12-08T08:38:16.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.466+0000 7f251e7fc640 1 -- 172.21.15.16:0/2855283774 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f252015edb0 con 0x7f25200b4150 2023-12-08T08:38:16.468 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:38:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.467+0000 7f2528141640 1 -- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f250c06ae40 msgr2=0x7f250c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.467+0000 7f2528141640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f250c06ae40 0x7f250c06d300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2520158710 tx=0x7f2514007d30 comp rx=0 tx=0).stop 2023-12-08T08:38:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.467+0000 7f2528141640 1 -- 172.21.15.16:0/2855283774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25200b4150 msgr2=0x7f25200b66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.467+0000 7f2528141640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25200b4150 0x7f25200b66b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f251000c790 tx=0x7f251000cc50 comp rx=0 tx=0).stop 2023-12-08T08:38:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.467+0000 7f2528141640 1 -- 172.21.15.16:0/2855283774 shutdown_connections 2023-12-08T08:38:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.467+0000 7f2528141640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f250c06ae40 0x7f250c06d300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.467+0000 7f2528141640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25200cfe10 0x7f2520158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.467+0000 7f2528141640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25200cfa30 0x7f25200b7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.467+0000 7f2528141640 1 --2- 172.21.15.16:0/2855283774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25200b4150 0x7f25200b66b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.467+0000 7f2528141640 1 -- 172.21.15.16:0/2855283774 >> 172.21.15.16:0/2855283774 conn(0x7f25200d3090 msgr2=0x7f25200d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.467+0000 7f2528141640 1 -- 172.21.15.16:0/2855283774 shutdown_connections 2023-12-08T08:38:16.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.467+0000 7f2528141640 1 -- 172.21.15.16:0/2855283774 wait complete. 2023-12-08T08:38:16.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-08T08:38:16.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.584+0000 7f9b9b1b6640 1 Processor -- start 2023-12-08T08:38:16.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.584+0000 7f9b9b1b6640 1 -- start start 2023-12-08T08:38:16.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.585+0000 7f9b9b1b6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b9415d580 0x7f9b940d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.585+0000 7f9b9b1b6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9415ee90 con 0x7f9b9415f890 2023-12-08T08:38:16.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.585+0000 7f9b9b1b6640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b940d0fe0 con 0x7f9b940d3ad0 2023-12-08T08:38:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.585+0000 7f9b9b1b6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b940d1280 con 0x7f9b9415d580 2023-12-08T08:38:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.585+0000 7f9b98f52640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b9415d580 0x7f9b940d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.585+0000 7f9b99753640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9b9415f890 0x7f9b9415d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57302/0 (socket says 172.21.15.16:57302) 2023-12-08T08:38:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.585+0000 7f9b99753640 1 -- 172.21.15.16:0/2110932798 learned_addr learned my addr 172.21.15.16:0/2110932798 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.585+0000 7f9b98f52640 1 -- 172.21.15.16:0/2110932798 >> v1:172.21.15.112:6789/0 conn(0x7f9b940d3ad0 legacy=0x7f9b9415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.585+0000 7f9b98f52640 1 -- 172.21.15.16:0/2110932798 >> v1:172.21.15.16:6789/0 conn(0x7f9b9415f890 legacy=0x7f9b9415d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.585+0000 7f9b98f52640 1 -- 172.21.15.16:0/2110932798 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b940d1560 con 0x7f9b9415d580 2023-12-08T08:38:16.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.586+0000 7f9b98f52640 1 --2- 172.21.15.16:0/2110932798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9415d580 0x7f9b940d1d60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b840097d0 tx=0x7f9b8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e27177478653ca server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:16.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.586+0000 7f9b937fe640 1 -- 172.21.15.16:0/2110932798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84031830 con 0x7f9b9415d580 2023-12-08T08:38:16.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.586+0000 7f9b937fe640 1 -- 172.21.15.16:0/2110932798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b84031990 con 0x7f9b9415d580 2023-12-08T08:38:16.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.586+0000 7f9b937fe640 1 -- 172.21.15.16:0/2110932798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84031d20 con 0x7f9b9415d580 2023-12-08T08:38:16.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.587+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/2110932798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9415d580 msgr2=0x7f9b940d1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.587+0000 7f9b9b1b6640 1 --2- 172.21.15.16:0/2110932798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9415d580 0x7f9b940d1d60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9b840097d0 tx=0x7f9b8402b8d0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.587+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/2110932798 shutdown_connections 2023-12-08T08:38:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.587+0000 7f9b9b1b6640 1 --2- 172.21.15.16:0/2110932798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9415d580 0x7f9b940d1d60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.587+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/2110932798 >> 172.21.15.16:0/2110932798 conn(0x7f9b940d2fb0 msgr2=0x7f9b940d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.587+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/2110932798 shutdown_connections 2023-12-08T08:38:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.587+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/2110932798 wait complete. 2023-12-08T08:38:16.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.588+0000 7f9b9b1b6640 1 Processor -- start 2023-12-08T08:38:16.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.588+0000 7f9b9b1b6640 1 -- start start 2023-12-08T08:38:16.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b9b1b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b940d3ad0 0x7f9b940cc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b9b1b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9415d580 0x7f9b940b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b9b1b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b9415f890 0x7f9b940c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b9b1b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b940c9220 con 0x7f9b9415d580 2023-12-08T08:38:16.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b9b1b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b941f65b0 con 0x7f9b940d3ad0 2023-12-08T08:38:16.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b9b1b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b941f3570 con 0x7f9b9415f890 2023-12-08T08:38:16.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9415d580 0x7f9b940b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b93fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9415d580 0x7f9b940b5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56398/0 (socket says 172.21.15.16:56398) 2023-12-08T08:38:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b93fff640 1 -- 172.21.15.16:0/3646506756 learned_addr learned my addr 172.21.15.16:0/3646506756 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b93fff640 1 -- 172.21.15.16:0/3646506756 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b9415f890 msgr2=0x7f9b940c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b93fff640 1 --2- 172.21.15.16:0/3646506756 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b9415f890 0x7f9b940c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b93fff640 1 -- 172.21.15.16:0/3646506756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b940d3ad0 msgr2=0x7f9b940cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b98f52640 1 --2- 172.21.15.16:0/3646506756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b940d3ad0 0x7f9b940cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b93fff640 1 --2- 172.21.15.16:0/3646506756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b940d3ad0 0x7f9b940cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b93fff640 1 -- 172.21.15.16:0/3646506756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b84009400 con 0x7f9b9415d580 2023-12-08T08:38:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b93fff640 1 --2- 172.21.15.16:0/3646506756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9415d580 0x7f9b940b5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800c430 tx=0x7f9b8800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b917fa640 1 -- 172.21.15.16:0/3646506756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b880052d0 con 0x7f9b9415d580 2023-12-08T08:38:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.589+0000 7f9b917fa640 1 -- 172.21.15.16:0/3646506756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b88005450 con 0x7f9b9415d580 2023-12-08T08:38:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.590+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/3646506756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b941f38b0 con 0x7f9b9415d580 2023-12-08T08:38:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.590+0000 7f9b917fa640 1 -- 172.21.15.16:0/3646506756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b88007260 con 0x7f9b9415d580 2023-12-08T08:38:16.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.590+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/3646506756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b941f3e10 con 0x7f9b9415d580 2023-12-08T08:38:16.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.591+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/3646506756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b941f6780 con 0x7f9b9415d580 2023-12-08T08:38:16.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.591+0000 7f9b917fa640 1 -- 172.21.15.16:0/3646506756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9b880073c0 con 0x7f9b9415d580 2023-12-08T08:38:16.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.591+0000 7f9b917fa640 1 --2- 172.21.15.16:0/3646506756 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b8006ae40 0x7f9b8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.592+0000 7f9b917fa640 1 -- 172.21.15.16:0/3646506756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9b88088650 con 0x7f9b9415d580 2023-12-08T08:38:16.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.592+0000 7f9b98f52640 1 --2- 172.21.15.16:0/3646506756 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b8006ae40 0x7f9b8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:16.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.592+0000 7f9b98f52640 1 --2- 172.21.15.16:0/3646506756 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b8006ae40 0x7f9b8006d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9b84036000 tx=0x7f9b84008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:16.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.596+0000 7f9b917fa640 1 -- 172.21.15.16:0/3646506756 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b941f6780 con 0x7f9b9415d580 2023-12-08T08:38:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.774+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/3646506756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9b941f6780 con 0x7f9b9415d580 2023-12-08T08:38:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.775+0000 7f9b917fa640 1 -- 172.21.15.16:0/3646506756 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9b941f6780 con 0x7f9b9415d580 2023-12-08T08:38:16.777 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:16.778 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-12-08T08:38:16.778 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:38:16.778 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-12-08T08:38:16.778 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-12-08T08:38:16.778 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:38:16.778 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-12-08T08:38:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.776+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/3646506756 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b8006ae40 msgr2=0x7f9b8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.776+0000 7f9b9b1b6640 1 --2- 172.21.15.16:0/3646506756 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b8006ae40 0x7f9b8006d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9b84036000 tx=0x7f9b84008030 comp rx=0 tx=0).stop 2023-12-08T08:38:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.776+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/3646506756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9415d580 msgr2=0x7f9b940b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.776+0000 7f9b9b1b6640 1 --2- 172.21.15.16:0/3646506756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9415d580 0x7f9b940b5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800c430 tx=0x7f9b8800c8f0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.776+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/3646506756 shutdown_connections 2023-12-08T08:38:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.776+0000 7f9b9b1b6640 1 --2- 172.21.15.16:0/3646506756 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b8006ae40 0x7f9b8006d300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.776+0000 7f9b9b1b6640 1 --2- 172.21.15.16:0/3646506756 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b9415f890 0x7f9b940c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.776+0000 7f9b9b1b6640 1 --2- 172.21.15.16:0/3646506756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b9415d580 0x7f9b940b5050 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.776+0000 7f9b9b1b6640 1 --2- 172.21.15.16:0/3646506756 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b940d3ad0 0x7f9b940cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.776+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/3646506756 >> 172.21.15.16:0/3646506756 conn(0x7f9b940d2fb0 msgr2=0x7f9b940d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.776+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/3646506756 shutdown_connections 2023-12-08T08:38:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.776+0000 7f9b9b1b6640 1 -- 172.21.15.16:0/3646506756 wait complete. 2023-12-08T08:38:16.790 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-08T08:38:16.791 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-08T08:38:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.901+0000 7f85807dc640 1 Processor -- start 2023-12-08T08:38:16.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.901+0000 7f85807dc640 1 -- start start 2023-12-08T08:38:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.902+0000 7f85807dc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f857806cac0 0x7f857815cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.902+0000 7f85807dc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85780ce900 con 0x7f85780cfd30 2023-12-08T08:38:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.902+0000 7f85807dc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857806bfd0 con 0x7f85780cf950 2023-12-08T08:38:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.902+0000 7f85807dc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85780cdae0 con 0x7f857806cac0 2023-12-08T08:38:16.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.902+0000 7f857dd77640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f85780cfd30 0x7f857806c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57306/0 (socket says 172.21.15.16:57306) 2023-12-08T08:38:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.902+0000 7f857dd77640 1 -- 172.21.15.16:0/1328585970 learned_addr learned my addr 172.21.15.16:0/1328585970 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.902+0000 7f857ed79640 1 --2- 172.21.15.16:0/1328585970 >> v2:172.21.15.112:3300/0 conn(0x7f857806cac0 0x7f857815cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.903+0000 7f857d576640 1 -- 172.21.15.16:0/1328585970 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1876143436 0 0) 0x7f85780ce900 con 0x7f85780cfd30 2023-12-08T08:38:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.903+0000 7f857d576640 1 -- 172.21.15.16:0/1328585970 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8560003520 con 0x7f85780cfd30 2023-12-08T08:38:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.903+0000 7f857d576640 1 -- 172.21.15.16:0/1328585970 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199936620 0 0) 0x7f857806bfd0 con 0x7f85780cf950 2023-12-08T08:38:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.903+0000 7f857d576640 1 -- 172.21.15.16:0/1328585970 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85780ce900 con 0x7f85780cf950 2023-12-08T08:38:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.903+0000 7f857d576640 1 -- 172.21.15.16:0/1328585970 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1691945312 0 0) 0x7f8560003520 con 0x7f85780cfd30 2023-12-08T08:38:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.903+0000 7f857ed79640 1 -- 172.21.15.16:0/1328585970 >> v1:172.21.15.112:6789/0 conn(0x7f85780cf950 legacy=0x7f85780d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.903+0000 7f857ed79640 1 -- 172.21.15.16:0/1328585970 >> v1:172.21.15.16:6789/0 conn(0x7f85780cfd30 legacy=0x7f857806c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.903+0000 7f857ed79640 1 -- 172.21.15.16:0/1328585970 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85780cdcf0 con 0x7f857806cac0 2023-12-08T08:38:16.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.904+0000 7f857ed79640 1 --2- 172.21.15.16:0/1328585970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f857806cac0 0x7f857815cc20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8568009d90 tx=0x7f856802bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8936520c75eefa7 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:16.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.904+0000 7f857d576640 1 -- 172.21.15.16:0/1328585970 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856c001670 con 0x7f857806cac0 2023-12-08T08:38:16.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.904+0000 7f857d576640 1 -- 172.21.15.16:0/1328585970 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856802c8b0 con 0x7f857806cac0 2023-12-08T08:38:16.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.904+0000 7f857d576640 1 -- 172.21.15.16:0/1328585970 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856802cc20 con 0x7f857806cac0 2023-12-08T08:38:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.904+0000 7f85807dc640 1 -- 172.21.15.16:0/1328585970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f857806cac0 msgr2=0x7f857815cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.904+0000 7f85807dc640 1 --2- 172.21.15.16:0/1328585970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f857806cac0 0x7f857815cc20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8568009d90 tx=0x7f856802bd80 comp rx=0 tx=0).stop 2023-12-08T08:38:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.904+0000 7f85807dc640 1 -- 172.21.15.16:0/1328585970 shutdown_connections 2023-12-08T08:38:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.904+0000 7f85807dc640 1 --2- 172.21.15.16:0/1328585970 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f857806cac0 0x7f857815cc20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.904+0000 7f85807dc640 1 -- 172.21.15.16:0/1328585970 >> 172.21.15.16:0/1328585970 conn(0x7f857809c2f0 msgr2=0x7f85780d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.904+0000 7f85807dc640 1 -- 172.21.15.16:0/1328585970 shutdown_connections 2023-12-08T08:38:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.905+0000 7f85807dc640 1 -- 172.21.15.16:0/1328585970 wait complete. 2023-12-08T08:38:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.906+0000 7f85807dc640 1 Processor -- start 2023-12-08T08:38:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.906+0000 7f85807dc640 1 -- start start 2023-12-08T08:38:16.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.906+0000 7f85807dc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f857806cac0 0x7f8578156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.906+0000 7f85807dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85780cf950 0x7f85780ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.906+0000 7f85807dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85780cfd30 0x7f8578148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.906+0000 7f85807dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f857815a050 con 0x7f85780cfd30 2023-12-08T08:38:16.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.906+0000 7f85807dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8578159480 con 0x7f85780cf950 2023-12-08T08:38:16.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.906+0000 7f85807dc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8578159720 con 0x7f857806cac0 2023-12-08T08:38:16.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.906+0000 7f857ed79640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85780cfd30 0x7f8578148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:16.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.906+0000 7f857ed79640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85780cfd30 0x7f8578148fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56406/0 (socket says 172.21.15.16:56406) 2023-12-08T08:38:16.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.906+0000 7f857ed79640 1 -- 172.21.15.16:0/950746446 learned_addr learned my addr 172.21.15.16:0/950746446 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:16.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.907+0000 7f857ed79640 1 -- 172.21.15.16:0/950746446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f857806cac0 msgr2=0x7f8578156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.907+0000 7f857ed79640 1 --2- 172.21.15.16:0/950746446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f857806cac0 0x7f8578156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.907+0000 7f857ed79640 1 -- 172.21.15.16:0/950746446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85780cf950 msgr2=0x7f85780ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:16.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.907+0000 7f857ed79640 1 --2- 172.21.15.16:0/950746446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85780cf950 0x7f85780ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:16.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.907+0000 7f857ed79640 1 -- 172.21.15.16:0/950746446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8568009a70 con 0x7f85780cfd30 2023-12-08T08:38:16.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.907+0000 7f857ed79640 1 --2- 172.21.15.16:0/950746446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85780cfd30 0x7f8578148fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8568009d90 tx=0x7f8568004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:16.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.907+0000 7f8576ffd640 1 -- 172.21.15.16:0/950746446 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856802cea0 con 0x7f85780cfd30 2023-12-08T08:38:16.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.907+0000 7f8576ffd640 1 -- 172.21.15.16:0/950746446 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8568009450 con 0x7f85780cfd30 2023-12-08T08:38:16.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.907+0000 7f8576ffd640 1 -- 172.21.15.16:0/950746446 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856803f420 con 0x7f85780cfd30 2023-12-08T08:38:16.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.907+0000 7f85807dc640 1 -- 172.21.15.16:0/950746446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8578159a00 con 0x7f85780cfd30 2023-12-08T08:38:16.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.907+0000 7f85807dc640 1 -- 172.21.15.16:0/950746446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8578157320 con 0x7f85780cfd30 2023-12-08T08:38:16.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.908+0000 7f85807dc640 1 -- 172.21.15.16:0/950746446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8578067540 con 0x7f85780cfd30 2023-12-08T08:38:16.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.908+0000 7f8576ffd640 1 -- 172.21.15.16:0/950746446 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f856803f580 con 0x7f85780cfd30 2023-12-08T08:38:16.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.909+0000 7f8576ffd640 1 --2- 172.21.15.16:0/950746446 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f856406ae40 0x7f856406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:16.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.909+0000 7f8576ffd640 1 -- 172.21.15.16:0/950746446 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f85680abbd0 con 0x7f85780cfd30 2023-12-08T08:38:16.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.909+0000 7f857e578640 1 --2- 172.21.15.16:0/950746446 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f856406ae40 0x7f856406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:16.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.910+0000 7f857e578640 1 --2- 172.21.15.16:0/950746446 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f856406ae40 0x7f856406d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f85700010e0 tx=0x7f8570007950 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:16.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:16.913+0000 7f8576ffd640 1 -- 172.21.15.16:0/950746446 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8568077180 con 0x7f85780cfd30 2023-12-08T08:38:17.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.091+0000 7f85807dc640 1 -- 172.21.15.16:0/950746446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f85780d0760 con 0x7f85780cfd30 2023-12-08T08:38:17.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.611+0000 7f8576ffd640 1 -- 172.21.15.16:0/950746446 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f85680430b0 con 0x7f85780cfd30 2023-12-08T08:38:17.613 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-12-08T08:38:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.613+0000 7f85807dc640 1 -- 172.21.15.16:0/950746446 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f856406ae40 msgr2=0x7f856406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.614+0000 7f85807dc640 1 --2- 172.21.15.16:0/950746446 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f856406ae40 0x7f856406d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f85700010e0 tx=0x7f8570007950 comp rx=0 tx=0).stop 2023-12-08T08:38:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.614+0000 7f85807dc640 1 -- 172.21.15.16:0/950746446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85780cfd30 msgr2=0x7f8578148fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.614+0000 7f85807dc640 1 --2- 172.21.15.16:0/950746446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85780cfd30 0x7f8578148fe0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8568009d90 tx=0x7f8568004a40 comp rx=0 tx=0).stop 2023-12-08T08:38:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.614+0000 7f85807dc640 1 -- 172.21.15.16:0/950746446 shutdown_connections 2023-12-08T08:38:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.614+0000 7f85807dc640 1 --2- 172.21.15.16:0/950746446 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f856406ae40 0x7f856406d300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.614+0000 7f85807dc640 1 --2- 172.21.15.16:0/950746446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85780cfd30 0x7f8578148fe0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.614+0000 7f85807dc640 1 --2- 172.21.15.16:0/950746446 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f85780cf950 0x7f85780ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.614+0000 7f85807dc640 1 --2- 172.21.15.16:0/950746446 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f857806cac0 0x7f8578156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.614+0000 7f85807dc640 1 -- 172.21.15.16:0/950746446 >> 172.21.15.16:0/950746446 conn(0x7f857809c2f0 msgr2=0x7f85780d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.614+0000 7f85807dc640 1 -- 172.21.15.16:0/950746446 shutdown_connections 2023-12-08T08:38:17.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.614+0000 7f85807dc640 1 -- 172.21.15.16:0/950746446 wait complete. 2023-12-08T08:38:17.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-12-08T08:38:17.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.747+0000 7f49671ea640 1 Processor -- start 2023-12-08T08:38:17.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.747+0000 7f49671ea640 1 -- start start 2023-12-08T08:38:17.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.747+0000 7f49671ea640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f496015b0c0 0x7f49600ced20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:17.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.747+0000 7f49671ea640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49600ce380 0x7f496006a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:17.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.747+0000 7f49671ea640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f496006ae60 con 0x7f496006aa10 2023-12-08T08:38:17.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.747+0000 7f49671ea640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49600cbe70 con 0x7f49600ce380 2023-12-08T08:38:17.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.747+0000 7f49671ea640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49600ce920 con 0x7f496015b0c0 2023-12-08T08:38:17.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.748+0000 7f495ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f49600ce380 0x7f496006a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:17.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.748+0000 7f4965787640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f496006aa10 0x7f496015ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57318/0 (socket says 172.21.15.16:57318) 2023-12-08T08:38:17.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.748+0000 7f4965787640 1 -- 172.21.15.16:0/2362003355 learned_addr learned my addr 172.21.15.16:0/2362003355 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:17.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.748+0000 7f4964f86640 1 --2- 172.21.15.16:0/2362003355 >> v2:172.21.15.112:3300/0 conn(0x7f496015b0c0 0x7f49600ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:17.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.748+0000 7f495ffff640 1 -- 172.21.15.16:0/2362003355 >> v2:172.21.15.112:3300/0 conn(0x7f496015b0c0 msgr2=0x7f49600ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:17.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.748+0000 7f495ffff640 1 --2- 172.21.15.16:0/2362003355 >> v2:172.21.15.112:3300/0 conn(0x7f496015b0c0 0x7f49600ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:17.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.748+0000 7f495ffff640 1 -- 172.21.15.16:0/2362003355 >> v1:172.21.15.16:6789/0 conn(0x7f496006aa10 legacy=0x7f496015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:17.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.748+0000 7f495f7fe640 1 -- 172.21.15.16:0/2362003355 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919745870 0 0) 0x7f4958001490 con 0x7f496006aa10 2023-12-08T08:38:17.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.748+0000 7f495ffff640 1 -- 172.21.15.16:0/2362003355 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49600cf520 con 0x7f49600ce380 2023-12-08T08:38:17.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.749+0000 7f495ffff640 1 --2- 172.21.15.16:0/2362003355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49600ce380 0x7f496006a4d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49600715d0 tx=0x7f494c02b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1616e00d1743c311 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:17.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.749+0000 7f495f7fe640 1 -- 172.21.15.16:0/2362003355 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c031710 con 0x7f49600ce380 2023-12-08T08:38:17.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.749+0000 7f495f7fe640 1 -- 172.21.15.16:0/2362003355 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c02f050 con 0x7f49600ce380 2023-12-08T08:38:17.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.749+0000 7f495f7fe640 1 -- 172.21.15.16:0/2362003355 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c031a90 con 0x7f49600ce380 2023-12-08T08:38:17.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.749+0000 7f49671ea640 1 -- 172.21.15.16:0/2362003355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49600ce380 msgr2=0x7f496006a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:17.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.749+0000 7f49671ea640 1 --2- 172.21.15.16:0/2362003355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49600ce380 0x7f496006a4d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49600715d0 tx=0x7f494c02b7b0 comp rx=0 tx=0).stop 2023-12-08T08:38:17.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.749+0000 7f49671ea640 1 -- 172.21.15.16:0/2362003355 shutdown_connections 2023-12-08T08:38:17.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.749+0000 7f49671ea640 1 --2- 172.21.15.16:0/2362003355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49600ce380 0x7f496006a4d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:17.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.749+0000 7f49671ea640 1 --2- 172.21.15.16:0/2362003355 >> v2:172.21.15.112:3300/0 conn(0x7f496015b0c0 0x7f49600ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:17.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.749+0000 7f49671ea640 1 -- 172.21.15.16:0/2362003355 >> 172.21.15.16:0/2362003355 conn(0x7f49600b1970 msgr2=0x7f49600b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:17.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.749+0000 7f49671ea640 1 -- 172.21.15.16:0/2362003355 shutdown_connections 2023-12-08T08:38:17.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.749+0000 7f49671ea640 1 -- 172.21.15.16:0/2362003355 wait complete. 2023-12-08T08:38:17.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.750+0000 7f49671ea640 1 Processor -- start 2023-12-08T08:38:17.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.751+0000 7f49671ea640 1 -- start start 2023-12-08T08:38:17.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.751+0000 7f49671ea640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f496006aa10 0x7f496006d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:17.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.751+0000 7f49671ea640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49600ce380 0x7f4960158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:17.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.751+0000 7f49671ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496015b0c0 0x7f496015d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:17.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.751+0000 7f49671ea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496006ded0 con 0x7f496015b0c0 2023-12-08T08:38:17.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.751+0000 7f49671ea640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496006e1b0 con 0x7f49600ce380 2023-12-08T08:38:17.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.751+0000 7f49671ea640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f496006e490 con 0x7f496006aa10 2023-12-08T08:38:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f4965787640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496015b0c0 0x7f496015d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f4965787640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496015b0c0 0x7f496015d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56418/0 (socket says 172.21.15.16:56418) 2023-12-08T08:38:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f4965787640 1 -- 172.21.15.16:0/124775583 learned_addr learned my addr 172.21.15.16:0/124775583 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f495ffff640 1 --2- 172.21.15.16:0/124775583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49600ce380 0x7f4960158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f4964f86640 1 --2- 172.21.15.16:0/124775583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f496006aa10 0x7f496006d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f4965787640 1 -- 172.21.15.16:0/124775583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f496006aa10 msgr2=0x7f496006d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f4965787640 1 --2- 172.21.15.16:0/124775583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f496006aa10 0x7f496006d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f4965787640 1 -- 172.21.15.16:0/124775583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49600ce380 msgr2=0x7f4960158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f4965787640 1 --2- 172.21.15.16:0/124775583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49600ce380 0x7f4960158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f4965787640 1 -- 172.21.15.16:0/124775583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c009400 con 0x7f496015b0c0 2023-12-08T08:38:17.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f4965787640 1 --2- 172.21.15.16:0/124775583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496015b0c0 0x7f496015d270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f495800c780 tx=0x7f495800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:17.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f495d7fa640 1 -- 172.21.15.16:0/124775583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495800ce90 con 0x7f496015b0c0 2023-12-08T08:38:17.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f495d7fa640 1 -- 172.21.15.16:0/124775583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495800a640 con 0x7f496015b0c0 2023-12-08T08:38:17.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f49671ea640 1 -- 172.21.15.16:0/124775583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496006e770 con 0x7f496015b0c0 2023-12-08T08:38:17.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f495d7fa640 1 -- 172.21.15.16:0/124775583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495800a9d0 con 0x7f496015b0c0 2023-12-08T08:38:17.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.752+0000 7f49671ea640 1 -- 172.21.15.16:0/124775583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496015def0 con 0x7f496015b0c0 2023-12-08T08:38:17.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.753+0000 7f49671ea640 1 -- 172.21.15.16:0/124775583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f496006e9b0 con 0x7f496015b0c0 2023-12-08T08:38:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.754+0000 7f495d7fa640 1 -- 172.21.15.16:0/124775583 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f495800ab30 con 0x7f496015b0c0 2023-12-08T08:38:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.754+0000 7f495d7fa640 1 --2- 172.21.15.16:0/124775583 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f494406ae40 0x7f494406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.754+0000 7f495d7fa640 1 -- 172.21.15.16:0/124775583 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4958088ab0 con 0x7f496015b0c0 2023-12-08T08:38:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.754+0000 7f4964f86640 1 --2- 172.21.15.16:0/124775583 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f494406ae40 0x7f494406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:17.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.755+0000 7f4964f86640 1 --2- 172.21.15.16:0/124775583 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f494406ae40 0x7f494406d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4954006fd0 tx=0x7f4954008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:17.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.759+0000 7f495d7fa640 1 -- 172.21.15.16:0/124775583 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4958053d60 con 0x7f496015b0c0 2023-12-08T08:38:17.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:17.935+0000 7f49671ea640 1 -- 172.21.15.16:0/124775583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f496006ae60 con 0x7f496015b0c0 2023-12-08T08:38:18.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.613+0000 7f495d7fa640 1 -- 172.21.15.16:0/124775583 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7f496006ae60 con 0x7f496015b0c0 2023-12-08T08:38:18.616 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-12-08T08:38:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.617+0000 7f49671ea640 1 -- 172.21.15.16:0/124775583 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f494406ae40 msgr2=0x7f494406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.617+0000 7f49671ea640 1 --2- 172.21.15.16:0/124775583 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f494406ae40 0x7f494406d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4954006fd0 tx=0x7f4954008040 comp rx=0 tx=0).stop 2023-12-08T08:38:18.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.617+0000 7f49671ea640 1 -- 172.21.15.16:0/124775583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496015b0c0 msgr2=0x7f496015d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:18.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.617+0000 7f49671ea640 1 --2- 172.21.15.16:0/124775583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496015b0c0 0x7f496015d270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f495800c780 tx=0x7f495800cc40 comp rx=0 tx=0).stop 2023-12-08T08:38:18.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.618+0000 7f49671ea640 1 -- 172.21.15.16:0/124775583 shutdown_connections 2023-12-08T08:38:18.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.618+0000 7f49671ea640 1 --2- 172.21.15.16:0/124775583 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f494406ae40 0x7f494406d300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:18.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.618+0000 7f49671ea640 1 --2- 172.21.15.16:0/124775583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f496015b0c0 0x7f496015d270 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:18.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.618+0000 7f49671ea640 1 --2- 172.21.15.16:0/124775583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49600ce380 0x7f4960158050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.618+0000 7f49671ea640 1 --2- 172.21.15.16:0/124775583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f496006aa10 0x7f496006d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.618+0000 7f49671ea640 1 -- 172.21.15.16:0/124775583 >> 172.21.15.16:0/124775583 conn(0x7f49600b1970 msgr2=0x7f496015a370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.618+0000 7f49671ea640 1 -- 172.21.15.16:0/124775583 shutdown_connections 2023-12-08T08:38:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.619+0000 7f49671ea640 1 -- 172.21.15.16:0/124775583 wait complete. 2023-12-08T08:38:18.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-08T08:38:18.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.760+0000 7f4acf3f5640 1 Processor -- start 2023-12-08T08:38:18.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.760+0000 7f4acf3f5640 1 -- start start 2023-12-08T08:38:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4acf3f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac80cf570 0x7f4ac80cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4acf3f5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac80cfef0 con 0x7f4ac80b8950 2023-12-08T08:38:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4acf3f5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac80b5f00 con 0x7f4ac80b6c50 2023-12-08T08:38:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4acf3f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80b6170 con 0x7f4ac80cf570 2023-12-08T08:38:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4acd191640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4ac80b8950 0x7f4ac80ce680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57334/0 (socket says 172.21.15.16:57334) 2023-12-08T08:38:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4acd191640 1 -- 172.21.15.16:0/946271552 learned_addr learned my addr 172.21.15.16:0/946271552 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4abffff640 1 -- 172.21.15.16:0/946271552 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820995893 0 0) 0x7f4ac80cfef0 con 0x7f4ac80b8950 2023-12-08T08:38:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4acc990640 1 --2- 172.21.15.16:0/946271552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac80cf570 0x7f4ac80cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4abffff640 1 -- 172.21.15.16:0/946271552 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ab4003520 con 0x7f4ac80b8950 2023-12-08T08:38:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4abffff640 1 -- 172.21.15.16:0/946271552 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2648200187 0 0) 0x7f4ab4003520 con 0x7f4ac80b8950 2023-12-08T08:38:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4abffff640 1 -- 172.21.15.16:0/946271552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac80cf570 msgr2=0x7f4ac80cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4abffff640 1 --2- 172.21.15.16:0/946271552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac80cf570 0x7f4ac80cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4abffff640 1 -- 172.21.15.16:0/946271552 >> v1:172.21.15.112:6789/0 conn(0x7f4ac80b6c50 legacy=0x7f4ac80b70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4abffff640 1 -- 172.21.15.16:0/946271552 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac80b6450 con 0x7f4ac80b8950 2023-12-08T08:38:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4abffff640 1 -- 172.21.15.16:0/946271552 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f4ac4001670 con 0x7f4ac80b8950 2023-12-08T08:38:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.761+0000 7f4abffff640 1 -- 172.21.15.16:0/946271552 >> v1:172.21.15.16:6789/0 conn(0x7f4ac80b8950 legacy=0x7f4ac80ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.762+0000 7f4abffff640 1 --2- 172.21.15.16:0/946271552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab40039a0 0x7f4ab4003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.762+0000 7f4abffff640 1 --2- 172.21.15.16:0/946271552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac80cf570 0x7f4ab40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.762+0000 7f4abffff640 1 --2- 172.21.15.16:0/946271552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ab4004d00 0x7f4ab4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.762+0000 7f4abffff640 1 -- 172.21.15.16:0/946271552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80b6170 con 0x7f4ab40039a0 2023-12-08T08:38:18.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.762+0000 7f4abffff640 1 -- 172.21.15.16:0/946271552 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab400ba30 con 0x7f4ab4004d00 2023-12-08T08:38:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.762+0000 7f4abffff640 1 -- 172.21.15.16:0/946271552 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab400bbf0 con 0x7f4ac80cf570 2023-12-08T08:38:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.762+0000 7f4acd191640 1 --2- 172.21.15.16:0/946271552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab40039a0 0x7f4ab4003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.762+0000 7f4acc990640 1 --2- 172.21.15.16:0/946271552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac80cf570 0x7f4ab40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.762+0000 7f4acd992640 1 --2- 172.21.15.16:0/946271552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ab4004d00 0x7f4ab4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.762+0000 7f4acd191640 1 -- 172.21.15.16:0/946271552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac80cf570 msgr2=0x7f4ab40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.762+0000 7f4acd191640 1 --2- 172.21.15.16:0/946271552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac80cf570 0x7f4ab40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.762+0000 7f4acd191640 1 -- 172.21.15.16:0/946271552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ab4004d00 msgr2=0x7f4ab4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.762+0000 7f4acd191640 1 --2- 172.21.15.16:0/946271552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ab4004d00 0x7f4ab4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.762+0000 7f4acd191640 1 -- 172.21.15.16:0/946271552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab4003520 con 0x7f4ab40039a0 2023-12-08T08:38:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.763+0000 7f4acd191640 1 --2- 172.21.15.16:0/946271552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab40039a0 0x7f4ab4003d80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400c240 tx=0x7f4ab40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a721f938b28051a server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.763+0000 7f4abffff640 1 -- 172.21.15.16:0/946271552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4005ad0 con 0x7f4ab40039a0 2023-12-08T08:38:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.763+0000 7f4abffff640 1 -- 172.21.15.16:0/946271552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac4005c30 con 0x7f4ab40039a0 2023-12-08T08:38:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.763+0000 7f4abffff640 1 -- 172.21.15.16:0/946271552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac402c8d0 con 0x7f4ab40039a0 2023-12-08T08:38:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.763+0000 7f4acf3f5640 1 -- 172.21.15.16:0/946271552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab40039a0 msgr2=0x7f4ab4003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.763+0000 7f4acf3f5640 1 --2- 172.21.15.16:0/946271552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab40039a0 0x7f4ab4003d80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400c240 tx=0x7f4ab40057f0 comp rx=0 tx=0).stop 2023-12-08T08:38:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.763+0000 7f4acd992640 1 -- 172.21.15.16:0/946271552 reap_dead start 2023-12-08T08:38:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.763+0000 7f4acf3f5640 1 -- 172.21.15.16:0/946271552 shutdown_connections 2023-12-08T08:38:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.763+0000 7f4acf3f5640 1 -- 172.21.15.16:0/946271552 >> 172.21.15.16:0/946271552 conn(0x7f4ac80d3010 msgr2=0x7f4ac80d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.763+0000 7f4acf3f5640 1 -- 172.21.15.16:0/946271552 shutdown_connections 2023-12-08T08:38:18.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.763+0000 7f4acf3f5640 1 -- 172.21.15.16:0/946271552 wait complete. 2023-12-08T08:38:18.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.765+0000 7f4acf3f5640 1 Processor -- start 2023-12-08T08:38:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.765+0000 7f4acf3f5640 1 -- start start 2023-12-08T08:38:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.765+0000 7f4acf3f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab40039a0 0x7f4ac8157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.765+0000 7f4acf3f5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac8156db0 0x7f4ac8156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.765+0000 7f4acf3f5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ac80d1520 0x7f4ac80d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.765+0000 7f4acf3f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80d1ef0 con 0x7f4ab40039a0 2023-12-08T08:38:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.765+0000 7f4acf3f5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac815a330 con 0x7f4ac80d1520 2023-12-08T08:38:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.765+0000 7f4acf3f5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac815cf30 con 0x7f4ac8156db0 2023-12-08T08:38:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.765+0000 7f4acd191640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab40039a0 0x7f4ac8157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.765+0000 7f4acd191640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab40039a0 0x7f4ac8157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56436/0 (socket says 172.21.15.16:56436) 2023-12-08T08:38:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.765+0000 7f4acd191640 1 -- 172.21.15.16:0/3005021561 learned_addr learned my addr 172.21.15.16:0/3005021561 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.766+0000 7f4acc990640 1 --2- 172.21.15.16:0/3005021561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac8156db0 0x7f4ac8156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.766+0000 7f4acd992640 1 --2- 172.21.15.16:0/3005021561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ac80d1520 0x7f4ac80d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:18.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.766+0000 7f4acd191640 1 -- 172.21.15.16:0/3005021561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac8156db0 msgr2=0x7f4ac8156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.766+0000 7f4acd191640 1 --2- 172.21.15.16:0/3005021561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac8156db0 0x7f4ac8156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.766+0000 7f4acd191640 1 -- 172.21.15.16:0/3005021561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ac80d1520 msgr2=0x7f4ac80d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.766+0000 7f4acd191640 1 --2- 172.21.15.16:0/3005021561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ac80d1520 0x7f4ac80d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.766+0000 7f4acd191640 1 -- 172.21.15.16:0/3005021561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac8157d00 con 0x7f4ab40039a0 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.766+0000 7f4acd191640 1 --2- 172.21.15.16:0/3005021561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab40039a0 0x7f4ac8157720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4001930 tx=0x7f4ac80cdb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.766+0000 7f4abdffb640 1 -- 172.21.15.16:0/3005021561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4039030 con 0x7f4ab40039a0 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.766+0000 7f4abdffb640 1 -- 172.21.15.16:0/3005021561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac402cce0 con 0x7f4ab40039a0 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.766+0000 7f4acf3f5640 1 -- 172.21.15.16:0/3005021561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac815d1f0 con 0x7f4ab40039a0 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.766+0000 7f4abdffb640 1 -- 172.21.15.16:0/3005021561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac403c040 con 0x7f4ab40039a0 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.766+0000 7f4acf3f5640 1 -- 172.21.15.16:0/3005021561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac81566d0 con 0x7f4ab40039a0 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.767+0000 7f4acf3f5640 1 -- 172.21.15.16:0/3005021561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac8067430 con 0x7f4ab40039a0 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.768+0000 7f4abdffb640 1 -- 172.21.15.16:0/3005021561 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4ac4032070 con 0x7f4ab40039a0 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.768+0000 7f4abdffb640 1 --2- 172.21.15.16:0/3005021561 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4aac06ae40 0x7f4aac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.768+0000 7f4abdffb640 1 -- 172.21.15.16:0/3005021561 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4ac40aa810 con 0x7f4ab40039a0 2023-12-08T08:38:18.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.768+0000 7f4acc990640 1 --2- 172.21.15.16:0/3005021561 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4aac06ae40 0x7f4aac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:18.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.769+0000 7f4acc990640 1 --2- 172.21.15.16:0/3005021561 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4aac06ae40 0x7f4aac06d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009ae0 tx=0x7f4ab8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:18.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.773+0000 7f4abdffb640 1 -- 172.21.15.16:0/3005021561 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4ac4075950 con 0x7f4ab40039a0 2023-12-08T08:38:18.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:18.989+0000 7f4acf3f5640 1 -- 172.21.15.16:0/3005021561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4ac80c9ba0 con 0x7f4ab40039a0 2023-12-08T08:38:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.630+0000 7f4abdffb640 1 -- 172.21.15.16:0/3005021561 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4ac4078f70 con 0x7f4ab40039a0 2023-12-08T08:38:19.632 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2023-12-08T08:38:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.634+0000 7f4acf3f5640 1 -- 172.21.15.16:0/3005021561 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4aac06ae40 msgr2=0x7f4aac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.634+0000 7f4acf3f5640 1 --2- 172.21.15.16:0/3005021561 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4aac06ae40 0x7f4aac06d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009ae0 tx=0x7f4ab8009190 comp rx=0 tx=0).stop 2023-12-08T08:38:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.634+0000 7f4acf3f5640 1 -- 172.21.15.16:0/3005021561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab40039a0 msgr2=0x7f4ac8157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.634+0000 7f4acf3f5640 1 --2- 172.21.15.16:0/3005021561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab40039a0 0x7f4ac8157720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4001930 tx=0x7f4ac80cdb70 comp rx=0 tx=0).stop 2023-12-08T08:38:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.634+0000 7f4acf3f5640 1 -- 172.21.15.16:0/3005021561 shutdown_connections 2023-12-08T08:38:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.634+0000 7f4acf3f5640 1 --2- 172.21.15.16:0/3005021561 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4aac06ae40 0x7f4aac06d300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.634+0000 7f4acf3f5640 1 --2- 172.21.15.16:0/3005021561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4ac80d1520 0x7f4ac80d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.634+0000 7f4acf3f5640 1 --2- 172.21.15.16:0/3005021561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4ac8156db0 0x7f4ac8156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.634+0000 7f4acf3f5640 1 --2- 172.21.15.16:0/3005021561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ab40039a0 0x7f4ac8157720 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.634+0000 7f4acf3f5640 1 -- 172.21.15.16:0/3005021561 >> 172.21.15.16:0/3005021561 conn(0x7f4ac80d3010 msgr2=0x7f4ac80b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.634+0000 7f4acf3f5640 1 -- 172.21.15.16:0/3005021561 shutdown_connections 2023-12-08T08:38:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.634+0000 7f4acf3f5640 1 -- 172.21.15.16:0/3005021561 wait complete. 2023-12-08T08:38:19.654 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-12-08T08:38:19.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-12-08T08:38:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.769+0000 7f0538010640 1 Processor -- start 2023-12-08T08:38:19.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.769+0000 7f0538010640 1 -- start start 2023-12-08T08:38:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.770+0000 7f0538010640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05300b0b90 0x7f05300d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.770+0000 7f0538010640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f05300d0d60 0x7f05300d1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.770+0000 7f0538010640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05300b21f0 con 0x7f05300cf970 2023-12-08T08:38:19.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.770+0000 7f0538010640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05300ce880 con 0x7f05300d0d60 2023-12-08T08:38:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.770+0000 7f0538010640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05300cea90 con 0x7f05300b0b90 2023-12-08T08:38:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.770+0000 7f05355ab640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05300b0b90 0x7f05300d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.770+0000 7f05365ad640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f05300d0d60 0x7f05300d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.770+0000 7f0535dac640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f05300cf970 0x7f05300cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:34164/0 (socket says 172.21.15.16:34164) 2023-12-08T08:38:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.770+0000 7f0535dac640 1 -- 172.21.15.16:0/1806901323 learned_addr learned my addr 172.21.15.16:0/1806901323 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.771+0000 7f05355ab640 1 -- 172.21.15.16:0/1806901323 >> v2:172.21.15.112:3300/0 conn(0x7f05300d0d60 msgr2=0x7f05300d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:19.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.771+0000 7f05355ab640 1 --2- 172.21.15.16:0/1806901323 >> v2:172.21.15.112:3300/0 conn(0x7f05300d0d60 0x7f05300d1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.771+0000 7f05355ab640 1 -- 172.21.15.16:0/1806901323 >> v1:172.21.15.112:6789/0 conn(0x7f05300cf970 legacy=0x7f05300cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.771+0000 7f0534daa640 1 -- 172.21.15.16:0/1806901323 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019791002 0 0) 0x7f05300b21f0 con 0x7f05300cf970 2023-12-08T08:38:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.771+0000 7f05355ab640 1 -- 172.21.15.16:0/1806901323 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300ced70 con 0x7f05300b0b90 2023-12-08T08:38:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.771+0000 7f05365ad640 1 --2- 172.21.15.16:0/1806901323 >> v2:172.21.15.112:3300/0 conn(0x7f05300d0d60 0x7f05300d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.772+0000 7f05355ab640 1 --2- 172.21.15.16:0/1806901323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05300b0b90 0x7f05300d0460 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f05240097a0 tx=0x7f052402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec7fd73fc81d7b94 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.772+0000 7f0534daa640 1 -- 172.21.15.16:0/1806901323 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524031830 con 0x7f05300b0b90 2023-12-08T08:38:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.772+0000 7f0534daa640 1 -- 172.21.15.16:0/1806901323 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0524031990 con 0x7f05300b0b90 2023-12-08T08:38:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.772+0000 7f0534daa640 1 -- 172.21.15.16:0/1806901323 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524031d20 con 0x7f05300b0b90 2023-12-08T08:38:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.772+0000 7f0538010640 1 -- 172.21.15.16:0/1806901323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05300b0b90 msgr2=0x7f05300d0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.772+0000 7f0538010640 1 --2- 172.21.15.16:0/1806901323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05300b0b90 0x7f05300d0460 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f05240097a0 tx=0x7f052402b8d0 comp rx=0 tx=0).stop 2023-12-08T08:38:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.772+0000 7f0538010640 1 -- 172.21.15.16:0/1806901323 shutdown_connections 2023-12-08T08:38:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.772+0000 7f0538010640 1 --2- 172.21.15.16:0/1806901323 >> v2:172.21.15.112:3300/0 conn(0x7f05300d0d60 0x7f05300d1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:19.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.772+0000 7f0538010640 1 --2- 172.21.15.16:0/1806901323 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05300b0b90 0x7f05300d0460 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.772+0000 7f0538010640 1 -- 172.21.15.16:0/1806901323 >> 172.21.15.16:0/1806901323 conn(0x7f05300d2f80 msgr2=0x7f05300d3360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.772+0000 7f0538010640 1 -- 172.21.15.16:0/1806901323 shutdown_connections 2023-12-08T08:38:19.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.772+0000 7f0538010640 1 -- 172.21.15.16:0/1806901323 wait complete. 2023-12-08T08:38:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.774+0000 7f0538010640 1 Processor -- start 2023-12-08T08:38:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.774+0000 7f0538010640 1 -- start start 2023-12-08T08:38:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.774+0000 7f0538010640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05300b0b90 0x7f05301f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.774+0000 7f0538010640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05300cf970 0x7f05300cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.774+0000 7f0538010640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05300d0d60 0x7f05300b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.774+0000 7f0538010640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05300cb190 con 0x7f05300d0d60 2023-12-08T08:38:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.774+0000 7f0538010640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05300b84b0 con 0x7f05300cf970 2023-12-08T08:38:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.774+0000 7f0538010640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05300b8790 con 0x7f05300b0b90 2023-12-08T08:38:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.774+0000 7f05365ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05300d0d60 0x7f05300b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.774+0000 7f05365ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05300d0d60 0x7f05300b6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56440/0 (socket says 172.21.15.16:56440) 2023-12-08T08:38:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.774+0000 7f05365ad640 1 -- 172.21.15.16:0/1144183483 learned_addr learned my addr 172.21.15.16:0/1144183483 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:19.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.775+0000 7f0535dac640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05300b0b90 0x7f05301f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.775+0000 7f05355ab640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05300cf970 0x7f05300cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.775+0000 7f05365ad640 1 -- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05300b0b90 msgr2=0x7f05301f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.775+0000 7f05365ad640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05300b0b90 0x7f05301f2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.775+0000 7f05365ad640 1 -- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05300cf970 msgr2=0x7f05300cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.775+0000 7f05365ad640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05300cf970 0x7f05300cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.775+0000 7f05365ad640 1 -- 172.21.15.16:0/1144183483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0520009a70 con 0x7f05300d0d60 2023-12-08T08:38:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.775+0000 7f05355ab640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05300cf970 0x7f05300cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.775+0000 7f05365ad640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05300d0d60 0x7f05300b6320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f052000efc0 tx=0x7f052000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.775+0000 7f052e7fc640 1 -- 172.21.15.16:0/1144183483 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520009cd0 con 0x7f05300d0d60 2023-12-08T08:38:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.775+0000 7f052e7fc640 1 -- 172.21.15.16:0/1144183483 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052000f040 con 0x7f05300d0d60 2023-12-08T08:38:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.775+0000 7f0538010640 1 -- 172.21.15.16:0/1144183483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0524009400 con 0x7f05300d0d60 2023-12-08T08:38:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.775+0000 7f052e7fc640 1 -- 172.21.15.16:0/1144183483 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520003990 con 0x7f05300d0d60 2023-12-08T08:38:19.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.775+0000 7f0538010640 1 -- 172.21.15.16:0/1144183483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05300b8a70 con 0x7f05300d0d60 2023-12-08T08:38:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.776+0000 7f0538010640 1 -- 172.21.15.16:0/1144183483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05300cb640 con 0x7f05300d0d60 2023-12-08T08:38:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.777+0000 7f052e7fc640 1 -- 172.21.15.16:0/1144183483 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0520014480 con 0x7f05300d0d60 2023-12-08T08:38:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.777+0000 7f052e7fc640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f051c06ae70 0x7f051c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.777+0000 7f052e7fc640 1 -- 172.21.15.16:0/1144183483 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f052008a1b0 con 0x7f05300d0d60 2023-12-08T08:38:19.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.777+0000 7f0535dac640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f051c06ae70 0x7f051c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:19.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.778+0000 7f0535dac640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f051c06ae70 0x7f051c06d330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0528000f60 tx=0x7f0528007b50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:19.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.781+0000 7f052e7fc640 1 -- 172.21.15.16:0/1144183483 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05300cb640 con 0x7f05300d0d60 2023-12-08T08:38:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:19.967+0000 7f0538010640 1 -- 172.21.15.16:0/1144183483 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f05300cb640 con 0x7f05300d0d60 2023-12-08T08:38:20.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.607+0000 7f052e7fc640 1 -- 172.21.15.16:0/1144183483 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f05300cb640 con 0x7f05300d0d60 2023-12-08T08:38:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.610+0000 7f0538010640 1 -- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f051c06ae70 msgr2=0x7f051c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.610+0000 7f0538010640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f051c06ae70 0x7f051c06d330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0528000f60 tx=0x7f0528007b50 comp rx=0 tx=0).stop 2023-12-08T08:38:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.610+0000 7f0538010640 1 -- 172.21.15.16:0/1144183483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05300d0d60 msgr2=0x7f05300b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.610+0000 7f0538010640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05300d0d60 0x7f05300b6320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f052000efc0 tx=0x7f052000c490 comp rx=0 tx=0).stop 2023-12-08T08:38:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.610+0000 7f0538010640 1 -- 172.21.15.16:0/1144183483 shutdown_connections 2023-12-08T08:38:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.610+0000 7f0538010640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f051c06ae70 0x7f051c06d330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.610+0000 7f0538010640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05300d0d60 0x7f05300b6320 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:20.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.610+0000 7f0538010640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f05300cf970 0x7f05300cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:20.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.610+0000 7f0538010640 1 --2- 172.21.15.16:0/1144183483 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f05300b0b90 0x7f05301f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:20.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.610+0000 7f0538010640 1 -- 172.21.15.16:0/1144183483 >> 172.21.15.16:0/1144183483 conn(0x7f05300d2f80 msgr2=0x7f053006c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:20.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.610+0000 7f0538010640 1 -- 172.21.15.16:0/1144183483 shutdown_connections 2023-12-08T08:38:20.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.610+0000 7f0538010640 1 -- 172.21.15.16:0/1144183483 wait complete. 2023-12-08T08:38:20.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-12-08T08:38:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.745+0000 7fc3d8c7c640 1 Processor -- start 2023-12-08T08:38:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.745+0000 7fc3d8c7c640 1 -- start start 2023-12-08T08:38:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.746+0000 7fc3d8c7c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3d40b3080 0x7fc3d40b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.746+0000 7fc3d8c7c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d40ac210 con 0x7fc3d4070810 2023-12-08T08:38:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.746+0000 7fc3d8c7c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d40abee0 con 0x7fc3d4159e10 2023-12-08T08:38:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.746+0000 7fc3d8c7c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d415e990 con 0x7fc3d40b3080 2023-12-08T08:38:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.746+0000 7fc3d1d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3d40b3080 0x7fc3d40b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.746+0000 7fc3d2d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc3d4070810 0x7fc3d40ad7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57348/0 (socket says 172.21.15.16:57348) 2023-12-08T08:38:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.746+0000 7fc3d1d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3d40b3080 0x7fc3d40b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56444/0 (socket says 172.21.15.16:56444) 2023-12-08T08:38:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.746+0000 7fc3d2d9d640 1 -- 172.21.15.16:0/555795792 learned_addr learned my addr 172.21.15.16:0/555795792 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.746+0000 7fc3d1d9b640 1 -- 172.21.15.16:0/555795792 >> v1:172.21.15.112:6789/0 conn(0x7fc3d4159e10 legacy=0x7fc3d4070100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.746+0000 7fc3d1d9b640 1 -- 172.21.15.16:0/555795792 >> v1:172.21.15.16:6789/0 conn(0x7fc3d4070810 legacy=0x7fc3d40ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.746+0000 7fc3d1d9b640 1 -- 172.21.15.16:0/555795792 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d415e3d0 con 0x7fc3d40b3080 2023-12-08T08:38:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.747+0000 7fc3d1d9b640 1 --2- 172.21.15.16:0/555795792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40b3080 0x7fc3d40b3dd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009790 tx=0x7fc3c802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b0bb8cf71211d4 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.747+0000 7fc3d159a640 1 -- 172.21.15.16:0/555795792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c802e070 con 0x7fc3d40b3080 2023-12-08T08:38:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.747+0000 7fc3d159a640 1 -- 172.21.15.16:0/555795792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c80315a0 con 0x7fc3d40b3080 2023-12-08T08:38:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.747+0000 7fc3d159a640 1 -- 172.21.15.16:0/555795792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8031930 con 0x7fc3d40b3080 2023-12-08T08:38:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.747+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/555795792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40b3080 msgr2=0x7fc3d40b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.747+0000 7fc3d8c7c640 1 --2- 172.21.15.16:0/555795792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40b3080 0x7fc3d40b3dd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009790 tx=0x7fc3c802b840 comp rx=0 tx=0).stop 2023-12-08T08:38:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.748+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/555795792 shutdown_connections 2023-12-08T08:38:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.748+0000 7fc3d8c7c640 1 --2- 172.21.15.16:0/555795792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d40b3080 0x7fc3d40b3dd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.748+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/555795792 >> 172.21.15.16:0/555795792 conn(0x7fc3d406a920 msgr2=0x7fc3d406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.748+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/555795792 shutdown_connections 2023-12-08T08:38:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.748+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/555795792 wait complete. 2023-12-08T08:38:20.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.749+0000 7fc3d8c7c640 1 Processor -- start 2023-12-08T08:38:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.749+0000 7fc3d8c7c640 1 -- start start 2023-12-08T08:38:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d8c7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4070810 0x7fc3d41f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d8c7c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3d40b3080 0x7fc3d413c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d8c7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d4159e10 0x7fc3d415c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d8c7c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d41f2170 con 0x7fc3d4070810 2023-12-08T08:38:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d8c7c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d414bf20 con 0x7fc3d4159e10 2023-12-08T08:38:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d8c7c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d41f4820 con 0x7fc3d40b3080 2023-12-08T08:38:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4070810 0x7fc3d41f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4070810 0x7fc3d41f1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56456/0 (socket says 172.21.15.16:56456) 2023-12-08T08:38:20.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d259c640 1 -- 172.21.15.16:0/1526382116 learned_addr learned my addr 172.21.15.16:0/1526382116 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d1d9b640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3d40b3080 0x7fc3d413c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d2d9d640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d4159e10 0x7fc3d415c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d259c640 1 -- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3d40b3080 msgr2=0x7fc3d413c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d259c640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3d40b3080 0x7fc3d413c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d259c640 1 -- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d4159e10 msgr2=0x7fc3d415c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d259c640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d4159e10 0x7fc3d415c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:20.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d259c640 1 -- 172.21.15.16:0/1526382116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c80093a0 con 0x7fc3d4070810 2023-12-08T08:38:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d2d9d640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d4159e10 0x7fc3d415c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d259c640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4070810 0x7fc3d41f1c30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400b3f0 tx=0x7fc3c400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3c2ffd640 1 -- 172.21.15.16:0/1526382116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4011830 con 0x7fc3d4070810 2023-12-08T08:38:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3c2ffd640 1 -- 172.21.15.16:0/1526382116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c4011990 con 0x7fc3d4070810 2023-12-08T08:38:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/1526382116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d41f4b00 con 0x7fc3d4070810 2023-12-08T08:38:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3c2ffd640 1 -- 172.21.15.16:0/1526382116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4011d40 con 0x7fc3d4070810 2023-12-08T08:38:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.750+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/1526382116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d41f50c0 con 0x7fc3d4070810 2023-12-08T08:38:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.751+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/1526382116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d4067430 con 0x7fc3d4070810 2023-12-08T08:38:20.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.752+0000 7fc3c2ffd640 1 -- 172.21.15.16:0/1526382116 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc3c401a400 con 0x7fc3d4070810 2023-12-08T08:38:20.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.752+0000 7fc3c2ffd640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc3b406ae40 0x7fc3b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:20.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.752+0000 7fc3c2ffd640 1 -- 172.21.15.16:0/1526382116 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5935+0+0 (secure 0 0 0) 0x7fc3c4089470 con 0x7fc3d4070810 2023-12-08T08:38:20.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.752+0000 7fc3d1d9b640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc3b406ae40 0x7fc3b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:20.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.753+0000 7fc3d1d9b640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc3b406ae40 0x7fc3b406d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8036000 tx=0x7fc3c803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:20.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.756+0000 7fc3c2ffd640 1 -- 172.21.15.16:0/1526382116 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc3c40544d0 con 0x7fc3d4070810 2023-12-08T08:38:20.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:20.931+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/1526382116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc3d40cdfa0 con 0x7fc3d4070810 2023-12-08T08:38:22.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.651+0000 7fc3c2ffd640 1 -- 172.21.15.16:0/1526382116 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7fc3c4057af0 con 0x7fc3d4070810 2023-12-08T08:38:22.653 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' created 2023-12-08T08:38:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.654+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc3b406ae40 msgr2=0x7fc3b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.654+0000 7fc3d8c7c640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc3b406ae40 0x7fc3b406d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8036000 tx=0x7fc3c803d040 comp rx=0 tx=0).stop 2023-12-08T08:38:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.654+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/1526382116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4070810 msgr2=0x7fc3d41f1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.654+0000 7fc3d8c7c640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4070810 0x7fc3d41f1c30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400b3f0 tx=0x7fc3c400b8b0 comp rx=0 tx=0).stop 2023-12-08T08:38:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.654+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/1526382116 shutdown_connections 2023-12-08T08:38:22.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.654+0000 7fc3d8c7c640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc3b406ae40 0x7fc3b406d300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.654+0000 7fc3d8c7c640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc3d4159e10 0x7fc3d415c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.654+0000 7fc3d8c7c640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc3d40b3080 0x7fc3d413c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.654+0000 7fc3d8c7c640 1 --2- 172.21.15.16:0/1526382116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3d4070810 0x7fc3d41f1c30 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400b3f0 tx=0x7fc3c400b8b0 comp rx=0 tx=0).stop 2023-12-08T08:38:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.654+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/1526382116 >> 172.21.15.16:0/1526382116 conn(0x7fc3d406a920 msgr2=0x7fc3d409be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.654+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/1526382116 shutdown_connections 2023-12-08T08:38:22.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.654+0000 7fc3d8c7c640 1 -- 172.21.15.16:0/1526382116 wait complete. 2023-12-08T08:38:22.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-12-08T08:38:22.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.791+0000 7f1d07341640 1 Processor -- start 2023-12-08T08:38:22.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.791+0000 7f1d07341640 1 -- start start 2023-12-08T08:38:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.792+0000 7f1d07341640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1d0006ecf0 0x7f1d0006f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.792+0000 7f1d07341640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d0015d600 0x7f1d000cd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.792+0000 7f1d07341640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0006f610 con 0x7f1d000cfa20 2023-12-08T08:38:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.792+0000 7f1d07341640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d000cff90 con 0x7f1d0006ecf0 2023-12-08T08:38:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.792+0000 7f1d07341640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000d0270 con 0x7f1d0015d600 2023-12-08T08:38:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.792+0000 7f1d058de640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1d000cfa20 0x7f1d000cdb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57358/0 (socket says 172.21.15.16:57358) 2023-12-08T08:38:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.792+0000 7f1d058de640 1 -- 172.21.15.16:0/471785275 learned_addr learned my addr 172.21.15.16:0/471785275 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.792+0000 7f1cfffff640 1 -- 172.21.15.16:0/471785275 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555495809 0 0) 0x7f1d0006f610 con 0x7f1d000cfa20 2023-12-08T08:38:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.792+0000 7f1d050dd640 1 --2- 172.21.15.16:0/471785275 >> v2:172.21.15.112:3300/0 conn(0x7f1d0006ecf0 0x7f1d0006f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 -- 172.21.15.16:0/471785275 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ce8003520 con 0x7f1d000cfa20 2023-12-08T08:38:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1d048dc640 1 --2- 172.21.15.16:0/471785275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d0015d600 0x7f1d000cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 -- 172.21.15.16:0/471785275 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2333650345 0 0) 0x7f1ce8003520 con 0x7f1d000cfa20 2023-12-08T08:38:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 -- 172.21.15.16:0/471785275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d0015d600 msgr2=0x7f1d000cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 --2- 172.21.15.16:0/471785275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d0015d600 0x7f1d000cd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 -- 172.21.15.16:0/471785275 >> v2:172.21.15.112:3300/0 conn(0x7f1d0006ecf0 msgr2=0x7f1d0006f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 --2- 172.21.15.16:0/471785275 >> v2:172.21.15.112:3300/0 conn(0x7f1d0006ecf0 0x7f1d0006f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 -- 172.21.15.16:0/471785275 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d000d0550 con 0x7f1d000cfa20 2023-12-08T08:38:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 -- 172.21.15.16:0/471785275 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f1cf00019f0 con 0x7f1d000cfa20 2023-12-08T08:38:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 -- 172.21.15.16:0/471785275 >> v1:172.21.15.16:6789/0 conn(0x7f1d000cfa20 legacy=0x7f1d000cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 --2- 172.21.15.16:0/471785275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce8003970 0x7f1ce8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 --2- 172.21.15.16:0/471785275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ce80047c0 0x7f1ce8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 --2- 172.21.15.16:0/471785275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d0015d600 0x7f1ce8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 -- 172.21.15.16:0/471785275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000cff90 con 0x7f1ce8003970 2023-12-08T08:38:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 -- 172.21.15.16:0/471785275 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000d0270 con 0x7f1ce80047c0 2023-12-08T08:38:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1cfffff640 1 -- 172.21.15.16:0/471785275 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ce800b980 con 0x7f1d0015d600 2023-12-08T08:38:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1d050dd640 1 --2- 172.21.15.16:0/471785275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce8003970 0x7f1ce8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1d048dc640 1 --2- 172.21.15.16:0/471785275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ce80047c0 0x7f1ce8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1d058de640 1 --2- 172.21.15.16:0/471785275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d0015d600 0x7f1ce8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1d050dd640 1 -- 172.21.15.16:0/471785275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d0015d600 msgr2=0x7f1ce8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1d050dd640 1 --2- 172.21.15.16:0/471785275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d0015d600 0x7f1ce8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1d050dd640 1 -- 172.21.15.16:0/471785275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ce80047c0 msgr2=0x7f1ce8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1d050dd640 1 --2- 172.21.15.16:0/471785275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ce80047c0 0x7f1ce8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1d050dd640 1 -- 172.21.15.16:0/471785275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce8003520 con 0x7f1ce8003970 2023-12-08T08:38:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1d058de640 1 --2- 172.21.15.16:0/471785275 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d0015d600 0x7f1ce8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.793+0000 7f1d048dc640 1 --2- 172.21.15.16:0/471785275 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1ce80047c0 0x7f1ce8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.794+0000 7f1d050dd640 1 --2- 172.21.15.16:0/471785275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce8003970 0x7f1ce8003d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009ac0 tx=0x7f1ce80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416e74a2ef9e282f server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.794+0000 7f1cfffff640 1 -- 172.21.15.16:0/471785275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf802c710 con 0x7f1ce8003970 2023-12-08T08:38:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.794+0000 7f1cfffff640 1 -- 172.21.15.16:0/471785275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf802c870 con 0x7f1ce8003970 2023-12-08T08:38:22.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.794+0000 7f1cfffff640 1 -- 172.21.15.16:0/471785275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf802cc00 con 0x7f1ce8003970 2023-12-08T08:38:22.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.795+0000 7f1d07341640 1 -- 172.21.15.16:0/471785275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce8003970 msgr2=0x7f1ce8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:22.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.795+0000 7f1d07341640 1 --2- 172.21.15.16:0/471785275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce8003970 0x7f1ce8003d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009ac0 tx=0x7f1ce80057f0 comp rx=0 tx=0).stop 2023-12-08T08:38:22.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.795+0000 7f1d058de640 1 -- 172.21.15.16:0/471785275 reap_dead start 2023-12-08T08:38:22.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.795+0000 7f1d07341640 1 -- 172.21.15.16:0/471785275 shutdown_connections 2023-12-08T08:38:22.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.795+0000 7f1d07341640 1 -- 172.21.15.16:0/471785275 >> 172.21.15.16:0/471785275 conn(0x7f1d000d0d80 msgr2=0x7f1d000d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:22.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.795+0000 7f1d07341640 1 -- 172.21.15.16:0/471785275 shutdown_connections 2023-12-08T08:38:22.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.795+0000 7f1d07341640 1 -- 172.21.15.16:0/471785275 wait complete. 2023-12-08T08:38:22.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.796+0000 7f1d07341640 1 Processor -- start 2023-12-08T08:38:22.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.796+0000 7f1d07341640 1 -- start start 2023-12-08T08:38:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d07341640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce8003970 0x7f1d0014dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d07341640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d0015ee40 0x7f1d0015f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d07341640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d000aeac0 0x7f1d000aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d07341640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d000af480 con 0x7f1ce8003970 2023-12-08T08:38:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d07341640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0014e020 con 0x7f1d0015ee40 2023-12-08T08:38:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d07341640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0015f900 con 0x7f1d000aeac0 2023-12-08T08:38:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d050dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce8003970 0x7f1d0014dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d050dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce8003970 0x7f1d0014dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56476/0 (socket says 172.21.15.16:56476) 2023-12-08T08:38:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d050dd640 1 -- 172.21.15.16:0/2506530316 learned_addr learned my addr 172.21.15.16:0/2506530316 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d048dc640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d0015ee40 0x7f1d0015f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d058de640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d000aeac0 0x7f1d000aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d050dd640 1 -- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d000aeac0 msgr2=0x7f1d000aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d050dd640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d000aeac0 0x7f1d000aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d050dd640 1 -- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d0015ee40 msgr2=0x7f1d0015f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d050dd640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d0015ee40 0x7f1d0015f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d050dd640 1 -- 172.21.15.16:0/2506530316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0015fbe0 con 0x7f1ce8003970 2023-12-08T08:38:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d058de640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d000aeac0 0x7f1d000aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d048dc640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d0015ee40 0x7f1d0015f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d050dd640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce8003970 0x7f1d0014dae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1cf802c2b0 tx=0x7f1cf8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1cfdffb640 1 -- 172.21.15.16:0/2506530316 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8003810 con 0x7f1ce8003970 2023-12-08T08:38:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1cfdffb640 1 -- 172.21.15.16:0/2506530316 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf802de90 con 0x7f1ce8003970 2023-12-08T08:38:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.797+0000 7f1d07341640 1 -- 172.21.15.16:0/2506530316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0015fec0 con 0x7f1ce8003970 2023-12-08T08:38:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.798+0000 7f1cfdffb640 1 -- 172.21.15.16:0/2506530316 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8003b70 con 0x7f1ce8003970 2023-12-08T08:38:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.798+0000 7f1d07341640 1 -- 172.21.15.16:0/2506530316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d00160480 con 0x7f1ce8003970 2023-12-08T08:38:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.799+0000 7f1d07341640 1 -- 172.21.15.16:0/2506530316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d00067540 con 0x7f1ce8003970 2023-12-08T08:38:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.799+0000 7f1cfdffb640 1 -- 172.21.15.16:0/2506530316 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1cf8003cd0 con 0x7f1ce8003970 2023-12-08T08:38:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.800+0000 7f1cfdffb640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1cec06ae70 0x7f1cec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:22.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.800+0000 7f1cfdffb640 1 -- 172.21.15.16:0/2506530316 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f1cf80ab990 con 0x7f1ce8003970 2023-12-08T08:38:22.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.800+0000 7f1d048dc640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1cec06ae70 0x7f1cec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:22.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.800+0000 7f1d048dc640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1cec06ae70 0x7f1cec06d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1d00150a00 tx=0x7f1cf4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:22.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.804+0000 7f1cfdffb640 1 -- 172.21.15.16:0/2506530316 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1cf8076770 con 0x7f1ce8003970 2023-12-08T08:38:22.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:22.978+0000 7f1d07341640 1 -- 172.21.15.16:0/2506530316 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f1d000d0b60 con 0x7f1ce8003970 2023-12-08T08:38:23.637 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-12-08T08:38:23.635+0000 7f4eb8a35640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-08T08:38:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.652+0000 7f1cfdffb640 1 -- 172.21.15.16:0/2506530316 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1cf8079d90 con 0x7f1ce8003970 2023-12-08T08:38:23.654 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 allow_ec_overwrites to true 2023-12-08T08:38:23.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.657+0000 7f1d07341640 1 -- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1cec06ae70 msgr2=0x7f1cec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:23.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.657+0000 7f1d07341640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1cec06ae70 0x7f1cec06d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1d00150a00 tx=0x7f1cf4008040 comp rx=0 tx=0).stop 2023-12-08T08:38:23.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.657+0000 7f1d07341640 1 -- 172.21.15.16:0/2506530316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce8003970 msgr2=0x7f1d0014dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:23.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.657+0000 7f1d07341640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce8003970 0x7f1d0014dae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1cf802c2b0 tx=0x7f1cf8004730 comp rx=0 tx=0).stop 2023-12-08T08:38:23.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.658+0000 7f1d07341640 1 -- 172.21.15.16:0/2506530316 shutdown_connections 2023-12-08T08:38:23.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.658+0000 7f1d07341640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1cec06ae70 0x7f1cec06d330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:23.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.658+0000 7f1d07341640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d000aeac0 0x7f1d000aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:23.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.658+0000 7f1d07341640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d0015ee40 0x7f1d0015f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:23.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.658+0000 7f1d07341640 1 --2- 172.21.15.16:0/2506530316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce8003970 0x7f1d0014dae0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:23.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.658+0000 7f1d07341640 1 -- 172.21.15.16:0/2506530316 >> 172.21.15.16:0/2506530316 conn(0x7f1d000d0d80 msgr2=0x7f1d0015c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:23.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.661+0000 7f1d07341640 1 -- 172.21.15.16:0/2506530316 shutdown_connections 2023-12-08T08:38:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.666+0000 7f1d07341640 1 -- 172.21.15.16:0/2506530316 wait complete. 2023-12-08T08:38:23.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-12-08T08:38:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.801+0000 7f64f2789640 1 Processor -- start 2023-12-08T08:38:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.801+0000 7f64f2789640 1 -- start start 2023-12-08T08:38:23.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64f2789640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f64ec145d80 0x7f64ec071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64f2789640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64ec0b4920 0x7f64ec0acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64f2789640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64ec0ad290 0x7f64ec06a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64f2789640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec0b4e00 con 0x7f64ec0b4920 2023-12-08T08:38:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64f2789640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec0b50e0 con 0x7f64ec145d80 2023-12-08T08:38:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64f2789640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec06cfb0 con 0x7f64ec0ad290 2023-12-08T08:38:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64eb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64ec0b4920 0x7f64ec0acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64eb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64ec0b4920 0x7f64ec0acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56480/0 (socket says 172.21.15.16:56480) 2023-12-08T08:38:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64eb7fe640 1 -- 172.21.15.16:0/1701427495 learned_addr learned my addr 172.21.15.16:0/1701427495 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64eb7fe640 1 -- 172.21.15.16:0/1701427495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64ec0ad290 msgr2=0x7f64ec06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64ebfff640 1 --2- 172.21.15.16:0/1701427495 >> v2:172.21.15.112:3300/0 conn(0x7f64ec145d80 0x7f64ec071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64f0d26640 1 --2- 172.21.15.16:0/1701427495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64ec0ad290 0x7f64ec06a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:23.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64eb7fe640 1 --2- 172.21.15.16:0/1701427495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64ec0ad290 0x7f64ec06a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64eb7fe640 1 -- 172.21.15.16:0/1701427495 >> v2:172.21.15.112:3300/0 conn(0x7f64ec145d80 msgr2=0x7f64ec071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.802+0000 7f64eb7fe640 1 --2- 172.21.15.16:0/1701427495 >> v2:172.21.15.112:3300/0 conn(0x7f64ec145d80 0x7f64ec071ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.803+0000 7f64eb7fe640 1 -- 172.21.15.16:0/1701427495 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec06d290 con 0x7f64ec0b4920 2023-12-08T08:38:23.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.803+0000 7f64ebfff640 1 --2- 172.21.15.16:0/1701427495 >> v2:172.21.15.112:3300/0 conn(0x7f64ec145d80 0x7f64ec071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.803+0000 7f64eb7fe640 1 --2- 172.21.15.16:0/1701427495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64ec0b4920 0x7f64ec0acd50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64ec09df90 tx=0x7f64dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c180fa34ef53c2c server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.803+0000 7f64eaffd640 1 -- 172.21.15.16:0/1701427495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc031870 con 0x7f64ec0b4920 2023-12-08T08:38:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.803+0000 7f64eaffd640 1 -- 172.21.15.16:0/1701427495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64dc0319d0 con 0x7f64ec0b4920 2023-12-08T08:38:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.803+0000 7f64eaffd640 1 -- 172.21.15.16:0/1701427495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64dc031d60 con 0x7f64ec0b4920 2023-12-08T08:38:23.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.804+0000 7f64f2789640 1 -- 172.21.15.16:0/1701427495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64ec0b4920 msgr2=0x7f64ec0acd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.804+0000 7f64f2789640 1 --2- 172.21.15.16:0/1701427495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64ec0b4920 0x7f64ec0acd50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f64ec09df90 tx=0x7f64dc02b910 comp rx=0 tx=0).stop 2023-12-08T08:38:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.804+0000 7f64f2789640 1 -- 172.21.15.16:0/1701427495 shutdown_connections 2023-12-08T08:38:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.804+0000 7f64f2789640 1 --2- 172.21.15.16:0/1701427495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64ec0ad290 0x7f64ec06a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.804+0000 7f64f2789640 1 --2- 172.21.15.16:0/1701427495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64ec0b4920 0x7f64ec0acd50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.804+0000 7f64f2789640 1 --2- 172.21.15.16:0/1701427495 >> v2:172.21.15.112:3300/0 conn(0x7f64ec145d80 0x7f64ec071ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.804+0000 7f64f2789640 1 -- 172.21.15.16:0/1701427495 >> 172.21.15.16:0/1701427495 conn(0x7f64ec06b7c0 msgr2=0x7f64ec06bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.804+0000 7f64f2789640 1 -- 172.21.15.16:0/1701427495 shutdown_connections 2023-12-08T08:38:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.804+0000 7f64f2789640 1 -- 172.21.15.16:0/1701427495 wait complete. 2023-12-08T08:38:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.805+0000 7f64f2789640 1 Processor -- start 2023-12-08T08:38:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.805+0000 7f64f2789640 1 -- start start 2023-12-08T08:38:23.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.806+0000 7f64f2789640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64ec0ad290 0x7f64ec13de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:23.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.806+0000 7f64f2789640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64ec0b4920 0x7f64ec0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.806+0000 7f64f2789640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64ec145d80 0x7f64ec140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.806+0000 7f64f2789640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec13e340 con 0x7f64ec0ad290 2023-12-08T08:38:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.806+0000 7f64f2789640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0ce320 con 0x7f64ec145d80 2023-12-08T08:38:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.806+0000 7f64f2789640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0cf580 con 0x7f64ec0b4920 2023-12-08T08:38:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.806+0000 7f64ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64ec0ad290 0x7f64ec13de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.806+0000 7f64ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64ec0ad290 0x7f64ec13de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56492/0 (socket says 172.21.15.16:56492) 2023-12-08T08:38:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.806+0000 7f64ebfff640 1 -- 172.21.15.16:0/85368152 learned_addr learned my addr 172.21.15.16:0/85368152 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.806+0000 7f64eb7fe640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64ec0b4920 0x7f64ec0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.806+0000 7f64f0d26640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64ec145d80 0x7f64ec140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.807+0000 7f64ebfff640 1 -- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64ec0b4920 msgr2=0x7f64ec0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.807+0000 7f64ebfff640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64ec0b4920 0x7f64ec0cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.807+0000 7f64ebfff640 1 -- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64ec145d80 msgr2=0x7f64ec140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.807+0000 7f64ebfff640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64ec145d80 0x7f64ec140610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.807+0000 7f64ebfff640 1 -- 172.21.15.16:0/85368152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc0094d0 con 0x7f64ec0ad290 2023-12-08T08:38:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.807+0000 7f64eb7fe640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64ec0b4920 0x7f64ec0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.807+0000 7f64f0d26640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64ec145d80 0x7f64ec140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.807+0000 7f64ebfff640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64ec0ad290 0x7f64ec13de00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f64e000c820 tx=0x7f64e000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.807+0000 7f64e8ff9640 1 -- 172.21.15.16:0/85368152 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e0007a30 con 0x7f64ec0ad290 2023-12-08T08:38:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.807+0000 7f64e8ff9640 1 -- 172.21.15.16:0/85368152 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e0007b90 con 0x7f64ec0ad290 2023-12-08T08:38:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.807+0000 7f64f2789640 1 -- 172.21.15.16:0/85368152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec0cd710 con 0x7f64ec0ad290 2023-12-08T08:38:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.807+0000 7f64e8ff9640 1 -- 172.21.15.16:0/85368152 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e000ce10 con 0x7f64ec0ad290 2023-12-08T08:38:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.807+0000 7f64f2789640 1 -- 172.21.15.16:0/85368152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec0cdcd0 con 0x7f64ec0ad290 2023-12-08T08:38:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.808+0000 7f64f2789640 1 -- 172.21.15.16:0/85368152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec071460 con 0x7f64ec0ad290 2023-12-08T08:38:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.808+0000 7f64e8ff9640 1 -- 172.21.15.16:0/85368152 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f64e00113f0 con 0x7f64ec0ad290 2023-12-08T08:38:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.809+0000 7f64e8ff9640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f64d806ae40 0x7f64d806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:23.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.809+0000 7f64e8ff9640 1 -- 172.21.15.16:0/85368152 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f64e00899d0 con 0x7f64ec0ad290 2023-12-08T08:38:23.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.809+0000 7f64eb7fe640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f64d806ae40 0x7f64d806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:23.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.810+0000 7f64eb7fe640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f64d806ae40 0x7f64d806d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f64dc036000 tx=0x7f64dc03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:23.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:23.813+0000 7f64e8ff9640 1 -- 172.21.15.16:0/85368152 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f64e00547b0 con 0x7f64ec0ad290 2023-12-08T08:38:24.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.029+0000 7f64f2789640 1 -- 172.21.15.16:0/85368152 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f64ec067430 con 0x7f64ec0ad290 2023-12-08T08:38:24.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.666+0000 7f64e8ff9640 1 -- 172.21.15.16:0/85368152 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f64e001f0b0 con 0x7f64ec0ad290 2023-12-08T08:38:24.668 INFO:teuthology.orchestra.run.smithi016.stderr:added data pool 4 to fsmap 2023-12-08T08:38:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.669+0000 7f64f2789640 1 -- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f64d806ae40 msgr2=0x7f64d806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.669+0000 7f64f2789640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f64d806ae40 0x7f64d806d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f64dc036000 tx=0x7f64dc03c040 comp rx=0 tx=0).stop 2023-12-08T08:38:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.669+0000 7f64f2789640 1 -- 172.21.15.16:0/85368152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64ec0ad290 msgr2=0x7f64ec13de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.669+0000 7f64f2789640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64ec0ad290 0x7f64ec13de00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f64e000c820 tx=0x7f64e000cce0 comp rx=0 tx=0).stop 2023-12-08T08:38:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.669+0000 7f64f2789640 1 -- 172.21.15.16:0/85368152 shutdown_connections 2023-12-08T08:38:24.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.669+0000 7f64f2789640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f64d806ae40 0x7f64d806d300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.669+0000 7f64f2789640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64ec145d80 0x7f64ec140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.669+0000 7f64f2789640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64ec0b4920 0x7f64ec0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.669+0000 7f64f2789640 1 --2- 172.21.15.16:0/85368152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64ec0ad290 0x7f64ec13de00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.669+0000 7f64f2789640 1 -- 172.21.15.16:0/85368152 >> 172.21.15.16:0/85368152 conn(0x7f64ec06b7c0 msgr2=0x7f64ec06d990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.669+0000 7f64f2789640 1 -- 172.21.15.16:0/85368152 shutdown_connections 2023-12-08T08:38:24.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.669+0000 7f64f2789640 1 -- 172.21.15.16:0/85368152 wait complete. 2023-12-08T08:38:24.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-08T08:38:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.795+0000 7f570d431640 1 Processor -- start 2023-12-08T08:38:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.795+0000 7f570d431640 1 -- start start 2023-12-08T08:38:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.795+0000 7f570d431640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f57080cf760 0x7f570806a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.795+0000 7f570d431640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f570815e150 0x7f5708161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.796+0000 7f570d431640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f570806b070 con 0x7f57080cf380 2023-12-08T08:38:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.796+0000 7f570d431640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5708072290 con 0x7f570815e150 2023-12-08T08:38:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.796+0000 7f570d431640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57080d0d00 con 0x7f57080cf760 2023-12-08T08:38:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.796+0000 7f57077fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f570815e150 0x7f5708161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.796+0000 7f5706ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f57080cf380 0x7f5708071b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57368/0 (socket says 172.21.15.16:57368) 2023-12-08T08:38:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.796+0000 7f5706ffd640 1 -- 172.21.15.16:0/2383202299 learned_addr learned my addr 172.21.15.16:0/2383202299 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.796+0000 7f56fffff640 1 --2- 172.21.15.16:0/2383202299 >> v2:172.21.15.112:3300/0 conn(0x7f57080cf760 0x7f570806a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.796+0000 7f57077fe640 1 -- 172.21.15.16:0/2383202299 >> v2:172.21.15.112:3300/0 conn(0x7f57080cf760 msgr2=0x7f570806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.796+0000 7f57077fe640 1 --2- 172.21.15.16:0/2383202299 >> v2:172.21.15.112:3300/0 conn(0x7f57080cf760 0x7f570806a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.796+0000 7f57077fe640 1 -- 172.21.15.16:0/2383202299 >> v1:172.21.15.16:6789/0 conn(0x7f57080cf380 legacy=0x7f5708071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.796+0000 7f57077fe640 1 -- 172.21.15.16:0/2383202299 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570806aab0 con 0x7f570815e150 2023-12-08T08:38:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.796+0000 7f57067fc640 1 -- 172.21.15.16:0/2383202299 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325688447 0 0) 0x7f570806b070 con 0x7f57080cf380 2023-12-08T08:38:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.797+0000 7f57077fe640 1 --2- 172.21.15.16:0/2383202299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570815e150 0x7f5708161690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f57080cd830 tx=0x7f570002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadc0cd4be2fd37f server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.797+0000 7f57067fc640 1 -- 172.21.15.16:0/2383202299 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570002cbd0 con 0x7f570815e150 2023-12-08T08:38:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.797+0000 7f57067fc640 1 -- 172.21.15.16:0/2383202299 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570002cd30 con 0x7f570815e150 2023-12-08T08:38:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.797+0000 7f57067fc640 1 -- 172.21.15.16:0/2383202299 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570002cea0 con 0x7f570815e150 2023-12-08T08:38:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.797+0000 7f570d431640 1 -- 172.21.15.16:0/2383202299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570815e150 msgr2=0x7f5708161690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.797+0000 7f570d431640 1 --2- 172.21.15.16:0/2383202299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570815e150 0x7f5708161690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f57080cd830 tx=0x7f570002be20 comp rx=0 tx=0).stop 2023-12-08T08:38:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.797+0000 7f570d431640 1 -- 172.21.15.16:0/2383202299 shutdown_connections 2023-12-08T08:38:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.797+0000 7f570d431640 1 --2- 172.21.15.16:0/2383202299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570815e150 0x7f5708161690 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.797+0000 7f570d431640 1 --2- 172.21.15.16:0/2383202299 >> v2:172.21.15.112:3300/0 conn(0x7f57080cf760 0x7f570806a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.797+0000 7f570d431640 1 -- 172.21.15.16:0/2383202299 >> 172.21.15.16:0/2383202299 conn(0x7f57080cfda0 msgr2=0x7f57080d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.798+0000 7f570d431640 1 -- 172.21.15.16:0/2383202299 shutdown_connections 2023-12-08T08:38:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.798+0000 7f570d431640 1 -- 172.21.15.16:0/2383202299 wait complete. 2023-12-08T08:38:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.799+0000 7f570d431640 1 Processor -- start 2023-12-08T08:38:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.799+0000 7f570d431640 1 -- start start 2023-12-08T08:38:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.799+0000 7f570d431640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57080cf380 0x7f5708147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.799+0000 7f570d431640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57080cf760 0x7f57080b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.799+0000 7f570d431640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570815e150 0x7f57080b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.799+0000 7f570d431640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57081f6c80 con 0x7f570815e150 2023-12-08T08:38:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.799+0000 7f570d431640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57081491e0 con 0x7f57080cf380 2023-12-08T08:38:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.799+0000 7f570d431640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5708148420 con 0x7f57080cf760 2023-12-08T08:38:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f57077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570815e150 0x7f57080b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f57077fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570815e150 0x7f57080b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56508/0 (socket says 172.21.15.16:56508) 2023-12-08T08:38:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f57077fe640 1 -- 172.21.15.16:0/3131475647 learned_addr learned my addr 172.21.15.16:0/3131475647 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f5706ffd640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57080cf380 0x7f5708147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f56fffff640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57080cf760 0x7f57080b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f57077fe640 1 -- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57080cf760 msgr2=0x7f57080b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f57077fe640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57080cf760 0x7f57080b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f57077fe640 1 -- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57080cf380 msgr2=0x7f5708147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f57077fe640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57080cf380 0x7f5708147b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f57077fe640 1 -- 172.21.15.16:0/3131475647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5700009a70 con 0x7f570815e150 2023-12-08T08:38:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f5706ffd640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57080cf380 0x7f5708147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f57077fe640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570815e150 0x7f57080b1080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5700009d90 tx=0x7f5700004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f56ff7fe640 1 -- 172.21.15.16:0/3131475647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570002cea0 con 0x7f570815e150 2023-12-08T08:38:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f56ff7fe640 1 -- 172.21.15.16:0/3131475647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5700040670 con 0x7f570815e150 2023-12-08T08:38:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f570d431640 1 -- 172.21.15.16:0/3131475647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5708148700 con 0x7f570815e150 2023-12-08T08:38:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f56ff7fe640 1 -- 172.21.15.16:0/3131475647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700040a20 con 0x7f570815e150 2023-12-08T08:38:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.800+0000 7f570d431640 1 -- 172.21.15.16:0/3131475647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5708148cc0 con 0x7f570815e150 2023-12-08T08:38:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.801+0000 7f570d431640 1 -- 172.21.15.16:0/3131475647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5708067430 con 0x7f570815e150 2023-12-08T08:38:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.802+0000 7f56ff7fe640 1 -- 172.21.15.16:0/3131475647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5700040b80 con 0x7f570815e150 2023-12-08T08:38:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.802+0000 7f56ff7fe640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f56e806ae40 0x7f56e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.802+0000 7f56ff7fe640 1 -- 172.21.15.16:0/3131475647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f57000ac6b0 con 0x7f570815e150 2023-12-08T08:38:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.802+0000 7f5706ffd640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f56e806ae40 0x7f56e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.803+0000 7f5706ffd640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f56e806ae40 0x7f56e806d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f56f8000f60 tx=0x7f56f8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:24.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:24.807+0000 7f56ff7fe640 1 -- 172.21.15.16:0/3131475647 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5700077590 con 0x7f570815e150 2023-12-08T08:38:25.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.019+0000 7f570d431640 1 -- 172.21.15.16:0/3131475647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5708148e50 con 0x7f570815e150 2023-12-08T08:38:25.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.020+0000 7f56ff7fe640 1 -- 172.21.15.16:0/3131475647 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f570007abb0 con 0x7f570815e150 2023-12-08T08:38:25.022 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:25.022 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/307039451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":307039451},{"type":"v1","addr":"172.21.15.112:6836","nonce":307039451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1424152341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1424152341},{"type":"v1","addr":"172.21.15.16:6833","nonce":1424152341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3597811670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3597811670},{"type":"v1","addr":"172.21.15.16:6835","nonce":3597811670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-08T08:38:18.991090+0000","modified":"2023-12-08T08:38:24.030777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1082241707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1082241707},{"type":"v1","addr":"172.21.15.112:6837","nonce":1082241707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-12-08T08:38:25.022 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2023-12-08T08:38:25.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.021+0000 7f570d431640 1 -- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f56e806ae40 msgr2=0x7f56e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.021+0000 7f570d431640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f56e806ae40 0x7f56e806d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f56f8000f60 tx=0x7f56f8009040 comp rx=0 tx=0).stop 2023-12-08T08:38:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.021+0000 7f570d431640 1 -- 172.21.15.16:0/3131475647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570815e150 msgr2=0x7f57080b1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.021+0000 7f570d431640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570815e150 0x7f57080b1080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5700009d90 tx=0x7f5700004a40 comp rx=0 tx=0).stop 2023-12-08T08:38:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.021+0000 7f570d431640 1 -- 172.21.15.16:0/3131475647 shutdown_connections 2023-12-08T08:38:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.021+0000 7f570d431640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f56e806ae40 0x7f56e806d300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.021+0000 7f570d431640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570815e150 0x7f57080b1080 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.021+0000 7f570d431640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f57080cf760 0x7f57080b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.021+0000 7f570d431640 1 --2- 172.21.15.16:0/3131475647 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f57080cf380 0x7f5708147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.021+0000 7f570d431640 1 -- 172.21.15.16:0/3131475647 >> 172.21.15.16:0/3131475647 conn(0x7f57080cfda0 msgr2=0x7f57080cc000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.022+0000 7f570d431640 1 -- 172.21.15.16:0/3131475647 shutdown_connections 2023-12-08T08:38:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.022+0000 7f570d431640 1 -- 172.21.15.16:0/3131475647 wait complete. 2023-12-08T08:38:25.034 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:25.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.139+0000 7f30a9516640 1 Processor -- start 2023-12-08T08:38:25.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.139+0000 7f30a9516640 1 -- start start 2023-12-08T08:38:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f30a9516640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30a40b3340 0x7f30a40d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f30a9516640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f30a40d1d00 0x7f30a40b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f30a9516640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a40cdc20 con 0x7f30a40b3bf0 2023-12-08T08:38:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f30a9516640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a40d11d0 con 0x7f30a40d1d00 2023-12-08T08:38:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f30a9516640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40d3a60 con 0x7f30a40b3340 2023-12-08T08:38:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f30a37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f30a40d1d00 0x7f30a40b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f30a37fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f30a40d1d00 0x7f30a40b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56510/0 (socket says 172.21.15.16:56510) 2023-12-08T08:38:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f309bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f30a40b3bf0 0x7f30a40d15f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57372/0 (socket says 172.21.15.16:57372) 2023-12-08T08:38:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f30a37fe640 1 -- 172.21.15.16:0/758573814 learned_addr learned my addr 172.21.15.16:0/758573814 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f30a2ffd640 1 --2- 172.21.15.16:0/758573814 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30a40b3340 0x7f30a40d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f30a37fe640 1 -- 172.21.15.16:0/758573814 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30a40b3340 msgr2=0x7f30a40d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:25.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f30a37fe640 1 --2- 172.21.15.16:0/758573814 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30a40b3340 0x7f30a40d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f30a37fe640 1 -- 172.21.15.16:0/758573814 >> v1:172.21.15.16:6789/0 conn(0x7f30a40b3bf0 legacy=0x7f30a40d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f30a37fe640 1 -- 172.21.15.16:0/758573814 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a40d3d40 con 0x7f30a40d1d00 2023-12-08T08:38:25.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.140+0000 7f30a2ffd640 1 --2- 172.21.15.16:0/758573814 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30a40b3340 0x7f30a40d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:25.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.141+0000 7f30a37fe640 1 --2- 172.21.15.16:0/758573814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a40d1d00 0x7f30a40b5e40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f309c009e50 tx=0x7f309c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e588ae39caad86b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:25.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.141+0000 7f30a27fc640 1 -- 172.21.15.16:0/758573814 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c02ca70 con 0x7f30a40d1d00 2023-12-08T08:38:25.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.141+0000 7f30a27fc640 1 -- 172.21.15.16:0/758573814 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309c02cbd0 con 0x7f30a40d1d00 2023-12-08T08:38:25.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.141+0000 7f30a27fc640 1 -- 172.21.15.16:0/758573814 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c0343f0 con 0x7f30a40d1d00 2023-12-08T08:38:25.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.142+0000 7f30a9516640 1 -- 172.21.15.16:0/758573814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a40d1d00 msgr2=0x7f30a40b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.142+0000 7f30a9516640 1 --2- 172.21.15.16:0/758573814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a40d1d00 0x7f30a40b5e40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f309c009e50 tx=0x7f309c02bed0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.142+0000 7f30a9516640 1 -- 172.21.15.16:0/758573814 shutdown_connections 2023-12-08T08:38:25.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.142+0000 7f30a9516640 1 --2- 172.21.15.16:0/758573814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a40d1d00 0x7f30a40b5e40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.142+0000 7f30a9516640 1 --2- 172.21.15.16:0/758573814 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30a40b3340 0x7f30a40d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.142+0000 7f30a9516640 1 -- 172.21.15.16:0/758573814 >> 172.21.15.16:0/758573814 conn(0x7f30a409c2f0 msgr2=0x7f30a40d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:25.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.142+0000 7f30a9516640 1 -- 172.21.15.16:0/758573814 shutdown_connections 2023-12-08T08:38:25.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.142+0000 7f30a9516640 1 -- 172.21.15.16:0/758573814 wait complete. 2023-12-08T08:38:25.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.143+0000 7f30a9516640 1 Processor -- start 2023-12-08T08:38:25.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.143+0000 7f30a9516640 1 -- start start 2023-12-08T08:38:25.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a9516640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a40b3340 0x7f30a40c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a9516640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30a40b3bf0 0x7f30a41f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a9516640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30a40d1d00 0x7f30a4151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a9516640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a41f3f70 con 0x7f30a40b3340 2023-12-08T08:38:25.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a9516640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a41519d0 con 0x7f30a40b3bf0 2023-12-08T08:38:25.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a9516640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a40b68b0 con 0x7f30a40d1d00 2023-12-08T08:38:25.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a40b3340 0x7f30a40c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a40b3340 0x7f30a40c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56520/0 (socket says 172.21.15.16:56520) 2023-12-08T08:38:25.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a2ffd640 1 -- 172.21.15.16:0/3393307609 learned_addr learned my addr 172.21.15.16:0/3393307609 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:25.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a2ffd640 1 -- 172.21.15.16:0/3393307609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30a40d1d00 msgr2=0x7f30a4151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a2ffd640 1 --2- 172.21.15.16:0/3393307609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30a40d1d00 0x7f30a4151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a2ffd640 1 -- 172.21.15.16:0/3393307609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30a40b3bf0 msgr2=0x7f30a41f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a2ffd640 1 --2- 172.21.15.16:0/3393307609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30a40b3bf0 0x7f30a41f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a2ffd640 1 -- 172.21.15.16:0/3393307609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c009b30 con 0x7f30a40b3340 2023-12-08T08:38:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a2ffd640 1 --2- 172.21.15.16:0/3393307609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a40b3340 0x7f30a40c9460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f309400c820 tx=0x7f309400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f309b7fe640 1 -- 172.21.15.16:0/3393307609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094007a30 con 0x7f30a40b3340 2023-12-08T08:38:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f309b7fe640 1 -- 172.21.15.16:0/3393307609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3094007b90 con 0x7f30a40b3340 2023-12-08T08:38:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a9516640 1 -- 172.21.15.16:0/3393307609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a40b6b90 con 0x7f30a40b3340 2023-12-08T08:38:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f309b7fe640 1 -- 172.21.15.16:0/3393307609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309400ce70 con 0x7f30a40b3340 2023-12-08T08:38:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.144+0000 7f30a9516640 1 -- 172.21.15.16:0/3393307609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a40b7150 con 0x7f30a40b3340 2023-12-08T08:38:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.145+0000 7f30a9516640 1 -- 172.21.15.16:0/3393307609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a4067430 con 0x7f30a40b3340 2023-12-08T08:38:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.146+0000 7f309b7fe640 1 -- 172.21.15.16:0/3393307609 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f30940113f0 con 0x7f30a40b3340 2023-12-08T08:38:25.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.146+0000 7f309b7fe640 1 --2- 172.21.15.16:0/3393307609 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f308406ae70 0x7f308406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.146+0000 7f309b7fe640 1 -- 172.21.15.16:0/3393307609 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3094089aa0 con 0x7f30a40b3340 2023-12-08T08:38:25.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.146+0000 7f309bfff640 1 --2- 172.21.15.16:0/3393307609 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f308406ae70 0x7f308406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.147+0000 7f309bfff640 1 --2- 172.21.15.16:0/3393307609 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f308406ae70 0x7f308406d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f30a40c9250 tx=0x7f308c0079c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:25.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.151+0000 7f309b7fe640 1 -- 172.21.15.16:0/3393307609 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3094054860 con 0x7f30a40b3340 2023-12-08T08:38:25.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.328+0000 7f30a9516640 1 -- 172.21.15.16:0/3393307609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30a40d13d0 con 0x7f30a40b3340 2023-12-08T08:38:25.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.329+0000 7f309b7fe640 1 -- 172.21.15.16:0/3393307609 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f309401f090 con 0x7f30a40b3340 2023-12-08T08:38:25.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:25.332 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:24.030625+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:25.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.331+0000 7f30a9516640 1 -- 172.21.15.16:0/3393307609 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f308406ae70 msgr2=0x7f308406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.331+0000 7f30a9516640 1 --2- 172.21.15.16:0/3393307609 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f308406ae70 0x7f308406d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f30a40c9250 tx=0x7f308c0079c0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.331+0000 7f30a9516640 1 -- 172.21.15.16:0/3393307609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a40b3340 msgr2=0x7f30a40c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.331+0000 7f30a9516640 1 --2- 172.21.15.16:0/3393307609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a40b3340 0x7f30a40c9460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f309400c820 tx=0x7f309400cc20 comp rx=0 tx=0).stop 2023-12-08T08:38:25.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.331+0000 7f30a9516640 1 -- 172.21.15.16:0/3393307609 shutdown_connections 2023-12-08T08:38:25.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.331+0000 7f30a9516640 1 --2- 172.21.15.16:0/3393307609 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f308406ae70 0x7f308406d330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.331+0000 7f30a9516640 1 --2- 172.21.15.16:0/3393307609 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f30a40d1d00 0x7f30a4151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.331+0000 7f30a9516640 1 --2- 172.21.15.16:0/3393307609 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f30a40b3bf0 0x7f30a41f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.331+0000 7f30a9516640 1 --2- 172.21.15.16:0/3393307609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30a40b3340 0x7f30a40c9460 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.331+0000 7f30a9516640 1 -- 172.21.15.16:0/3393307609 >> 172.21.15.16:0/3393307609 conn(0x7f30a409c2f0 msgr2=0x7f30a40ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:25.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.331+0000 7f30a9516640 1 -- 172.21.15.16:0/3393307609 shutdown_connections 2023-12-08T08:38:25.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.331+0000 7f30a9516640 1 -- 172.21.15.16:0/3393307609 wait complete. 2023-12-08T08:38:25.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:25.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.448+0000 7f8f46272640 1 Processor -- start 2023-12-08T08:38:25.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.448+0000 7f8f46272640 1 -- start start 2023-12-08T08:38:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.449+0000 7f8f46272640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f400cd240 0x7f8f400cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.449+0000 7f8f46272640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8f4015f4e0 0x7f8f4015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.449+0000 7f8f46272640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f400cfb90 con 0x7f8f400cf730 2023-12-08T08:38:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.449+0000 7f8f46272640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f400cd620 con 0x7f8f400cd240 2023-12-08T08:38:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.449+0000 7f8f46272640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4015c150 con 0x7f8f4015f4e0 2023-12-08T08:38:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.449+0000 7f8f3ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f400cd240 0x7f8f400cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.449+0000 7f8f3ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f400cd240 0x7f8f400cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56522/0 (socket says 172.21.15.16:56522) 2023-12-08T08:38:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.449+0000 7f8f3ffff640 1 -- 172.21.15.16:0/383690331 learned_addr learned my addr 172.21.15.16:0/383690331 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.450+0000 7f8f4480f640 1 --2- 172.21.15.16:0/383690331 >> v2:172.21.15.112:3300/0 conn(0x7f8f4015f4e0 0x7f8f4015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.450+0000 7f8f3ffff640 1 -- 172.21.15.16:0/383690331 >> v2:172.21.15.112:3300/0 conn(0x7f8f4015f4e0 msgr2=0x7f8f4015a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:25.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.450+0000 7f8f3ffff640 1 --2- 172.21.15.16:0/383690331 >> v2:172.21.15.112:3300/0 conn(0x7f8f4015f4e0 0x7f8f4015a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.450+0000 7f8f3ffff640 1 -- 172.21.15.16:0/383690331 >> v1:172.21.15.112:6789/0 conn(0x7f8f400cf730 legacy=0x7f8f4015edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:25.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.450+0000 7f8f3ffff640 1 -- 172.21.15.16:0/383690331 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4015fb00 con 0x7f8f400cd240 2023-12-08T08:38:25.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.450+0000 7f8f3ffff640 1 --2- 172.21.15.16:0/383690331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f400cd240 0x7f8f400cf1f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f400b0730 tx=0x7f8f3002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae531521e7d741d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:25.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.451+0000 7f8f3effd640 1 -- 172.21.15.16:0/383690331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30030860 con 0x7f8f400cd240 2023-12-08T08:38:25.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.451+0000 7f8f3effd640 1 -- 172.21.15.16:0/383690331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f300309c0 con 0x7f8f400cd240 2023-12-08T08:38:25.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.451+0000 7f8f3effd640 1 -- 172.21.15.16:0/383690331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30030d50 con 0x7f8f400cd240 2023-12-08T08:38:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.451+0000 7f8f46272640 1 -- 172.21.15.16:0/383690331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f400cd240 msgr2=0x7f8f400cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.451+0000 7f8f46272640 1 --2- 172.21.15.16:0/383690331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f400cd240 0x7f8f400cf1f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f400b0730 tx=0x7f8f3002b9a0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.451+0000 7f8f46272640 1 -- 172.21.15.16:0/383690331 shutdown_connections 2023-12-08T08:38:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.451+0000 7f8f46272640 1 --2- 172.21.15.16:0/383690331 >> v2:172.21.15.112:3300/0 conn(0x7f8f4015f4e0 0x7f8f4015a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.451+0000 7f8f46272640 1 --2- 172.21.15.16:0/383690331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f400cd240 0x7f8f400cf1f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.451+0000 7f8f46272640 1 -- 172.21.15.16:0/383690331 >> 172.21.15.16:0/383690331 conn(0x7f8f40099a10 msgr2=0x7f8f40099e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.451+0000 7f8f46272640 1 -- 172.21.15.16:0/383690331 shutdown_connections 2023-12-08T08:38:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.451+0000 7f8f46272640 1 -- 172.21.15.16:0/383690331 wait complete. 2023-12-08T08:38:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.452+0000 7f8f46272640 1 Processor -- start 2023-12-08T08:38:25.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.453+0000 7f8f46272640 1 -- start start 2023-12-08T08:38:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.453+0000 7f8f46272640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f400cd240 0x7f8f400b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.453+0000 7f8f46272640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f400cf730 0x7f8f4014f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.453+0000 7f8f46272640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4015f4e0 0x7f8f400b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.453+0000 7f8f46272640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f400b2a40 con 0x7f8f400cd240 2023-12-08T08:38:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.453+0000 7f8f46272640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f400b26d0 con 0x7f8f4015f4e0 2023-12-08T08:38:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.453+0000 7f8f46272640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f400b0d40 con 0x7f8f400cf730 2023-12-08T08:38:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.453+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f400cd240 0x7f8f400b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.453+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f400cd240 0x7f8f400b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56528/0 (socket says 172.21.15.16:56528) 2023-12-08T08:38:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.453+0000 7f8f3ffff640 1 -- 172.21.15.16:0/1813107803 learned_addr learned my addr 172.21.15.16:0/1813107803 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f4480f640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4015f4e0 0x7f8f400b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f3ffff640 1 -- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f400cf730 msgr2=0x7f8f4014f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f3f7fe640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f400cf730 0x7f8f4014f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f3ffff640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f400cf730 0x7f8f4014f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f3ffff640 1 -- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4015f4e0 msgr2=0x7f8f400b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f3ffff640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4015f4e0 0x7f8f400b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f3ffff640 1 -- 172.21.15.16:0/1813107803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f300094d0 con 0x7f8f400cd240 2023-12-08T08:38:25.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f4480f640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4015f4e0 0x7f8f400b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f3f7fe640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f400cf730 0x7f8f4014f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f3ffff640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f400cd240 0x7f8f400b0800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f30000c00 tx=0x7f8f30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f3cff9640 1 -- 172.21.15.16:0/1813107803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30039b60 con 0x7f8f400cd240 2023-12-08T08:38:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f3cff9640 1 -- 172.21.15.16:0/1813107803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f30039ce0 con 0x7f8f400cd240 2023-12-08T08:38:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f3cff9640 1 -- 172.21.15.16:0/1813107803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30043650 con 0x7f8f400cd240 2023-12-08T08:38:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f46272640 1 -- 172.21.15.16:0/1813107803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f400b0fe0 con 0x7f8f400cd240 2023-12-08T08:38:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.454+0000 7f8f46272640 1 -- 172.21.15.16:0/1813107803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f400b15a0 con 0x7f8f400cd240 2023-12-08T08:38:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.455+0000 7f8f46272640 1 -- 172.21.15.16:0/1813107803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f40067ad0 con 0x7f8f400cd240 2023-12-08T08:38:25.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.455+0000 7f8f3cff9640 1 -- 172.21.15.16:0/1813107803 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f3004c3b0 con 0x7f8f400cd240 2023-12-08T08:38:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.456+0000 7f8f3cff9640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8f2c06ae40 0x7f8f2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.456+0000 7f8f3cff9640 1 -- 172.21.15.16:0/1813107803 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8f300abfa0 con 0x7f8f400cd240 2023-12-08T08:38:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.456+0000 7f8f3f7fe640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8f2c06ae40 0x7f8f2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.457+0000 7f8f3f7fe640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8f2c06ae40 0x7f8f2c06d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f40157ac0 tx=0x7f8f34009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:25.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.460+0000 7f8f3cff9640 1 -- 172.21.15.16:0/1813107803 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f300dee90 con 0x7f8f400cd240 2023-12-08T08:38:25.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.639+0000 7f8f46272640 1 -- 172.21.15.16:0/1813107803 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f400cedc0 con 0x7f8f400cd240 2023-12-08T08:38:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.640+0000 7f8f3cff9640 1 -- 172.21.15.16:0/1813107803 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f8f30034090 con 0x7f8f400cd240 2023-12-08T08:38:25.643 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:25.643 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:24.030625+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.642+0000 7f8f46272640 1 -- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8f2c06ae40 msgr2=0x7f8f2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.642+0000 7f8f46272640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8f2c06ae40 0x7f8f2c06d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8f40157ac0 tx=0x7f8f34009040 comp rx=0 tx=0).stop 2023-12-08T08:38:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.642+0000 7f8f46272640 1 -- 172.21.15.16:0/1813107803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f400cd240 msgr2=0x7f8f400b0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.642+0000 7f8f46272640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f400cd240 0x7f8f400b0800 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f30000c00 tx=0x7f8f30004730 comp rx=0 tx=0).stop 2023-12-08T08:38:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.642+0000 7f8f46272640 1 -- 172.21.15.16:0/1813107803 shutdown_connections 2023-12-08T08:38:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.642+0000 7f8f46272640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8f2c06ae40 0x7f8f2c06d300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.642+0000 7f8f46272640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8f4015f4e0 0x7f8f400b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.642+0000 7f8f46272640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8f400cf730 0x7f8f4014f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.642+0000 7f8f46272640 1 --2- 172.21.15.16:0/1813107803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f400cd240 0x7f8f400b0800 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.642+0000 7f8f46272640 1 -- 172.21.15.16:0/1813107803 >> 172.21.15.16:0/1813107803 conn(0x7f8f40099a10 msgr2=0x7f8f4015e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:25.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.643+0000 7f8f46272640 1 -- 172.21.15.16:0/1813107803 shutdown_connections 2023-12-08T08:38:25.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.643+0000 7f8f46272640 1 -- 172.21.15.16:0/1813107803 wait complete. 2023-12-08T08:38:25.668 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-08T08:38:25.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.772+0000 7f9804064640 1 Processor -- start 2023-12-08T08:38:25.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.772+0000 7f9804064640 1 -- start start 2023-12-08T08:38:25.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.773+0000 7f9804064640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97fc0cc2e0 0x7f97fc0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.773+0000 7f9804064640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97fc0cfb50 con 0x7f97fc144930 2023-12-08T08:38:25.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.773+0000 7f9804064640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97fc0711e0 con 0x7f97fc144440 2023-12-08T08:38:25.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.773+0000 7f9804064640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0ca720 con 0x7f97fc0cc2e0 2023-12-08T08:38:25.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9802601640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f97fc144930 0x7f97fc0cd970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57384/0 (socket says 172.21.15.16:57384) 2023-12-08T08:38:25.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9802601640 1 -- 172.21.15.16:0/2309239561 learned_addr learned my addr 172.21.15.16:0/2309239561 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f98015ff640 1 --2- 172.21.15.16:0/2309239561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97fc0cc2e0 0x7f97fc0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 905598052 0 0) 0x7f97fc0cfb50 con 0x7f97fc144930 2023-12-08T08:38:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97e8003520 con 0x7f97fc144930 2023-12-08T08:38:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901525174 0 0) 0x7f97fc0711e0 con 0x7f97fc144440 2023-12-08T08:38:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97fc0cfb50 con 0x7f97fc144440 2023-12-08T08:38:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1983505993 0 0) 0x7f97e8003520 con 0x7f97fc144930 2023-12-08T08:38:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97fc0cc2e0 msgr2=0x7f97fc0cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 --2- 172.21.15.16:0/2309239561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97fc0cc2e0 0x7f97fc0cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 >> v1:172.21.15.112:6789/0 conn(0x7f97fc144440 legacy=0x7f97fc0cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc0714a0 con 0x7f97fc144930 2023-12-08T08:38:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f97f4001670 con 0x7f97fc144930 2023-12-08T08:38:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 >> v1:172.21.15.16:6789/0 conn(0x7f97fc144930 legacy=0x7f97fc0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 --2- 172.21.15.16:0/2309239561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e8003b30 0x7f97e8003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 --2- 172.21.15.16:0/2309239561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e80047c0 0x7f97e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 --2- 172.21.15.16:0/2309239561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97fc0cc2e0 0x7f97e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0ca720 con 0x7f97e80047c0 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e800ba30 con 0x7f97e8003b30 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e800bbf0 con 0x7f97fc0cc2e0 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f98015ff640 1 --2- 172.21.15.16:0/2309239561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e80047c0 0x7f97e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9801e00640 1 --2- 172.21.15.16:0/2309239561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e8003b30 0x7f97e8003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.774+0000 7f9802601640 1 --2- 172.21.15.16:0/2309239561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97fc0cc2e0 0x7f97e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.775+0000 7f98015ff640 1 -- 172.21.15.16:0/2309239561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97fc0cc2e0 msgr2=0x7f97e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.775+0000 7f98015ff640 1 --2- 172.21.15.16:0/2309239561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97fc0cc2e0 0x7f97e8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.775+0000 7f98015ff640 1 -- 172.21.15.16:0/2309239561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e8003b30 msgr2=0x7f97e8003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.775+0000 7f98015ff640 1 --2- 172.21.15.16:0/2309239561 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97e8003b30 0x7f97e8003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.775+0000 7f98015ff640 1 -- 172.21.15.16:0/2309239561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8003520 con 0x7f97e80047c0 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.775+0000 7f98015ff640 1 --2- 172.21.15.16:0/2309239561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e80047c0 0x7f97e8004c40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009ac0 tx=0x7f97e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6405e443edb6780 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.776+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec02c6b0 con 0x7f97e80047c0 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.776+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec02c810 con 0x7f97e80047c0 2023-12-08T08:38:25.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.776+0000 7f9800dfe640 1 -- 172.21.15.16:0/2309239561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec02cba0 con 0x7f97e80047c0 2023-12-08T08:38:25.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.776+0000 7f9804064640 1 -- 172.21.15.16:0/2309239561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e80047c0 msgr2=0x7f97e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.776+0000 7f9804064640 1 --2- 172.21.15.16:0/2309239561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e80047c0 0x7f97e8004c40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f97ec009ac0 tx=0x7f97e80057f0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.776+0000 7f9802601640 1 -- 172.21.15.16:0/2309239561 reap_dead start 2023-12-08T08:38:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.776+0000 7f9804064640 1 -- 172.21.15.16:0/2309239561 shutdown_connections 2023-12-08T08:38:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.776+0000 7f9804064640 1 -- 172.21.15.16:0/2309239561 >> 172.21.15.16:0/2309239561 conn(0x7f97fc0cfda0 msgr2=0x7f97fc0d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.776+0000 7f9804064640 1 -- 172.21.15.16:0/2309239561 shutdown_connections 2023-12-08T08:38:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.776+0000 7f9804064640 1 -- 172.21.15.16:0/2309239561 wait complete. 2023-12-08T08:38:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.778+0000 7f9804064640 1 Processor -- start 2023-12-08T08:38:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.778+0000 7f9804064640 1 -- start start 2023-12-08T08:38:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.778+0000 7f9804064640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97e80047c0 0x7f97fc141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.778+0000 7f9804064640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97fc0b29b0 0x7f97fc0b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.778+0000 7f9804064640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc142010 0x7f97fc142470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.778+0000 7f9804064640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc1e0530 con 0x7f97fc0b29b0 2023-12-08T08:38:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.778+0000 7f9804064640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc1429b0 con 0x7f97fc142010 2023-12-08T08:38:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.778+0000 7f9804064640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc09dbc0 con 0x7f97e80047c0 2023-12-08T08:38:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.778+0000 7f98015ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97fc0b29b0 0x7f97fc0b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.778+0000 7f98015ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97fc0b29b0 0x7f97fc0b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56550/0 (socket says 172.21.15.16:56550) 2023-12-08T08:38:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.778+0000 7f98015ff640 1 -- 172.21.15.16:0/3894612028 learned_addr learned my addr 172.21.15.16:0/3894612028 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f9801e00640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97e80047c0 0x7f97fc141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f9802601640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc142010 0x7f97fc142470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f98015ff640 1 -- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97e80047c0 msgr2=0x7f97fc141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f98015ff640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97e80047c0 0x7f97fc141630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f98015ff640 1 -- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc142010 msgr2=0x7f97fc142470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f98015ff640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc142010 0x7f97fc142470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f98015ff640 1 -- 172.21.15.16:0/3894612028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc0b3520 con 0x7f97fc0b29b0 2023-12-08T08:38:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f9801e00640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97e80047c0 0x7f97fc141630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f9802601640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc142010 0x7f97fc142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f98015ff640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97fc0b29b0 0x7f97fc0b2e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f97ec02c250 tx=0x7f97ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f97f27fc640 1 -- 172.21.15.16:0/3894612028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec003810 con 0x7f97fc0b29b0 2023-12-08T08:38:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f97f27fc640 1 -- 172.21.15.16:0/3894612028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec003970 con 0x7f97fc0b29b0 2023-12-08T08:38:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f9804064640 1 -- 172.21.15.16:0/3894612028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97fc0b3800 con 0x7f97fc0b29b0 2023-12-08T08:38:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f97f27fc640 1 -- 172.21.15.16:0/3894612028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec003ce0 con 0x7f97fc0b29b0 2023-12-08T08:38:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.779+0000 7f9804064640 1 -- 172.21.15.16:0/3894612028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97fc0b3dc0 con 0x7f97fc0b29b0 2023-12-08T08:38:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.780+0000 7f9804064640 1 -- 172.21.15.16:0/3894612028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97fc067430 con 0x7f97fc0b29b0 2023-12-08T08:38:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.780+0000 7f97f27fc640 1 -- 172.21.15.16:0/3894612028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f97ec047450 con 0x7f97fc0b29b0 2023-12-08T08:38:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.781+0000 7f97f27fc640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f97e006ae40 0x7f97e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.781+0000 7f97f27fc640 1 -- 172.21.15.16:0/3894612028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f97ec0aba50 con 0x7f97fc0b29b0 2023-12-08T08:38:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.781+0000 7f9801e00640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f97e006ae40 0x7f97e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.782+0000 7f9801e00640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f97e006ae40 0x7f97e006d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f97f8003e90 tx=0x7f97f8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.785+0000 7f97f27fc640 1 -- 172.21.15.16:0/3894612028 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f97ec076950 con 0x7f97fc0b29b0 2023-12-08T08:38:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.997+0000 7f9804064640 1 -- 172.21.15.16:0/3894612028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97fc141dd0 con 0x7f97fc0b29b0 2023-12-08T08:38:25.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.997+0000 7f97f27fc640 1 -- 172.21.15.16:0/3894612028 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f97ec0044e0 con 0x7f97fc0b29b0 2023-12-08T08:38:26.000 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:26.000 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/307039451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":307039451},{"type":"v1","addr":"172.21.15.112:6836","nonce":307039451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1424152341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1424152341},{"type":"v1","addr":"172.21.15.16:6833","nonce":1424152341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3597811670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3597811670},{"type":"v1","addr":"172.21.15.16:6835","nonce":3597811670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-08T08:38:18.991090+0000","modified":"2023-12-08T08:38:24.030777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1082241707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1082241707},{"type":"v1","addr":"172.21.15.112:6837","nonce":1082241707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-12-08T08:38:26.000 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2023-12-08T08:38:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.999+0000 7f9804064640 1 -- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f97e006ae40 msgr2=0x7f97e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.999+0000 7f9804064640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f97e006ae40 0x7f97e006d300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f97f8003e90 tx=0x7f97f8009040 comp rx=0 tx=0).stop 2023-12-08T08:38:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.999+0000 7f9804064640 1 -- 172.21.15.16:0/3894612028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97fc0b29b0 msgr2=0x7f97fc0b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.999+0000 7f9804064640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97fc0b29b0 0x7f97fc0b2e30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f97ec02c250 tx=0x7f97ec004730 comp rx=0 tx=0).stop 2023-12-08T08:38:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.999+0000 7f9804064640 1 -- 172.21.15.16:0/3894612028 shutdown_connections 2023-12-08T08:38:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.999+0000 7f9804064640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f97e006ae40 0x7f97e006d300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.999+0000 7f9804064640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97fc142010 0x7f97fc142470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.999+0000 7f9804064640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97fc0b29b0 0x7f97fc0b2e30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.999+0000 7f9804064640 1 --2- 172.21.15.16:0/3894612028 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97e80047c0 0x7f97fc141630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.999+0000 7f9804064640 1 -- 172.21.15.16:0/3894612028 >> 172.21.15.16:0/3894612028 conn(0x7f97fc0cfda0 msgr2=0x7f97fc143570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.999+0000 7f9804064640 1 -- 172.21.15.16:0/3894612028 shutdown_connections 2023-12-08T08:38:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:25.999+0000 7f9804064640 1 -- 172.21.15.16:0/3894612028 wait complete. 2023-12-08T08:38:26.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.116+0000 7f77cd236640 1 Processor -- start 2023-12-08T08:38:26.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.116+0000 7f77cd236640 1 -- start start 2023-12-08T08:38:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.117+0000 7f77cd236640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77c815d580 0x7f77c80d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.117+0000 7f77cd236640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f77c815f890 0x7f77c815d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.117+0000 7f77cd236640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77c815ee90 con 0x7f77c80d3ad0 2023-12-08T08:38:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.117+0000 7f77cd236640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77c80d0fe0 con 0x7f77c815f890 2023-12-08T08:38:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.117+0000 7f77cd236640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c80d1280 con 0x7f77c815d580 2023-12-08T08:38:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.117+0000 7f77c759e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f77c815f890 0x7f77c815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.117+0000 7f77c6d9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77c815d580 0x7f77c80d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.117+0000 7f77c659c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f77c80d3ad0 0x7f77c815f180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:34198/0 (socket says 172.21.15.16:34198) 2023-12-08T08:38:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.117+0000 7f77c659c640 1 -- 172.21.15.16:0/522995574 learned_addr learned my addr 172.21.15.16:0/522995574 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.118+0000 7f77c6d9d640 1 -- 172.21.15.16:0/522995574 >> v2:172.21.15.112:3300/0 conn(0x7f77c815f890 msgr2=0x7f77c815d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.118+0000 7f77c6d9d640 1 --2- 172.21.15.16:0/522995574 >> v2:172.21.15.112:3300/0 conn(0x7f77c815f890 0x7f77c815d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.118+0000 7f77c6d9d640 1 -- 172.21.15.16:0/522995574 >> v1:172.21.15.112:6789/0 conn(0x7f77c80d3ad0 legacy=0x7f77c815f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.118+0000 7f77c6d9d640 1 -- 172.21.15.16:0/522995574 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c80d1560 con 0x7f77c815d580 2023-12-08T08:38:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.118+0000 7f77c759e640 1 --2- 172.21.15.16:0/522995574 >> v2:172.21.15.112:3300/0 conn(0x7f77c815f890 0x7f77c815d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:26.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.118+0000 7f77c6d9d640 1 --2- 172.21.15.16:0/522995574 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77c815d580 0x7f77c80d1d60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f77b80097a0 tx=0x7f77b802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34c4735147ccc37c server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.119+0000 7f77c5d9b640 1 -- 172.21.15.16:0/522995574 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b802e070 con 0x7f77c815d580 2023-12-08T08:38:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.119+0000 7f77c5d9b640 1 -- 172.21.15.16:0/522995574 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b80315a0 con 0x7f77c815d580 2023-12-08T08:38:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.119+0000 7f77c5d9b640 1 -- 172.21.15.16:0/522995574 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b8031930 con 0x7f77c815d580 2023-12-08T08:38:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.119+0000 7f77cd236640 1 -- 172.21.15.16:0/522995574 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77c815d580 msgr2=0x7f77c80d1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.119+0000 7f77cd236640 1 --2- 172.21.15.16:0/522995574 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77c815d580 0x7f77c80d1d60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f77b80097a0 tx=0x7f77b802b840 comp rx=0 tx=0).stop 2023-12-08T08:38:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.119+0000 7f77cd236640 1 -- 172.21.15.16:0/522995574 shutdown_connections 2023-12-08T08:38:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.119+0000 7f77cd236640 1 --2- 172.21.15.16:0/522995574 >> v2:172.21.15.112:3300/0 conn(0x7f77c815f890 0x7f77c815d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.119+0000 7f77cd236640 1 --2- 172.21.15.16:0/522995574 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77c815d580 0x7f77c80d1d60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.119+0000 7f77cd236640 1 -- 172.21.15.16:0/522995574 >> 172.21.15.16:0/522995574 conn(0x7f77c80d2fb0 msgr2=0x7f77c80d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.119+0000 7f77cd236640 1 -- 172.21.15.16:0/522995574 shutdown_connections 2023-12-08T08:38:26.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.119+0000 7f77cd236640 1 -- 172.21.15.16:0/522995574 wait complete. 2023-12-08T08:38:26.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.121+0000 7f77cd236640 1 Processor -- start 2023-12-08T08:38:26.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.121+0000 7f77cd236640 1 -- start start 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.121+0000 7f77cd236640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c80d3ad0 0x7f77c80ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.121+0000 7f77cd236640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c815d580 0x7f77c80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77cd236640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77c815f890 0x7f77c80c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77cd236640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c80b2440 con 0x7f77c80d3ad0 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77cd236640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77c80b26a0 con 0x7f77c815d580 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77cd236640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77c80b2980 con 0x7f77c815f890 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c80d3ad0 0x7f77c80ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c80d3ad0 0x7f77c80ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56564/0 (socket says 172.21.15.16:56564) 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77c6d9d640 1 -- 172.21.15.16:0/3479722021 learned_addr learned my addr 172.21.15.16:0/3479722021 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77c659c640 1 --2- 172.21.15.16:0/3479722021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c815d580 0x7f77c80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77c759e640 1 --2- 172.21.15.16:0/3479722021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77c815f890 0x7f77c80c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77c6d9d640 1 -- 172.21.15.16:0/3479722021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77c815f890 msgr2=0x7f77c80c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77c6d9d640 1 --2- 172.21.15.16:0/3479722021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77c815f890 0x7f77c80c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77c6d9d640 1 -- 172.21.15.16:0/3479722021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c815d580 msgr2=0x7f77c80b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77c6d9d640 1 --2- 172.21.15.16:0/3479722021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c815d580 0x7f77c80b1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77c6d9d640 1 -- 172.21.15.16:0/3479722021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c0009a70 con 0x7f77c80d3ad0 2023-12-08T08:38:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77c6d9d640 1 --2- 172.21.15.16:0/3479722021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c80d3ad0 0x7f77c80ce800 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f77b8034040 tx=0x7f77b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77af7fe640 1 -- 172.21.15.16:0/3479722021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b8032840 con 0x7f77c80d3ad0 2023-12-08T08:38:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77af7fe640 1 -- 172.21.15.16:0/3479722021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77b80329c0 con 0x7f77c80d3ad0 2023-12-08T08:38:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77af7fe640 1 -- 172.21.15.16:0/3479722021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77b8032d70 con 0x7f77c80d3ad0 2023-12-08T08:38:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77cd236640 1 -- 172.21.15.16:0/3479722021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b8009400 con 0x7f77c80d3ad0 2023-12-08T08:38:26.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.122+0000 7f77cd236640 1 -- 172.21.15.16:0/3479722021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77c80b2f40 con 0x7f77c80d3ad0 2023-12-08T08:38:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.123+0000 7f77cd236640 1 -- 172.21.15.16:0/3479722021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77c80d09d0 con 0x7f77c80d3ad0 2023-12-08T08:38:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.124+0000 7f77af7fe640 1 -- 172.21.15.16:0/3479722021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f77b803b3f0 con 0x7f77c80d3ad0 2023-12-08T08:38:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.124+0000 7f77af7fe640 1 --2- 172.21.15.16:0/3479722021 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f77b406ae70 0x7f77b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.124+0000 7f77af7fe640 1 -- 172.21.15.16:0/3479722021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f77b80ab7b0 con 0x7f77c80d3ad0 2023-12-08T08:38:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.125+0000 7f77c659c640 1 --2- 172.21.15.16:0/3479722021 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f77b406ae70 0x7f77b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.125+0000 7f77c659c640 1 --2- 172.21.15.16:0/3479722021 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f77b406ae70 0x7f77b406d330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f77c80c64b0 tx=0x7f77bc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:26.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.129+0000 7f77af7fe640 1 -- 172.21.15.16:0/3479722021 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f77b8076570 con 0x7f77c80d3ad0 2023-12-08T08:38:26.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.304+0000 7f77cd236640 1 -- 172.21.15.16:0/3479722021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77c8067430 con 0x7f77c80d3ad0 2023-12-08T08:38:26.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.305+0000 7f77af7fe640 1 -- 172.21.15.16:0/3479722021 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f77b8079b90 con 0x7f77c80d3ad0 2023-12-08T08:38:26.307 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:26.308 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:24.030625+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:26.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.307+0000 7f77cd236640 1 -- 172.21.15.16:0/3479722021 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f77b406ae70 msgr2=0x7f77b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.307+0000 7f77cd236640 1 --2- 172.21.15.16:0/3479722021 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f77b406ae70 0x7f77b406d330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f77c80c64b0 tx=0x7f77bc009040 comp rx=0 tx=0).stop 2023-12-08T08:38:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.307+0000 7f77cd236640 1 -- 172.21.15.16:0/3479722021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c80d3ad0 msgr2=0x7f77c80ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.307+0000 7f77cd236640 1 --2- 172.21.15.16:0/3479722021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c80d3ad0 0x7f77c80ce800 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f77b8034040 tx=0x7f77b80046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.307+0000 7f77cd236640 1 -- 172.21.15.16:0/3479722021 shutdown_connections 2023-12-08T08:38:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.307+0000 7f77cd236640 1 --2- 172.21.15.16:0/3479722021 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f77b406ae70 0x7f77b406d330 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.307+0000 7f77cd236640 1 --2- 172.21.15.16:0/3479722021 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77c815f890 0x7f77c80c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.307+0000 7f77cd236640 1 --2- 172.21.15.16:0/3479722021 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77c815d580 0x7f77c80b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.307+0000 7f77cd236640 1 --2- 172.21.15.16:0/3479722021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77c80d3ad0 0x7f77c80ce800 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.307+0000 7f77cd236640 1 -- 172.21.15.16:0/3479722021 >> 172.21.15.16:0/3479722021 conn(0x7f77c80d2fb0 msgr2=0x7f77c806b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.307+0000 7f77cd236640 1 -- 172.21.15.16:0/3479722021 shutdown_connections 2023-12-08T08:38:26.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.307+0000 7f77cd236640 1 -- 172.21.15.16:0/3479722021 wait complete. 2023-12-08T08:38:26.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-08T08:38:18.991090+0000', 'modified': '2023-12-08T08:38:24.030777+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4297}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4297': {'gid': 4297, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6837/1082241707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6835', 'nonce': 1082241707}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 1082241707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-12-08T08:38:26.722 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-08T08:38:26.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-08T08:38:26.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.835+0000 7f7104a97640 1 Processor -- start 2023-12-08T08:38:26.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.835+0000 7f7104a97640 1 -- start start 2023-12-08T08:38:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f7104a97640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71000b3bf0 0x7f71000d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f7104a97640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71000cdc20 con 0x7f71000d1d00 2023-12-08T08:38:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f7104a97640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71000d11d0 con 0x7f71000b3340 2023-12-08T08:38:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f7104a97640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71000d3a60 con 0x7f71000b3bf0 2023-12-08T08:38:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f70f7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f71000d1d00 0x7f71000ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52054/0 (socket says 172.21.15.16:52054) 2023-12-08T08:38:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f70f7fff640 1 -- 172.21.15.16:0/1360713032 learned_addr learned my addr 172.21.15.16:0/1360713032 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f70fed9d640 1 --2- 172.21.15.16:0/1360713032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71000b3bf0 0x7f71000d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f70fdd9b640 1 -- 172.21.15.16:0/1360713032 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565644046 0 0) 0x7f71000cdc20 con 0x7f71000d1d00 2023-12-08T08:38:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f70fdd9b640 1 -- 172.21.15.16:0/1360713032 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70ec003520 con 0x7f71000d1d00 2023-12-08T08:38:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f70fdd9b640 1 -- 172.21.15.16:0/1360713032 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1443728579 0 0) 0x7f70ec003520 con 0x7f71000d1d00 2023-12-08T08:38:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f70fdd9b640 1 -- 172.21.15.16:0/1360713032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71000b3bf0 msgr2=0x7f71000d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f70fdd9b640 1 --2- 172.21.15.16:0/1360713032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71000b3bf0 0x7f71000d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f70fdd9b640 1 -- 172.21.15.16:0/1360713032 >> v1:172.21.15.112:6789/0 conn(0x7f71000b3340 legacy=0x7f71000d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f70fdd9b640 1 -- 172.21.15.16:0/1360713032 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71000d3d40 con 0x7f71000d1d00 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.836+0000 7f70fdd9b640 1 -- 172.21.15.16:0/1360713032 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f70e8001670 con 0x7f71000d1d00 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70fdd9b640 1 -- 172.21.15.16:0/1360713032 >> v1:172.21.15.16:6789/0 conn(0x7f71000d1d00 legacy=0x7f71000ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70fdd9b640 1 --2- 172.21.15.16:0/1360713032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70ec0039a0 0x7f70ec003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70fdd9b640 1 --2- 172.21.15.16:0/1360713032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71000b3bf0 0x7f70ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70fdd9b640 1 --2- 172.21.15.16:0/1360713032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70ec004d00 0x7f70ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70fdd9b640 1 -- 172.21.15.16:0/1360713032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71000d3a60 con 0x7f70ec0039a0 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70fdd9b640 1 -- 172.21.15.16:0/1360713032 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec00ba30 con 0x7f70ec004d00 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70fdd9b640 1 -- 172.21.15.16:0/1360713032 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec00bbf0 con 0x7f71000b3bf0 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70fe59c640 1 --2- 172.21.15.16:0/1360713032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70ec0039a0 0x7f70ec003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70f7fff640 1 --2- 172.21.15.16:0/1360713032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71000b3bf0 0x7f70ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70fed9d640 1 --2- 172.21.15.16:0/1360713032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70ec004d00 0x7f70ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70fe59c640 1 -- 172.21.15.16:0/1360713032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71000b3bf0 msgr2=0x7f70ec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70fe59c640 1 --2- 172.21.15.16:0/1360713032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71000b3bf0 0x7f70ec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70fe59c640 1 -- 172.21.15.16:0/1360713032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70ec004d00 msgr2=0x7f70ec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:26.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70fe59c640 1 --2- 172.21.15.16:0/1360713032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f70ec004d00 0x7f70ec005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.837+0000 7f70fe59c640 1 -- 172.21.15.16:0/1360713032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec003520 con 0x7f70ec0039a0 2023-12-08T08:38:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.838+0000 7f70fe59c640 1 --2- 172.21.15.16:0/1360713032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70ec0039a0 0x7f70ec003d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f70f0001860 tx=0x7f70ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c83af7500e04e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.838+0000 7f70fdd9b640 1 -- 172.21.15.16:0/1360713032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0033630 con 0x7f70ec0039a0 2023-12-08T08:38:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.838+0000 7f70fdd9b640 1 -- 172.21.15.16:0/1360713032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f00337b0 con 0x7f70ec0039a0 2023-12-08T08:38:26.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.838+0000 7f70fdd9b640 1 -- 172.21.15.16:0/1360713032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0033b60 con 0x7f70ec0039a0 2023-12-08T08:38:26.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.838+0000 7f7104a97640 1 -- 172.21.15.16:0/1360713032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70ec0039a0 msgr2=0x7f70ec003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.838+0000 7f7104a97640 1 --2- 172.21.15.16:0/1360713032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70ec0039a0 0x7f70ec003d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f70f0001860 tx=0x7f70ec0057f0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.838+0000 7f70fed9d640 1 -- 172.21.15.16:0/1360713032 reap_dead start 2023-12-08T08:38:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.838+0000 7f7104a97640 1 -- 172.21.15.16:0/1360713032 shutdown_connections 2023-12-08T08:38:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.838+0000 7f7104a97640 1 -- 172.21.15.16:0/1360713032 >> 172.21.15.16:0/1360713032 conn(0x7f710009c2f0 msgr2=0x7f71000d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.838+0000 7f7104a97640 1 -- 172.21.15.16:0/1360713032 shutdown_connections 2023-12-08T08:38:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.838+0000 7f7104a97640 1 -- 172.21.15.16:0/1360713032 wait complete. 2023-12-08T08:38:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.840+0000 7f7104a97640 1 Processor -- start 2023-12-08T08:38:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.840+0000 7f7104a97640 1 -- start start 2023-12-08T08:38:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f7104a97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70ec0039a0 0x7f71000c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f7104a97640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7100149910 0x7f7100156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f7104a97640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7100148c90 0x7f7100149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f7104a97640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710015a170 con 0x7f70ec0039a0 2023-12-08T08:38:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f7104a97640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7100149650 con 0x7f7100148c90 2023-12-08T08:38:26.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f7104a97640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7100159530 con 0x7f7100149910 2023-12-08T08:38:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f70fe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70ec0039a0 0x7f71000c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f70fe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70ec0039a0 0x7f71000c92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39254/0 (socket says 172.21.15.16:39254) 2023-12-08T08:38:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f70fe59c640 1 -- 172.21.15.16:0/1940986224 learned_addr learned my addr 172.21.15.16:0/1940986224 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f70fed9d640 1 --2- 172.21.15.16:0/1940986224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7100148c90 0x7f7100149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f70f7fff640 1 --2- 172.21.15.16:0/1940986224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7100149910 0x7f7100156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f70fe59c640 1 -- 172.21.15.16:0/1940986224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7100149910 msgr2=0x7f7100156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f70fe59c640 1 --2- 172.21.15.16:0/1940986224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7100149910 0x7f7100156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f70fe59c640 1 -- 172.21.15.16:0/1940986224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7100148c90 msgr2=0x7f7100149110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f70fe59c640 1 --2- 172.21.15.16:0/1940986224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7100148c90 0x7f7100149110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f70fe59c640 1 -- 172.21.15.16:0/1940986224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71000c97f0 con 0x7f70ec0039a0 2023-12-08T08:38:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f70fe59c640 1 --2- 172.21.15.16:0/1940986224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70ec0039a0 0x7f71000c92b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70f002bdf0 tx=0x7f70f0001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f70f6ffd640 1 -- 172.21.15.16:0/1940986224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f00037f0 con 0x7f70ec0039a0 2023-12-08T08:38:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f70f6ffd640 1 -- 172.21.15.16:0/1940986224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f0003950 con 0x7f70ec0039a0 2023-12-08T08:38:26.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f7104a97640 1 -- 172.21.15.16:0/1940986224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71001597d0 con 0x7f70ec0039a0 2023-12-08T08:38:26.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.841+0000 7f70f6ffd640 1 -- 172.21.15.16:0/1940986224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0003d00 con 0x7f70ec0039a0 2023-12-08T08:38:26.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.842+0000 7f7104a97640 1 -- 172.21.15.16:0/1940986224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7100159d90 con 0x7f70ec0039a0 2023-12-08T08:38:26.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.843+0000 7f70f6ffd640 1 -- 172.21.15.16:0/1940986224 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f70f0047460 con 0x7f70ec0039a0 2023-12-08T08:38:26.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.843+0000 7f70f6ffd640 1 --2- 172.21.15.16:0/1940986224 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f70e006ae40 0x7f70e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.843+0000 7f70f6ffd640 1 -- 172.21.15.16:0/1940986224 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f70f00abfb0 con 0x7f70ec0039a0 2023-12-08T08:38:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.843+0000 7f70f7fff640 1 --2- 172.21.15.16:0/1940986224 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f70e006ae40 0x7f70e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.844+0000 7f70f7fff640 1 --2- 172.21.15.16:0/1940986224 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f70e006ae40 0x7f70e006d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f710006cd50 tx=0x7f70e8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.844+0000 7f710762a740 1 Processor -- start 2023-12-08T08:38:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.845+0000 7f710762a740 1 -- start start 2023-12-08T08:38:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.845+0000 7f710762a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12cc50 0x56407d0ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.845+0000 7f710762a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12d030 0x56407d12f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.845+0000 7f710762a740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d100130 0x56407d12ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.845+0000 7f710762a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56407d0c0980 con 0x56407d12cc50 2023-12-08T08:38:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.845+0000 7f710762a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56407d072b90 con 0x56407d12d030 2023-12-08T08:38:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.845+0000 7f710762a740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56407d070280 con 0x56407d100130 2023-12-08T08:38:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.845+0000 7f70fe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12cc50 0x56407d0ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.845+0000 7f70fe59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12cc50 0x56407d0ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39266/0 (socket says 172.21.15.16:39266) 2023-12-08T08:38:26.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.845+0000 7f70fe59c640 1 -- 172.21.15.16:0/3263947903 learned_addr learned my addr 172.21.15.16:0/3263947903 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.846+0000 7f70f7fff640 1 --2- 172.21.15.16:0/3263947903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12d030 0x56407d12f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.846+0000 7f70fed9d640 1 --2- 172.21.15.16:0/3263947903 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d100130 0x56407d12ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.846+0000 7f70fe59c640 1 -- 172.21.15.16:0/3263947903 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d100130 msgr2=0x56407d12ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.846+0000 7f70fe59c640 1 --2- 172.21.15.16:0/3263947903 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d100130 0x56407d12ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.846+0000 7f70fe59c640 1 -- 172.21.15.16:0/3263947903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12d030 msgr2=0x56407d12f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.846+0000 7f70fe59c640 1 --2- 172.21.15.16:0/3263947903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12d030 0x56407d12f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.846+0000 7f70fe59c640 1 -- 172.21.15.16:0/3263947903 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56407d031500 con 0x56407d12cc50 2023-12-08T08:38:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.846+0000 7f70fe59c640 1 --2- 172.21.15.16:0/3263947903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12cc50 0x56407d0ffbf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f70f002d010 tx=0x7f70f007a390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a5e03d6bb2f84c server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.846+0000 7f70dffff640 1 -- 172.21.15.16:0/3263947903 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f007b130 con 0x56407d12cc50 2023-12-08T08:38:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.846+0000 7f70dffff640 1 -- 172.21.15.16:0/3263947903 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f007aa30 con 0x56407d12cc50 2023-12-08T08:38:26.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.846+0000 7f70dffff640 1 -- 172.21.15.16:0/3263947903 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f007eb80 con 0x56407d12cc50 2023-12-08T08:38:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.847+0000 7f710762a740 1 -- 172.21.15.16:0/3263947903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12cc50 msgr2=0x56407d0ffbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.847+0000 7f710762a740 1 --2- 172.21.15.16:0/3263947903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12cc50 0x56407d0ffbf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f70f002d010 tx=0x7f70f007a390 comp rx=0 tx=0).stop 2023-12-08T08:38:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.847+0000 7f710762a740 1 -- 172.21.15.16:0/3263947903 shutdown_connections 2023-12-08T08:38:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.847+0000 7f710762a740 1 --2- 172.21.15.16:0/3263947903 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d100130 0x56407d12ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.847+0000 7f710762a740 1 --2- 172.21.15.16:0/3263947903 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12d030 0x56407d12f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.847+0000 7f710762a740 1 --2- 172.21.15.16:0/3263947903 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12cc50 0x56407d0ffbf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.847+0000 7f710762a740 1 -- 172.21.15.16:0/3263947903 >> 172.21.15.16:0/3263947903 conn(0x56407d101e60 msgr2=0x56407d14a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.847+0000 7f710762a740 1 -- 172.21.15.16:0/3263947903 shutdown_connections 2023-12-08T08:38:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.847+0000 7f710762a740 1 -- 172.21.15.16:0/3263947903 wait complete. 2023-12-08T08:38:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.847+0000 7f710762a740 1 -- start start 2023-12-08T08:38:26.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.847+0000 7f710762a740 1 Processor -- start 2023-12-08T08:38:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.848+0000 7f710762a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 0x56407d120fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.848+0000 7f710762a740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d12d030 0x56407d132400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.848+0000 7f710762a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d1487a0 0x56407d147c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.848+0000 7f710762a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56407d06bb70 con 0x56407d1487a0 2023-12-08T08:38:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.848+0000 7f710762a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56407d025e00 con 0x56407d12cc50 2023-12-08T08:38:26.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.848+0000 7f710762a740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56407d025aa0 con 0x56407d12d030 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.848+0000 7f710762a740 10 client.0 _ll_register_callbacks cb 0x56407cd7dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.848+0000 7f70fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d1487a0 0x56407d147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d1487a0 0x56407d147c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39270/0 (socket says 172.21.15.16:39270) 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70fed9d640 1 -- 172.21.15.16:0/3355356268 learned_addr learned my addr 172.21.15.16:0/3355356268 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70f7fff640 1 --2- 172.21.15.16:0/3355356268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d12d030 0x56407d132400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70fe59c640 1 --2- 172.21.15.16:0/3355356268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 0x56407d120fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70fed9d640 1 -- 172.21.15.16:0/3355356268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d12d030 msgr2=0x56407d132400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70fed9d640 1 --2- 172.21.15.16:0/3355356268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d12d030 0x56407d132400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70fed9d640 1 -- 172.21.15.16:0/3355356268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 msgr2=0x56407d120fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70fed9d640 1 --2- 172.21.15.16:0/3355356268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 0x56407d120fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70fed9d640 1 -- 172.21.15.16:0/3355356268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7100159d90 con 0x56407d1487a0 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70fed9d640 1 --2- 172.21.15.16:0/3355356268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d1487a0 0x56407d147c00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f70f8015d20 tx=0x7f70f8008690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883caa04ec8c0bbc server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70dd7fa640 1 -- 172.21.15.16:0/3355356268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f800e040 con 0x56407d1487a0 2023-12-08T08:38:26.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70dd7fa640 1 -- 172.21.15.16:0/3355356268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f70f8003990 con 0x56407d1487a0 2023-12-08T08:38:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f710762a740 1 -- 172.21.15.16:0/3355356268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56407d064b50 con 0x56407d1487a0 2023-12-08T08:38:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70dd7fa640 1 -- 172.21.15.16:0/3355356268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70f8009ea0 con 0x56407d1487a0 2023-12-08T08:38:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f710762a740 10 client.4422 fetch_fsmap learned FSMap version 7 2023-12-08T08:38:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f710762a740 1 -- 172.21.15.16:0/3355356268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x56407d06a1f0 con 0x56407d1487a0 2023-12-08T08:38:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.849+0000 7f70dd7fa640 1 -- 172.21.15.16:0/3355356268 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56407d064b50 con 0x56407d1487a0 2023-12-08T08:38:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.850+0000 7f70dd7fa640 1 -- 172.21.15.16:0/3355356268 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f70f801d020 con 0x56407d1487a0 2023-12-08T08:38:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.850+0000 7f710762a740 10 client.4422 fetch_fsmap finished waiting for FSMap version 7 2023-12-08T08:38:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.850+0000 7f710762a740 10 client.4422 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-08T08:38:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.850+0000 7f710762a740 20 client.4422 populate_metadata read hostname 'smithi016' 2023-12-08T08:38:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.850+0000 7f710762a740 1 --2- 172.21.15.16:0/3355356268 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x56407d14a070 0x56407d145df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.850+0000 7f710762a740 4 client.4422 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-08T08:38:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.850+0000 7f710762a740 1 -- 172.21.15.16:0/3355356268 --> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56407d064870 con 0x56407d14a070 2023-12-08T08:38:26.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.850+0000 7f70fe59c640 1 --2- 172.21.15.16:0/3355356268 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x56407d14a070 0x56407d145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.850+0000 7f70fe59c640 1 --2- 172.21.15.16:0/3355356268 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x56407d14a070 0x56407d145df0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b21edd454843310 server_cookie=a34134940ec3a36e in_seq=0 out_seq=0 2023-12-08T08:38:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.850+0000 7f70dd7fa640 10 client.4422 ms_handle_connect on v2:172.21.15.112:6835/1082241707 2023-12-08T08:38:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f70dd7fa640 1 -- 172.21.15.16:0/3355356268 <== mds.0 v2:172.21.15.112:6835/1082241707 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x56407d064870 con 0x56407d14a070 2023-12-08T08:38:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f70dd7fa640 10 client.4422 handle_command_reply: tid=0 2023-12-08T08:38:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f710762a740 1 client.4422 shutdown 2023-12-08T08:38:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f710762a740 1 -- 172.21.15.16:0/3355356268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d1487a0 msgr2=0x56407d147c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f710762a740 1 --2- 172.21.15.16:0/3355356268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d1487a0 0x56407d147c00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f70f8015d20 tx=0x7f70f8008690 comp rx=0 tx=0).stop 2023-12-08T08:38:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f710762a740 1 -- 172.21.15.16:0/3355356268 shutdown_connections 2023-12-08T08:38:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f710762a740 1 --2- 172.21.15.16:0/3355356268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d1487a0 0x56407d147c00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f710762a740 1 --2- 172.21.15.16:0/3355356268 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x56407d14a070 0x56407d145df0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f710762a740 1 --2- 172.21.15.16:0/3355356268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d12d030 0x56407d132400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f710762a740 1 --2- 172.21.15.16:0/3355356268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 0x56407d120fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f710762a740 1 -- 172.21.15.16:0/3355356268 >> 172.21.15.16:0/3355356268 conn(0x56407d100130 msgr2=0x56407d12f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:26.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f70dd7fa640 0 client.4422 ms_handle_reset on v2:172.21.15.112:6835/1082241707 2023-12-08T08:38:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f710762a740 1 -- 172.21.15.16:0/3355356268 shutdown_connections 2023-12-08T08:38:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f710762a740 1 -- 172.21.15.16:0/3355356268 wait complete. 2023-12-08T08:38:26.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.851+0000 7f710762a740 20 client.4422 trim_cache size 0 max 16384 2023-12-08T08:38:26.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.867+0000 7f710762a740 1 Processor -- start 2023-12-08T08:38:26.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.867+0000 7f710762a740 1 -- start start 2023-12-08T08:38:26.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f710762a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 0x56407d1037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f710762a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12d030 0x56407d103d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f710762a740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d1487a0 0x56407d14d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f710762a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56407cfeb270 con 0x56407d12d030 2023-12-08T08:38:26.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f710762a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56407d12fa40 con 0x56407d12cc50 2023-12-08T08:38:26.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f710762a740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56407d12d410 con 0x56407d1487a0 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f70f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12d030 0x56407d103d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f70f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12d030 0x56407d103d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39286/0 (socket says 172.21.15.16:39286) 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f70f7fff640 1 -- 172.21.15.16:0/3610967959 learned_addr learned my addr 172.21.15.16:0/3610967959 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f70fed9d640 1 --2- 172.21.15.16:0/3610967959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d1487a0 0x56407d14d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f70fe59c640 1 --2- 172.21.15.16:0/3610967959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 0x56407d1037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f70f7fff640 1 -- 172.21.15.16:0/3610967959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d1487a0 msgr2=0x56407d14d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f70f7fff640 1 --2- 172.21.15.16:0/3610967959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d1487a0 0x56407d14d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f70f7fff640 1 -- 172.21.15.16:0/3610967959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 msgr2=0x56407d1037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f70f7fff640 1 --2- 172.21.15.16:0/3610967959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 0x56407d1037f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.868+0000 7f70f7fff640 1 -- 172.21.15.16:0/3610967959 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56407d106310 con 0x56407d12d030 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.869+0000 7f70f7fff640 1 --2- 172.21.15.16:0/3610967959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12d030 0x56407d103d30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f70e8001060 tx=0x7f70e8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f5797c6421e1aa server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.869+0000 7f70f4ff9640 1 -- 172.21.15.16:0/3610967959 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8014b80 con 0x56407d12d030 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.869+0000 7f70f4ff9640 1 -- 172.21.15.16:0/3610967959 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e8014ce0 con 0x56407d12d030 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.869+0000 7f70f4ff9640 1 -- 172.21.15.16:0/3610967959 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8023620 con 0x56407d12d030 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.869+0000 7f710762a740 1 -- 172.21.15.16:0/3610967959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12d030 msgr2=0x56407d103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.869+0000 7f710762a740 1 --2- 172.21.15.16:0/3610967959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12d030 0x56407d103d30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f70e8001060 tx=0x7f70e8013000 comp rx=0 tx=0).stop 2023-12-08T08:38:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.869+0000 7f710762a740 1 -- 172.21.15.16:0/3610967959 shutdown_connections 2023-12-08T08:38:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.869+0000 7f710762a740 1 --2- 172.21.15.16:0/3610967959 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d1487a0 0x56407d14d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.869+0000 7f710762a740 1 --2- 172.21.15.16:0/3610967959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12d030 0x56407d103d30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.869+0000 7f710762a740 1 --2- 172.21.15.16:0/3610967959 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 0x56407d1037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.869+0000 7f710762a740 1 -- 172.21.15.16:0/3610967959 >> 172.21.15.16:0/3610967959 conn(0x56407d100130 msgr2=0x56407d144390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.869+0000 7f710762a740 1 -- 172.21.15.16:0/3610967959 shutdown_connections 2023-12-08T08:38:26.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.869+0000 7f710762a740 1 -- 172.21.15.16:0/3610967959 wait complete. 2023-12-08T08:38:26.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.870+0000 7f710762a740 1 -- start start 2023-12-08T08:38:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.870+0000 7f710762a740 1 Processor -- start 2023-12-08T08:38:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f710762a740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 0x56407d14ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f710762a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12d030 0x56407d14b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f710762a740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d1487a0 0x56407d1b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f710762a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x56407d1b5bc0 con 0x56407d12d030 2023-12-08T08:38:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f710762a740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x56407d104020 con 0x56407d12cc50 2023-12-08T08:38:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f710762a740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x56407d146f30 con 0x56407d1487a0 2023-12-08T08:38:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f710762a740 10 client.0 _ll_register_callbacks cb 0x7f70f0033700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-08T08:38:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f70f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12d030 0x56407d14b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f70f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12d030 0x56407d14b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39288/0 (socket says 172.21.15.16:39288) 2023-12-08T08:38:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f70f7fff640 1 -- 172.21.15.16:0/13854241 learned_addr learned my addr 172.21.15.16:0/13854241 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f70fe59c640 1 --2- 172.21.15.16:0/13854241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 0x56407d14ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f70fed9d640 1 --2- 172.21.15.16:0/13854241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d1487a0 0x56407d1b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f70f7fff640 1 -- 172.21.15.16:0/13854241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d1487a0 msgr2=0x56407d1b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f70f7fff640 1 --2- 172.21.15.16:0/13854241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d1487a0 0x56407d1b4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f70f7fff640 1 -- 172.21.15.16:0/13854241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 msgr2=0x56407d14ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f70f7fff640 1 --2- 172.21.15.16:0/13854241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 0x56407d14ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f70f7fff640 1 -- 172.21.15.16:0/13854241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e800f040 con 0x56407d12d030 2023-12-08T08:38:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f70f7fff640 1 --2- 172.21.15.16:0/13854241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12d030 0x56407d14b1c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f70e801c000 tx=0x7f70e8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0771917f4474b23 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f70dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-12-08T08:38:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f70dd7fa640 1 -- 172.21.15.16:0/13854241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e8031070 con 0x56407d12d030 2023-12-08T08:38:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.871+0000 7f70dd7fa640 1 -- 172.21.15.16:0/13854241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f70e801c030 con 0x56407d12d030 2023-12-08T08:38:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f710762a740 1 -- 172.21.15.16:0/13854241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f70f007aa30 con 0x56407d12d030 2023-12-08T08:38:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f70dd7fa640 1 -- 172.21.15.16:0/13854241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70e8014e50 con 0x56407d12d030 2023-12-08T08:38:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f710762a740 10 client.4428 fetch_fsmap learned FSMap version 7 2023-12-08T08:38:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f710762a740 1 -- 172.21.15.16:0/13854241 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x56407d007d00 con 0x56407d12d030 2023-12-08T08:38:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f70dd7fa640 1 -- 172.21.15.16:0/13854241 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f007aa30 con 0x56407d12d030 2023-12-08T08:38:26.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f70dd7fa640 1 -- 172.21.15.16:0/13854241 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f70e8034020 con 0x56407d12d030 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f710762a740 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f710762a740 10 client.4428 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f710762a740 20 client.4428 populate_metadata read hostname 'smithi016' 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f710762a740 1 --2- 172.21.15.16:0/13854241 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x56407d1b5170 0x56407d163900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f710762a740 4 client.4428 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f710762a740 1 -- 172.21.15.16:0/13854241 --> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56407cd6c6c0 con 0x56407d1b5170 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f70fe59c640 1 --2- 172.21.15.16:0/13854241 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x56407d1b5170 0x56407d163900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f70dd7fa640 10 client.4428 ms_handle_connect on v2:172.21.15.112:6835/1082241707 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.872+0000 7f70fe59c640 1 --2- 172.21.15.16:0/13854241 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x56407d1b5170 0x56407d163900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58a0eb33f6e2013 server_cookie=59edff7ec88ed1b in_seq=0 out_seq=0 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f70dd7fa640 1 -- 172.21.15.16:0/13854241 <== mds.0 v2:172.21.15.112:6835/1082241707 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x56407cd6c6c0 con 0x56407d1b5170 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f70dd7fa640 10 client.4428 handle_command_reply: tid=0 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f710762a740 1 client.4428 shutdown 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f710762a740 1 -- 172.21.15.16:0/13854241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12d030 msgr2=0x56407d14b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f710762a740 1 --2- 172.21.15.16:0/13854241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12d030 0x56407d14b1c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f70e801c000 tx=0x7f70e8001a00 comp rx=0 tx=0).stop 2023-12-08T08:38:26.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f710762a740 1 -- 172.21.15.16:0/13854241 shutdown_connections 2023-12-08T08:38:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f710762a740 1 --2- 172.21.15.16:0/13854241 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x56407d1b5170 0x56407d163900 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f710762a740 1 --2- 172.21.15.16:0/13854241 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x56407d1487a0 0x56407d1b4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f710762a740 1 --2- 172.21.15.16:0/13854241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56407d12d030 0x56407d14b1c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f710762a740 1 --2- 172.21.15.16:0/13854241 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x56407d12cc50 0x56407d14ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f70dd7fa640 0 client.4428 ms_handle_reset on v2:172.21.15.112:6835/1082241707 2023-12-08T08:38:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f710762a740 1 -- 172.21.15.16:0/13854241 >> 172.21.15.16:0/13854241 conn(0x56407d100130 msgr2=0x56407d15d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f710762a740 1 -- 172.21.15.16:0/13854241 shutdown_connections 2023-12-08T08:38:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f710762a740 1 -- 172.21.15.16:0/13854241 wait complete. 2023-12-08T08:38:26.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f710762a740 20 client.4428 trim_cache size 0 max 16384 2023-12-08T08:38:26.880 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-12-08T08:38:26.880 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ead7ea87-fe4a-461b-916c-c6458a26d625", 2023-12-08T08:38:26.881 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-12-08T08:38:26.881 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4297, 2023-12-08T08:38:26.881 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-12-08T08:38:26.881 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-12-08T08:38:26.881 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-12-08T08:38:26.881 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 7.2405429769999996, 2023-12-08T08:38:26.881 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 7, 2023-12-08T08:38:26.881 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 24, 2023-12-08T08:38:26.881 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 24, 2023-12-08T08:38:26.881 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 14.254720581999999 2023-12-08T08:38:26.881 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-12-08T08:38:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f7104a97640 1 -- 172.21.15.16:0/1940986224 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f70e006ae40 msgr2=0x7f70e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f7104a97640 1 --2- 172.21.15.16:0/1940986224 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f70e006ae40 0x7f70e006d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f710006cd50 tx=0x7f70e8009040 comp rx=0 tx=0).stop 2023-12-08T08:38:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f7104a97640 1 -- 172.21.15.16:0/1940986224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70ec0039a0 msgr2=0x7f71000c92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.873+0000 7f7104a97640 1 --2- 172.21.15.16:0/1940986224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70ec0039a0 0x7f71000c92b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f70f002bdf0 tx=0x7f70f0001230 comp rx=0 tx=0).stop 2023-12-08T08:38:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.874+0000 7f7104a97640 1 -- 172.21.15.16:0/1940986224 shutdown_connections 2023-12-08T08:38:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.874+0000 7f7104a97640 1 --2- 172.21.15.16:0/1940986224 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f70e006ae40 0x7f70e006d300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.874+0000 7f7104a97640 1 --2- 172.21.15.16:0/1940986224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7100148c90 0x7f7100149110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.874+0000 7f7104a97640 1 --2- 172.21.15.16:0/1940986224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7100149910 0x7f7100156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.874+0000 7f7104a97640 1 --2- 172.21.15.16:0/1940986224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70ec0039a0 0x7f71000c92b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.874+0000 7f7104a97640 1 -- 172.21.15.16:0/1940986224 >> 172.21.15.16:0/1940986224 conn(0x7f710009c2f0 msgr2=0x7f71000ca6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.874+0000 7f7104a97640 1 -- 172.21.15.16:0/1940986224 shutdown_connections 2023-12-08T08:38:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.874+0000 7f7104a97640 1 -- 172.21.15.16:0/1940986224 wait complete. 2023-12-08T08:38:26.886 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-08T08:38:26.886 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-12-08T08:38:26.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-08T08:38:26.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.990+0000 7f51194da640 1 Processor -- start 2023-12-08T08:38:26.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.990+0000 7f51194da640 1 -- start start 2023-12-08T08:38:26.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f51194da640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51140d3ad0 0x7f51140afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f51194da640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51140d00c0 0x7f5114160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f51194da640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51140b20f0 con 0x7f511415e690 2023-12-08T08:38:26.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f51194da640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51140cefa0 con 0x7f51140d00c0 2023-12-08T08:38:26.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f51194da640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511415ea70 con 0x7f51140d3ad0 2023-12-08T08:38:26.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f5112ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f511415e690 0x7f51140cf9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52068/0 (socket says 172.21.15.16:52068) 2023-12-08T08:38:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f5112ffd640 1 -- 172.21.15.16:0/913127867 learned_addr learned my addr 172.21.15.16:0/913127867 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f51137fe640 1 --2- 172.21.15.16:0/913127867 >> v2:172.21.15.16:3300/0 conn(0x7f51140d00c0 0x7f5114160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f5111ffb640 1 -- 172.21.15.16:0/913127867 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537447018 0 0) 0x7f51140b20f0 con 0x7f511415e690 2023-12-08T08:38:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f51127fc640 1 --2- 172.21.15.16:0/913127867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51140d3ad0 0x7f51140afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f5111ffb640 1 -- 172.21.15.16:0/913127867 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5100003520 con 0x7f511415e690 2023-12-08T08:38:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f51137fe640 1 -- 172.21.15.16:0/913127867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51140d3ad0 msgr2=0x7f51140afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f51137fe640 1 --2- 172.21.15.16:0/913127867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51140d3ad0 0x7f51140afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f51137fe640 1 -- 172.21.15.16:0/913127867 >> v1:172.21.15.16:6789/0 conn(0x7f511415e690 legacy=0x7f51140cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f51137fe640 1 -- 172.21.15.16:0/913127867 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51140cf4d0 con 0x7f51140d00c0 2023-12-08T08:38:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.991+0000 7f51127fc640 1 --2- 172.21.15.16:0/913127867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51140d3ad0 0x7f51140afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.992+0000 7f51137fe640 1 --2- 172.21.15.16:0/913127867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51140d00c0 0x7f5114160fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5104009d90 tx=0x7f510402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baee793d90fc5136 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.992+0000 7f5111ffb640 1 -- 172.21.15.16:0/913127867 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510402cc50 con 0x7f51140d00c0 2023-12-08T08:38:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.992+0000 7f5111ffb640 1 -- 172.21.15.16:0/913127867 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510402cdb0 con 0x7f51140d00c0 2023-12-08T08:38:26.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.992+0000 7f5111ffb640 1 -- 172.21.15.16:0/913127867 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104034660 con 0x7f51140d00c0 2023-12-08T08:38:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.993+0000 7f51194da640 1 -- 172.21.15.16:0/913127867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51140d00c0 msgr2=0x7f5114160fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.993+0000 7f51194da640 1 --2- 172.21.15.16:0/913127867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51140d00c0 0x7f5114160fc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5104009d90 tx=0x7f510402bea0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.993+0000 7f51194da640 1 -- 172.21.15.16:0/913127867 shutdown_connections 2023-12-08T08:38:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.993+0000 7f51194da640 1 --2- 172.21.15.16:0/913127867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51140d00c0 0x7f5114160fc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.993+0000 7f51194da640 1 --2- 172.21.15.16:0/913127867 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51140d3ad0 0x7f51140afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.993+0000 7f51194da640 1 -- 172.21.15.16:0/913127867 >> 172.21.15.16:0/913127867 conn(0x7f51140d2fb0 msgr2=0x7f51140d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.993+0000 7f51194da640 1 -- 172.21.15.16:0/913127867 shutdown_connections 2023-12-08T08:38:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.993+0000 7f51194da640 1 -- 172.21.15.16:0/913127867 wait complete. 2023-12-08T08:38:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.994+0000 7f51194da640 1 Processor -- start 2023-12-08T08:38:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.994+0000 7f51194da640 1 -- start start 2023-12-08T08:38:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f51194da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51140d00c0 0x7f51140b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f51194da640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51140d3ad0 0x7f51141502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f51194da640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511415e690 0x7f51140cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f51194da640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51140c9e00 con 0x7f51140d00c0 2023-12-08T08:38:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f51194da640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51140c97b0 con 0x7f511415e690 2023-12-08T08:38:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f51194da640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51140c6d50 con 0x7f51140d3ad0 2023-12-08T08:38:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f5112ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51140d00c0 0x7f51140b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f5112ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51140d00c0 0x7f51140b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39296/0 (socket says 172.21.15.16:39296) 2023-12-08T08:38:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f5112ffd640 1 -- 172.21.15.16:0/3470691178 learned_addr learned my addr 172.21.15.16:0/3470691178 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f51137fe640 1 --2- 172.21.15.16:0/3470691178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511415e690 0x7f51140cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f51127fc640 1 --2- 172.21.15.16:0/3470691178 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51140d3ad0 0x7f51141502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f5112ffd640 1 -- 172.21.15.16:0/3470691178 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51140d3ad0 msgr2=0x7f51141502d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f5112ffd640 1 --2- 172.21.15.16:0/3470691178 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51140d3ad0 0x7f51141502d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f5112ffd640 1 -- 172.21.15.16:0/3470691178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511415e690 msgr2=0x7f51140cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.995+0000 7f5112ffd640 1 --2- 172.21.15.16:0/3470691178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511415e690 0x7f51140cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.996+0000 7f5112ffd640 1 -- 172.21.15.16:0/3470691178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5104009a70 con 0x7f51140d00c0 2023-12-08T08:38:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.996+0000 7f5112ffd640 1 --2- 172.21.15.16:0/3470691178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51140d00c0 0x7f51140b5e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f510c00c530 tx=0x7f510c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.996+0000 7f510b7fe640 1 -- 172.21.15.16:0/3470691178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c005ce0 con 0x7f51140d00c0 2023-12-08T08:38:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.996+0000 7f510b7fe640 1 -- 172.21.15.16:0/3470691178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510c005e40 con 0x7f51140d00c0 2023-12-08T08:38:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.996+0000 7f510b7fe640 1 -- 172.21.15.16:0/3470691178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c0072b0 con 0x7f51140d00c0 2023-12-08T08:38:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.996+0000 7f51194da640 1 -- 172.21.15.16:0/3470691178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51140c9a20 con 0x7f51140d00c0 2023-12-08T08:38:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.996+0000 7f51194da640 1 -- 172.21.15.16:0/3470691178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5114150810 con 0x7f51140d00c0 2023-12-08T08:38:27.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.997+0000 7f51194da640 1 -- 172.21.15.16:0/3470691178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51140cbae0 con 0x7f51140d00c0 2023-12-08T08:38:27.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.997+0000 7f510b7fe640 1 -- 172.21.15.16:0/3470691178 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f510c007410 con 0x7f51140d00c0 2023-12-08T08:38:27.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.998+0000 7f510b7fe640 1 --2- 172.21.15.16:0/3470691178 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f50f406ae40 0x7f50f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:27.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.998+0000 7f510b7fe640 1 -- 172.21.15.16:0/3470691178 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f510c088f90 con 0x7f51140d00c0 2023-12-08T08:38:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.998+0000 7f51127fc640 1 --2- 172.21.15.16:0/3470691178 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f50f406ae40 0x7f50f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:26.998+0000 7f51127fc640 1 --2- 172.21.15.16:0/3470691178 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f50f406ae40 0x7f50f406d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5114153dd0 tx=0x7f50fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:27.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.002+0000 7f510b7fe640 1 -- 172.21.15.16:0/3470691178 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f510c053d70 con 0x7f51140d00c0 2023-12-08T08:38:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.215+0000 7f51194da640 1 -- 172.21.15.16:0/3470691178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5114067430 con 0x7f51140d00c0 2023-12-08T08:38:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.215+0000 7f510b7fe640 1 -- 172.21.15.16:0/3470691178 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f510c017020 con 0x7f51140d00c0 2023-12-08T08:38:27.217 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-08T08:38:27.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.217+0000 7f51194da640 1 -- 172.21.15.16:0/3470691178 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f50f406ae40 msgr2=0x7f50f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:27.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.217+0000 7f51194da640 1 --2- 172.21.15.16:0/3470691178 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f50f406ae40 0x7f50f406d300 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5114153dd0 tx=0x7f50fc008040 comp rx=0 tx=0).stop 2023-12-08T08:38:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.217+0000 7f51194da640 1 -- 172.21.15.16:0/3470691178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51140d00c0 msgr2=0x7f51140b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.217+0000 7f51194da640 1 --2- 172.21.15.16:0/3470691178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51140d00c0 0x7f51140b5e30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f510c00c530 tx=0x7f510c00c9f0 comp rx=0 tx=0).stop 2023-12-08T08:38:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.217+0000 7f51194da640 1 -- 172.21.15.16:0/3470691178 shutdown_connections 2023-12-08T08:38:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.217+0000 7f51194da640 1 --2- 172.21.15.16:0/3470691178 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f50f406ae40 0x7f50f406d300 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.217+0000 7f51194da640 1 --2- 172.21.15.16:0/3470691178 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f511415e690 0x7f51140cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.217+0000 7f51194da640 1 --2- 172.21.15.16:0/3470691178 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f51140d3ad0 0x7f51141502d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.217+0000 7f51194da640 1 --2- 172.21.15.16:0/3470691178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51140d00c0 0x7f51140b5e30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.217+0000 7f51194da640 1 -- 172.21.15.16:0/3470691178 >> 172.21.15.16:0/3470691178 conn(0x7f51140d2fb0 msgr2=0x7f51140b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.217+0000 7f51194da640 1 -- 172.21.15.16:0/3470691178 shutdown_connections 2023-12-08T08:38:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:27.217+0000 7f51194da640 1 -- 172.21.15.16:0/3470691178 wait complete. 2023-12-08T08:38:27.230 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-08T08:38:27.230 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-08T08:38:27.230 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-12-08T08:38:27.230 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-12-08T08:38:27.230 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-12-08T08:38:27.230 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-08T08:38:27.230 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-08T08:38:27.230 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-08T08:38:27.230 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-12-08T08:38:27.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:27.230 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-12-08T08:38:27.246 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-08T08:38:27.246 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-08T08:38:27.246 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-12-08T08:38:27.246 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:27.246 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-12-08T08:38:27.246 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:27.246 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:27.246 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:27.246 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-12-08T08:38:27.246 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:27.247 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:27.247 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-12-08T08:38:27.247 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-08T08:38:27.247 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:27.247 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-12-08T08:38:27.247 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-08T08:38:27.247 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42605sec preferred_lft 42605sec 2023-12-08T08:38:27.247 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-12-08T08:38:27.247 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:27.247 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:27.247 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:27.247 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-12-08T08:38:27.248 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-08T08:38:27.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:27.248 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:38:27.248 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:38:27.248 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2023-12-08T08:38:27.248 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2023-12-08T08:38:27.248 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2023-12-08T08:38:27.249 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-08T08:38:27.249 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:38:27.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:27.444 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-08T08:38:27.475 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-12-08T08:38:27.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:27.477 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-12-08T08:38:27.551 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-12-08T08:38:27.552 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-08T08:38:27.552 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-08T08:38:27.552 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-08T08:38:27.552 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-08T08:38:27.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:27.552 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:38:27.552 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:38:27.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-08T08:38:27.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-08T08:38:27.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-08T08:38:27.553 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:38:27.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:27.847 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-12-08T08:38:27.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:27.863 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2023-12-08T08:38:27.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:27.918 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:38:27.918 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:38:27.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-08T08:38:27.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-12-08T08:38:27.918 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:38:28.007 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-12-08T08:38:28.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:28.008 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:38:28.008 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:38:28.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-12-08T08:38:28.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-08T08:38:28.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-12-08T08:38:28.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-12-08T08:38:28.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-12-08T08:38:28.008 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:38:28.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:28.120 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:38:28.120 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:38:28.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-12-08T08:38:28.120 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2023-12-08T08:38:28.120 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:38:28.169 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-12-08T08:38:28.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-08T08:38:28.169 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-12-08T08:38:28.183 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-12-08T08:38:28.184 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-08T08:38:28.184 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-12-08T08:38:28.238 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-12-08T08:38:28.310 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-12-08T08:38:28.365 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-12-08T08:38:28.366 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-12-08T08:38:28.366 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-12-08T08:38:28.366 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-12-08T08:38:28.366 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-12-08T08:38:28.366 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-12-08T08:38:28.366 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-12-08T08:38:28.366 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-12-08T08:38:28.366 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-12-08T08:38:28.366 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2023-12-08T08:38:28.366 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-12-08T08:38:28.366 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-12-08T08:38:28.366 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-12-08T08:38:28.367 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-12-08T08:38:28.367 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-12-08T08:38:28.367 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-12-08T08:38:28.367 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-12-08T08:38:28.367 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-12-08T08:38:28.367 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-12-08T08:38:28.367 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-12-08T08:38:28.367 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-08T08:38:28.367 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-08T08:38:28.367 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-12-08T08:38:28.367 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-12-08T08:38:28.367 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-12-08T08:38:28.367 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-12-08T08:38:28.367 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-12-08T08:38:28.368 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-12-08T08:38:28.368 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-12-08T08:38:28.368 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-12-08T08:38:28.368 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-08T08:38:28.368 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-08T08:38:28.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:28.368 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-12-08T08:38:28.421 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-08T08:38:28.421 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-12-08T08:38:28.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-12-08T08:38:28.494 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-12-08T08:38:28.550 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-12-08T08:38:28.550 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-12-08T08:38:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-08T08:38:28.552 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-08T08:38:28.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:28.553 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-12-08T08:38:28.569 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:2023-12-08T08:38:28.567+0000 7fb1fcd6e480 -1 init, newargv = 0x559c4b6951a0 newargc=15 2023-12-08T08:38:28.569 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[95582]: starting ceph client 2023-12-08T08:38:28.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[95582]: starting fuse 2023-12-08T08:38:28.584 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-12-08T08:38:28.584 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-08T08:38:28.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-08T08:38:28.585 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-08T08:38:28.586 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-08T08:38:28.586 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-12-08T08:38:28.586 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-12-08T08:38:28.586 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-12-08T08:38:28.586 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-08T08:38:28.586 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:38:28.586 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.admin")) 2023-12-08T08:38:28.586 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-12-08T08:38:28.710 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.admin.95582.asok 2023-12-08T08:38:28.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-08T08:38:28.720 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.95582.asok 2023-12-08T08:38:28.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:28.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.95582.asok status 2023-12-08T08:38:28.864 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-12-08T08:38:28.864 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-12-08T08:38:28.864 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "a2cd52c00149b5caff897ea7f79d0529b62620da", 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.0.0-46-ga2cd52c0 (a2cd52c00149b5caff897ea7f79d0529b62620da) reef (dev)", 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "admin", 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "95582", 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4440, 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4440 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:38:28.865 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-12-08T08:38:28.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-12-08T08:38:28.866 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-12-08T08:38:28.866 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 787522488 2023-12-08T08:38:28.866 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:38:28.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:38:28.866 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-12-08T08:38:28.866 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-12-08T08:38:28.866 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-12-08T08:38:28.866 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 787522488 2023-12-08T08:38:28.866 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:38:28.866 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4440 192.168.0.1:0/787522488", 2023-12-08T08:38:28.866 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/787522488", 2023-12-08T08:38:28.866 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-12-08T08:38:28.866 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 7, 2023-12-08T08:38:28.867 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 24, 2023-12-08T08:38:28.867 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-12-08T08:38:28.867 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-12-08T08:38:28.867 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-12-08T08:38:28.867 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-12-08T08:38:28.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:28.873 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-12-08T08:38:28.873 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-12-08T08:38:28.889 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-12-08T08:38:28.889 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-12-08T08:38:28.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:28.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-12-08T08:38:28.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:28.960 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-12-08T08:38:29.039 INFO:teuthology.orchestra.run.smithi016.stdout:# file: . 2023-12-08T08:38:29.039 INFO:teuthology.orchestra.run.smithi016.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-12-08T08:38:29.039 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:29.039 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:38:29.039 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T08:38:29.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:29.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.239+0000 7f9dcc690640 1 Processor -- start 2023-12-08T08:38:29.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.239+0000 7f9dcc690640 1 -- start start 2023-12-08T08:38:29.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.240+0000 7f9dcc690640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9dc41608a0 0x7f9dc40d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.240+0000 7f9dcc690640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9dc40b1d70 0x7f9dc40d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.240+0000 7f9dcc690640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc40d20b0 con 0x7f9dc40afce0 2023-12-08T08:38:29.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.240+0000 7f9dcc690640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc40d0620 con 0x7f9dc41608a0 2023-12-08T08:38:29.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.240+0000 7f9dcc690640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc40d1dd0 con 0x7f9dc40b1d70 2023-12-08T08:38:29.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.240+0000 7f9dca42c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9dc41608a0 0x7f9dc40d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.240+0000 7f9dca42c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9dc41608a0 0x7f9dc40d0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39310/0 (socket says 172.21.15.16:39310) 2023-12-08T08:38:29.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.240+0000 7f9dca42c640 1 -- 172.21.15.16:0/729937694 learned_addr learned my addr 172.21.15.16:0/729937694 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:29.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.241+0000 7f9dc9c2b640 1 --2- 172.21.15.16:0/729937694 >> v2:172.21.15.112:3300/0 conn(0x7f9dc40b1d70 0x7f9dc40d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.241+0000 7f9dca42c640 1 -- 172.21.15.16:0/729937694 >> v2:172.21.15.112:3300/0 conn(0x7f9dc40b1d70 msgr2=0x7f9dc40d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:29.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.241+0000 7f9dca42c640 1 --2- 172.21.15.16:0/729937694 >> v2:172.21.15.112:3300/0 conn(0x7f9dc40b1d70 0x7f9dc40d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.241+0000 7f9dca42c640 1 -- 172.21.15.16:0/729937694 >> v1:172.21.15.112:6789/0 conn(0x7f9dc40afce0 legacy=0x7f9dc40b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.241+0000 7f9dca42c640 1 -- 172.21.15.16:0/729937694 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc40cff60 con 0x7f9dc41608a0 2023-12-08T08:38:29.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.241+0000 7f9dca42c640 1 --2- 172.21.15.16:0/729937694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc41608a0 0x7f9dc40d0f30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40d4130 tx=0x7f9dc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be445a6ffa24e30 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:29.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.242+0000 7f9dc942a640 1 -- 172.21.15.16:0/729937694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0031870 con 0x7f9dc41608a0 2023-12-08T08:38:29.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.242+0000 7f9dc942a640 1 -- 172.21.15.16:0/729937694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc00319d0 con 0x7f9dc41608a0 2023-12-08T08:38:29.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.242+0000 7f9dc942a640 1 -- 172.21.15.16:0/729937694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0031d60 con 0x7f9dc41608a0 2023-12-08T08:38:29.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.242+0000 7f9dcc690640 1 -- 172.21.15.16:0/729937694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc41608a0 msgr2=0x7f9dc40d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.242+0000 7f9dcc690640 1 --2- 172.21.15.16:0/729937694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc41608a0 0x7f9dc40d0f30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40d4130 tx=0x7f9dc002b910 comp rx=0 tx=0).stop 2023-12-08T08:38:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.242+0000 7f9dcc690640 1 -- 172.21.15.16:0/729937694 shutdown_connections 2023-12-08T08:38:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.242+0000 7f9dcc690640 1 --2- 172.21.15.16:0/729937694 >> v2:172.21.15.112:3300/0 conn(0x7f9dc40b1d70 0x7f9dc40d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.242+0000 7f9dcc690640 1 --2- 172.21.15.16:0/729937694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc41608a0 0x7f9dc40d0f30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.242+0000 7f9dcc690640 1 -- 172.21.15.16:0/729937694 >> 172.21.15.16:0/729937694 conn(0x7f9dc40d3030 msgr2=0x7f9dc40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.242+0000 7f9dcc690640 1 -- 172.21.15.16:0/729937694 shutdown_connections 2023-12-08T08:38:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.242+0000 7f9dcc690640 1 -- 172.21.15.16:0/729937694 wait complete. 2023-12-08T08:38:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.244+0000 7f9dcc690640 1 Processor -- start 2023-12-08T08:38:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.244+0000 7f9dcc690640 1 -- start start 2023-12-08T08:38:29.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.244+0000 7f9dcc690640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc40afce0 0x7f9dc414c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.244+0000 7f9dcc690640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dc40b1d70 0x7f9dc415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.244+0000 7f9dcc690640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9dc41608a0 0x7f9dc40cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.244+0000 7f9dcc690640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc415f2e0 con 0x7f9dc40afce0 2023-12-08T08:38:29.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.244+0000 7f9dcc690640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc415e2e0 con 0x7f9dc40b1d70 2023-12-08T08:38:29.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.244+0000 7f9dcc690640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc415fd00 con 0x7f9dc41608a0 2023-12-08T08:38:29.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.244+0000 7f9dca42c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc40afce0 0x7f9dc414c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.244+0000 7f9dca42c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc40afce0 0x7f9dc414c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39318/0 (socket says 172.21.15.16:39318) 2023-12-08T08:38:29.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.244+0000 7f9dca42c640 1 -- 172.21.15.16:0/4178914990 learned_addr learned my addr 172.21.15.16:0/4178914990 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:29.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.245+0000 7f9dc9c2b640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dc40b1d70 0x7f9dc415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.245+0000 7f9dcac2d640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9dc41608a0 0x7f9dc40cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.245+0000 7f9dca42c640 1 -- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9dc41608a0 msgr2=0x7f9dc40cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.245+0000 7f9dca42c640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9dc41608a0 0x7f9dc40cc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.245+0000 7f9dca42c640 1 -- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dc40b1d70 msgr2=0x7f9dc415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.245+0000 7f9dca42c640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dc40b1d70 0x7f9dc415eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.245+0000 7f9dca42c640 1 -- 172.21.15.16:0/4178914990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc00094d0 con 0x7f9dc40afce0 2023-12-08T08:38:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.245+0000 7f9dc9c2b640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dc40b1d70 0x7f9dc415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.245+0000 7f9dca42c640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc40afce0 0x7f9dc414c5f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0000c00 tx=0x7f9dc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.245+0000 7f9dbaffd640 1 -- 172.21.15.16:0/4178914990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc003abe0 con 0x7f9dc40afce0 2023-12-08T08:38:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.245+0000 7f9dbaffd640 1 -- 172.21.15.16:0/4178914990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc003ad40 con 0x7f9dc40afce0 2023-12-08T08:38:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.245+0000 7f9dcc690640 1 -- 172.21.15.16:0/4178914990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc415ffe0 con 0x7f9dc40afce0 2023-12-08T08:38:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.245+0000 7f9dbaffd640 1 -- 172.21.15.16:0/4178914990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0044630 con 0x7f9dc40afce0 2023-12-08T08:38:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.245+0000 7f9dcc690640 1 -- 172.21.15.16:0/4178914990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc415e880 con 0x7f9dc40afce0 2023-12-08T08:38:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.246+0000 7f9dcc690640 1 -- 172.21.15.16:0/4178914990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc4067430 con 0x7f9dc40afce0 2023-12-08T08:38:29.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.246+0000 7f9dbaffd640 1 -- 172.21.15.16:0/4178914990 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9dc004d450 con 0x7f9dc40afce0 2023-12-08T08:38:29.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.247+0000 7f9dbaffd640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9da806ae70 0x7f9da806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.247+0000 7f9dbaffd640 1 -- 172.21.15.16:0/4178914990 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9dc002d040 con 0x7f9dc40afce0 2023-12-08T08:38:29.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.247+0000 7f9dc9c2b640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9da806ae70 0x7f9da806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.248+0000 7f9dc9c2b640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9da806ae70 0x7f9da806d330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40ca010 tx=0x7f9db4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:29.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.251+0000 7f9dbaffd640 1 -- 172.21.15.16:0/4178914990 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9dc00b2080 con 0x7f9dc40afce0 2023-12-08T08:38:29.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.427+0000 7f9dcc690640 1 -- 172.21.15.16:0/4178914990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dc40d3a10 con 0x7f9dc40afce0 2023-12-08T08:38:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.428+0000 7f9dbaffd640 1 -- 172.21.15.16:0/4178914990 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f9dc0076c10 con 0x7f9dc40afce0 2023-12-08T08:38:29.430 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:29.430 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:24.030625+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:29.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.430+0000 7f9dcc690640 1 -- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9da806ae70 msgr2=0x7f9da806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.430+0000 7f9dcc690640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9da806ae70 0x7f9da806d330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40ca010 tx=0x7f9db4008040 comp rx=0 tx=0).stop 2023-12-08T08:38:29.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.430+0000 7f9dcc690640 1 -- 172.21.15.16:0/4178914990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc40afce0 msgr2=0x7f9dc414c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.430+0000 7f9dcc690640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc40afce0 0x7f9dc414c5f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0000c00 tx=0x7f9dc00046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.430+0000 7f9dcc690640 1 -- 172.21.15.16:0/4178914990 shutdown_connections 2023-12-08T08:38:29.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.430+0000 7f9dcc690640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9da806ae70 0x7f9da806d330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.430+0000 7f9dcc690640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9dc41608a0 0x7f9dc40cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.430+0000 7f9dcc690640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9dc40b1d70 0x7f9dc415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.430+0000 7f9dcc690640 1 --2- 172.21.15.16:0/4178914990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9dc40afce0 0x7f9dc414c5f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.430+0000 7f9dcc690640 1 -- 172.21.15.16:0/4178914990 >> 172.21.15.16:0/4178914990 conn(0x7f9dc40d3030 msgr2=0x7f9dc40cecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:29.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.430+0000 7f9dcc690640 1 -- 172.21.15.16:0/4178914990 shutdown_connections 2023-12-08T08:38:29.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.430+0000 7f9dcc690640 1 -- 172.21.15.16:0/4178914990 wait complete. 2023-12-08T08:38:29.442 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:38:29.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:38:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.550+0000 7fcba8725640 1 Processor -- start 2023-12-08T08:38:29.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.550+0000 7fcba8725640 1 -- start start 2023-12-08T08:38:29.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba8725640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcba00cb370 0x7fcba015e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba8725640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcba006d7d0 0x7fcba00b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba8725640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba0157e00 con 0x7fcba0157a20 2023-12-08T08:38:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba8725640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba006dd20 con 0x7fcba006d7d0 2023-12-08T08:38:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba8725640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0070cd0 con 0x7fcba00cb370 2023-12-08T08:38:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba64c1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcba0157a20 0x7fcba015d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52082/0 (socket says 172.21.15.16:52082) 2023-12-08T08:38:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba64c1640 1 -- 172.21.15.16:0/4146160678 learned_addr learned my addr 172.21.15.16:0/4146160678 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba6cc2640 1 --2- 172.21.15.16:0/4146160678 >> v2:172.21.15.112:3300/0 conn(0x7fcba006d7d0 0x7fcba00b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba5cc0640 1 --2- 172.21.15.16:0/4146160678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcba00cb370 0x7fcba015e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba54bf640 1 -- 172.21.15.16:0/4146160678 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92957062 0 0) 0x7fcba0157e00 con 0x7fcba0157a20 2023-12-08T08:38:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba54bf640 1 -- 172.21.15.16:0/4146160678 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb8c005730 con 0x7fcba0157a20 2023-12-08T08:38:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba5cc0640 1 -- 172.21.15.16:0/4146160678 >> v2:172.21.15.112:3300/0 conn(0x7fcba006d7d0 msgr2=0x7fcba00b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba54bf640 1 -- 172.21.15.16:0/4146160678 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2570528012 0 0) 0x7fcb8c005730 con 0x7fcba0157a20 2023-12-08T08:38:29.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba5cc0640 1 --2- 172.21.15.16:0/4146160678 >> v2:172.21.15.112:3300/0 conn(0x7fcba006d7d0 0x7fcba00b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba5cc0640 1 -- 172.21.15.16:0/4146160678 >> v1:172.21.15.16:6789/0 conn(0x7fcba0157a20 legacy=0x7fcba015d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba5cc0640 1 -- 172.21.15.16:0/4146160678 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba0070fb0 con 0x7fcba00cb370 2023-12-08T08:38:29.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.551+0000 7fcba6cc2640 1 --2- 172.21.15.16:0/4146160678 >> v2:172.21.15.112:3300/0 conn(0x7fcba006d7d0 0x7fcba00b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.552+0000 7fcba5cc0640 1 --2- 172.21.15.16:0/4146160678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcba00cb370 0x7fcba015e0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcb900097a0 tx=0x7fcb9002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cbe7d0a290d9835 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.552+0000 7fcba54bf640 1 -- 172.21.15.16:0/4146160678 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c001670 con 0x7fcba00cb370 2023-12-08T08:38:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.552+0000 7fcba54bf640 1 -- 172.21.15.16:0/4146160678 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9002e050 con 0x7fcba00cb370 2023-12-08T08:38:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.552+0000 7fcba54bf640 1 -- 172.21.15.16:0/4146160678 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90030730 con 0x7fcba00cb370 2023-12-08T08:38:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.553+0000 7fcba8725640 1 -- 172.21.15.16:0/4146160678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcba00cb370 msgr2=0x7fcba015e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.553+0000 7fcba8725640 1 --2- 172.21.15.16:0/4146160678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcba00cb370 0x7fcba015e0e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fcb900097a0 tx=0x7fcb9002b790 comp rx=0 tx=0).stop 2023-12-08T08:38:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.553+0000 7fcba8725640 1 -- 172.21.15.16:0/4146160678 shutdown_connections 2023-12-08T08:38:29.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.553+0000 7fcba8725640 1 --2- 172.21.15.16:0/4146160678 >> v2:172.21.15.112:3300/0 conn(0x7fcba006d7d0 0x7fcba00b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.553+0000 7fcba8725640 1 --2- 172.21.15.16:0/4146160678 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcba00cb370 0x7fcba015e0e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.553+0000 7fcba8725640 1 -- 172.21.15.16:0/4146160678 >> 172.21.15.16:0/4146160678 conn(0x7fcba00b0850 msgr2=0x7fcba00b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.553+0000 7fcba8725640 1 -- 172.21.15.16:0/4146160678 shutdown_connections 2023-12-08T08:38:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.553+0000 7fcba8725640 1 -- 172.21.15.16:0/4146160678 wait complete. 2023-12-08T08:38:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.554+0000 7fcba8725640 1 Processor -- start 2023-12-08T08:38:29.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.554+0000 7fcba8725640 1 -- start start 2023-12-08T08:38:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba8725640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcba006d7d0 0x7fcba01478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba8725640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba00cb370 0x7fcba015b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba8725640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcba0157a20 0x7fcba00cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba8725640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0150e30 con 0x7fcba00cb370 2023-12-08T08:38:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba8725640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba01551b0 con 0x7fcba0157a20 2023-12-08T08:38:29.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba8725640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba0155490 con 0x7fcba006d7d0 2023-12-08T08:38:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba5cc0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba00cb370 0x7fcba015b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba5cc0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba00cb370 0x7fcba015b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39328/0 (socket says 172.21.15.16:39328) 2023-12-08T08:38:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba5cc0640 1 -- 172.21.15.16:0/3041941068 learned_addr learned my addr 172.21.15.16:0/3041941068 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba64c1640 1 --2- 172.21.15.16:0/3041941068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcba006d7d0 0x7fcba01478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba6cc2640 1 --2- 172.21.15.16:0/3041941068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcba0157a20 0x7fcba00cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba5cc0640 1 -- 172.21.15.16:0/3041941068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcba006d7d0 msgr2=0x7fcba01478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba5cc0640 1 --2- 172.21.15.16:0/3041941068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcba006d7d0 0x7fcba01478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba5cc0640 1 -- 172.21.15.16:0/3041941068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcba0157a20 msgr2=0x7fcba00cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba5cc0640 1 --2- 172.21.15.16:0/3041941068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcba0157a20 0x7fcba00cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.555+0000 7fcba5cc0640 1 -- 172.21.15.16:0/3041941068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb98009a70 con 0x7fcba00cb370 2023-12-08T08:38:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.556+0000 7fcba5cc0640 1 --2- 172.21.15.16:0/3041941068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba00cb370 0x7fcba015b8d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcb90000c00 tx=0x7fcb900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.556+0000 7fcb96ffd640 1 -- 172.21.15.16:0/3041941068 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9003b810 con 0x7fcba00cb370 2023-12-08T08:38:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.556+0000 7fcba8725640 1 -- 172.21.15.16:0/3041941068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb90009400 con 0x7fcba00cb370 2023-12-08T08:38:29.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.556+0000 7fcb96ffd640 1 -- 172.21.15.16:0/3041941068 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9003b970 con 0x7fcba00cb370 2023-12-08T08:38:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.556+0000 7fcb96ffd640 1 -- 172.21.15.16:0/3041941068 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9003bd20 con 0x7fcba00cb370 2023-12-08T08:38:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.556+0000 7fcba8725640 1 -- 172.21.15.16:0/3041941068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba0155a50 con 0x7fcba00cb370 2023-12-08T08:38:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.557+0000 7fcba8725640 1 -- 172.21.15.16:0/3041941068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba0067430 con 0x7fcba00cb370 2023-12-08T08:38:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.557+0000 7fcb96ffd640 1 -- 172.21.15.16:0/3041941068 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb90039040 con 0x7fcba00cb370 2023-12-08T08:38:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.558+0000 7fcb96ffd640 1 --2- 172.21.15.16:0/3041941068 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcb8406ae40 0x7fcb8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.558+0000 7fcb96ffd640 1 -- 172.21.15.16:0/3041941068 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fcb900ab0f0 con 0x7fcba00cb370 2023-12-08T08:38:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.558+0000 7fcba64c1640 1 --2- 172.21.15.16:0/3041941068 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcb8406ae40 0x7fcb8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.558+0000 7fcba64c1640 1 --2- 172.21.15.16:0/3041941068 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcb8406ae40 0x7fcb8406d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c001af0 tx=0x7fcb9c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:29.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.562+0000 7fcb96ffd640 1 -- 172.21.15.16:0/3041941068 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb90075da0 con 0x7fcba00cb370 2023-12-08T08:38:29.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.739+0000 7fcba8725640 1 -- 172.21.15.16:0/3041941068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcba0159fb0 con 0x7fcba00cb370 2023-12-08T08:38:29.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.739+0000 7fcb96ffd640 1 -- 172.21.15.16:0/3041941068 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb900793c0 con 0x7fcba00cb370 2023-12-08T08:38:29.741 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:38:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.741+0000 7fcba8725640 1 -- 172.21.15.16:0/3041941068 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcb8406ae40 msgr2=0x7fcb8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.741+0000 7fcba8725640 1 --2- 172.21.15.16:0/3041941068 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcb8406ae40 0x7fcb8406d300 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c001af0 tx=0x7fcb9c009040 comp rx=0 tx=0).stop 2023-12-08T08:38:29.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.741+0000 7fcba8725640 1 -- 172.21.15.16:0/3041941068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba00cb370 msgr2=0x7fcba015b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.741+0000 7fcba8725640 1 --2- 172.21.15.16:0/3041941068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba00cb370 0x7fcba015b8d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcb90000c00 tx=0x7fcb900046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.741+0000 7fcba8725640 1 -- 172.21.15.16:0/3041941068 shutdown_connections 2023-12-08T08:38:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.741+0000 7fcba8725640 1 --2- 172.21.15.16:0/3041941068 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcb8406ae40 0x7fcb8406d300 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.741+0000 7fcba8725640 1 --2- 172.21.15.16:0/3041941068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcba0157a20 0x7fcba00cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.741+0000 7fcba8725640 1 --2- 172.21.15.16:0/3041941068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba00cb370 0x7fcba015b8d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.741+0000 7fcba8725640 1 --2- 172.21.15.16:0/3041941068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcba006d7d0 0x7fcba01478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.741+0000 7fcba8725640 1 -- 172.21.15.16:0/3041941068 >> 172.21.15.16:0/3041941068 conn(0x7fcba00b0850 msgr2=0x7fcba00b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.741+0000 7fcba8725640 1 -- 172.21.15.16:0/3041941068 shutdown_connections 2023-12-08T08:38:29.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.741+0000 7fcba8725640 1 -- 172.21.15.16:0/3041941068 wait complete. 2023-12-08T08:38:29.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:38:29.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.861+0000 7f075e7b6640 1 Processor -- start 2023-12-08T08:38:29.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.861+0000 7f075e7b6640 1 -- start start 2023-12-08T08:38:29.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f075e7b6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07580b8950 0x7f07580ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f075e7b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f07580cebc0 0x7f07580cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f075e7b6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f07580cfab0 0x7f07580d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f075e7b6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07580cff40 con 0x7f07580b8950 2023-12-08T08:38:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f075e7b6640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07580b5f00 con 0x7f07580cfab0 2023-12-08T08:38:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f075e7b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07580b6170 con 0x7f07580cebc0 2023-12-08T08:38:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f0757fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07580b8950 0x7f07580ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f0757fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07580b8950 0x7f07580ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39338/0 (socket says 172.21.15.16:39338) 2023-12-08T08:38:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f0757fff640 1 -- 172.21.15.16:0/680126744 learned_addr learned my addr 172.21.15.16:0/680126744 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f075cd53640 1 --2- 172.21.15.16:0/680126744 >> v2:172.21.15.112:3300/0 conn(0x7f07580cfab0 0x7f07580d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f07577fe640 1 --2- 172.21.15.16:0/680126744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f07580cebc0 0x7f07580cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f0757fff640 1 -- 172.21.15.16:0/680126744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f07580cebc0 msgr2=0x7f07580cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f0757fff640 1 --2- 172.21.15.16:0/680126744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f07580cebc0 0x7f07580cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f0757fff640 1 -- 172.21.15.16:0/680126744 >> v2:172.21.15.112:3300/0 conn(0x7f07580cfab0 msgr2=0x7f07580d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:29.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f0757fff640 1 --2- 172.21.15.16:0/680126744 >> v2:172.21.15.112:3300/0 conn(0x7f07580cfab0 0x7f07580d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f0757fff640 1 -- 172.21.15.16:0/680126744 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07580b6450 con 0x7f07580b8950 2023-12-08T08:38:29.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.862+0000 7f075cd53640 1 --2- 172.21.15.16:0/680126744 >> v2:172.21.15.112:3300/0 conn(0x7f07580cfab0 0x7f07580d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.863+0000 7f0757fff640 1 --2- 172.21.15.16:0/680126744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07580b8950 0x7f07580ce680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f075808c470 tx=0x7f074802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623c561c4582e51a server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.863+0000 7f0756ffd640 1 -- 172.21.15.16:0/680126744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07480317e0 con 0x7f07580b8950 2023-12-08T08:38:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.863+0000 7f0756ffd640 1 -- 172.21.15.16:0/680126744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0748031940 con 0x7f07580b8950 2023-12-08T08:38:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.863+0000 7f0756ffd640 1 -- 172.21.15.16:0/680126744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0748031cd0 con 0x7f07580b8950 2023-12-08T08:38:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.864+0000 7f075e7b6640 1 -- 172.21.15.16:0/680126744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07580b8950 msgr2=0x7f07580ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.864+0000 7f075e7b6640 1 --2- 172.21.15.16:0/680126744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07580b8950 0x7f07580ce680 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f075808c470 tx=0x7f074802b880 comp rx=0 tx=0).stop 2023-12-08T08:38:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.864+0000 7f075e7b6640 1 -- 172.21.15.16:0/680126744 shutdown_connections 2023-12-08T08:38:29.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.864+0000 7f075e7b6640 1 --2- 172.21.15.16:0/680126744 >> v2:172.21.15.112:3300/0 conn(0x7f07580cfab0 0x7f07580d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.864+0000 7f075e7b6640 1 --2- 172.21.15.16:0/680126744 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f07580cebc0 0x7f07580cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.864+0000 7f075e7b6640 1 --2- 172.21.15.16:0/680126744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07580b8950 0x7f07580ce680 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.864+0000 7f075e7b6640 1 -- 172.21.15.16:0/680126744 >> 172.21.15.16:0/680126744 conn(0x7f07580d3010 msgr2=0x7f07580d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.864+0000 7f075e7b6640 1 -- 172.21.15.16:0/680126744 shutdown_connections 2023-12-08T08:38:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.864+0000 7f075e7b6640 1 -- 172.21.15.16:0/680126744 wait complete. 2023-12-08T08:38:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.865+0000 7f075e7b6640 1 Processor -- start 2023-12-08T08:38:29.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.865+0000 7f075e7b6640 1 -- start start 2023-12-08T08:38:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f075e7b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f07580b8950 0x7f075814d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f075e7b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07580cebc0 0x7f075815c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f075e7b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07580cfab0 0x7f075815a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f075e7b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075815bd60 con 0x7f07580cfab0 2023-12-08T08:38:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f075e7b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075815b0e0 con 0x7f07580cebc0 2023-12-08T08:38:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f075e7b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f075815b3c0 con 0x7f07580b8950 2023-12-08T08:38:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f075cd53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07580cfab0 0x7f075815a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f075cd53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07580cfab0 0x7f075815a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39348/0 (socket says 172.21.15.16:39348) 2023-12-08T08:38:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f075cd53640 1 -- 172.21.15.16:0/4294853370 learned_addr learned my addr 172.21.15.16:0/4294853370 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f07577fe640 1 --2- 172.21.15.16:0/4294853370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07580cebc0 0x7f075815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f0757fff640 1 --2- 172.21.15.16:0/4294853370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f07580b8950 0x7f075814d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f075cd53640 1 -- 172.21.15.16:0/4294853370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f07580b8950 msgr2=0x7f075814d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f075cd53640 1 --2- 172.21.15.16:0/4294853370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f07580b8950 0x7f075814d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f075cd53640 1 -- 172.21.15.16:0/4294853370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07580cebc0 msgr2=0x7f075815c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f075cd53640 1 --2- 172.21.15.16:0/4294853370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07580cebc0 0x7f075815c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.866+0000 7f075cd53640 1 -- 172.21.15.16:0/4294853370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07480094d0 con 0x7f07580cfab0 2023-12-08T08:38:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.867+0000 7f075cd53640 1 --2- 172.21.15.16:0/4294853370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07580cfab0 0x7f075815a660 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f075000cd50 tx=0x7f075000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.867+0000 7f0754ff9640 1 -- 172.21.15.16:0/4294853370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075000f680 con 0x7f07580cfab0 2023-12-08T08:38:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.867+0000 7f0754ff9640 1 -- 172.21.15.16:0/4294853370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075000f800 con 0x7f07580cfab0 2023-12-08T08:38:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.867+0000 7f0754ff9640 1 -- 172.21.15.16:0/4294853370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075000fbb0 con 0x7f07580cfab0 2023-12-08T08:38:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.867+0000 7f075e7b6640 1 -- 172.21.15.16:0/4294853370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075815b6a0 con 0x7f07580cfab0 2023-12-08T08:38:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.867+0000 7f075e7b6640 1 -- 172.21.15.16:0/4294853370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075814d900 con 0x7f07580cfab0 2023-12-08T08:38:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.868+0000 7f075e7b6640 1 -- 172.21.15.16:0/4294853370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f075808b150 con 0x7f07580cfab0 2023-12-08T08:38:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.868+0000 7f0754ff9640 1 -- 172.21.15.16:0/4294853370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f075000b040 con 0x7f07580cfab0 2023-12-08T08:38:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.869+0000 7f0754ff9640 1 --2- 172.21.15.16:0/4294853370 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f074406ae70 0x7f074406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.869+0000 7f0754ff9640 1 -- 172.21.15.16:0/4294853370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f075008a260 con 0x7f07580cfab0 2023-12-08T08:38:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.869+0000 7f0757fff640 1 --2- 172.21.15.16:0/4294853370 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f074406ae70 0x7f074406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.869+0000 7f0757fff640 1 --2- 172.21.15.16:0/4294853370 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f074406ae70 0x7f074406d330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0748036000 tx=0x7f074803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:29.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:29.873+0000 7f0754ff9640 1 -- 172.21.15.16:0/4294853370 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0750055030 con 0x7f07580cfab0 2023-12-08T08:38:30.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.047+0000 7f075e7b6640 1 -- 172.21.15.16:0/4294853370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f075815bea0 con 0x7f07580cfab0 2023-12-08T08:38:30.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.047+0000 7f0754ff9640 1 -- 172.21.15.16:0/4294853370 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f075815bea0 con 0x7f07580cfab0 2023-12-08T08:38:30.050 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:38:30.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.049+0000 7f075e7b6640 1 -- 172.21.15.16:0/4294853370 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f074406ae70 msgr2=0x7f074406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.049+0000 7f075e7b6640 1 --2- 172.21.15.16:0/4294853370 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f074406ae70 0x7f074406d330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0748036000 tx=0x7f074803d040 comp rx=0 tx=0).stop 2023-12-08T08:38:30.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.049+0000 7f075e7b6640 1 -- 172.21.15.16:0/4294853370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07580cfab0 msgr2=0x7f075815a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.049+0000 7f075e7b6640 1 --2- 172.21.15.16:0/4294853370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07580cfab0 0x7f075815a660 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f075000cd50 tx=0x7f075000a650 comp rx=0 tx=0).stop 2023-12-08T08:38:30.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.049+0000 7f075e7b6640 1 -- 172.21.15.16:0/4294853370 shutdown_connections 2023-12-08T08:38:30.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.049+0000 7f075e7b6640 1 --2- 172.21.15.16:0/4294853370 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f074406ae70 0x7f074406d330 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.049+0000 7f075e7b6640 1 --2- 172.21.15.16:0/4294853370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07580cfab0 0x7f075815a660 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.049+0000 7f075e7b6640 1 --2- 172.21.15.16:0/4294853370 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f07580cebc0 0x7f075815c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.049+0000 7f075e7b6640 1 --2- 172.21.15.16:0/4294853370 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f07580b8950 0x7f075814d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.049+0000 7f075e7b6640 1 -- 172.21.15.16:0/4294853370 >> 172.21.15.16:0/4294853370 conn(0x7f07580d3010 msgr2=0x7f075806b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:30.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.049+0000 7f075e7b6640 1 -- 172.21.15.16:0/4294853370 shutdown_connections 2023-12-08T08:38:30.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.049+0000 7f075e7b6640 1 -- 172.21.15.16:0/4294853370 wait complete. 2023-12-08T08:38:30.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:38:30.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.169+0000 7f6700b68640 1 Processor -- start 2023-12-08T08:38:30.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.169+0000 7f6700b68640 1 -- start start 2023-12-08T08:38:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.170+0000 7f6700b68640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66fc15d590 0x7f66fc0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.170+0000 7f6700b68640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66fc06d650 con 0x7f66fc0ae350 2023-12-08T08:38:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.170+0000 7f6700b68640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66fc0adbe0 con 0x7f66fc0afeb0 2023-12-08T08:38:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.170+0000 7f6700b68640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66fc15d970 con 0x7f66fc15d590 2023-12-08T08:38:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.170+0000 7f66fa59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66fc15d590 0x7f66fc0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.170+0000 7f66fad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f66fc0ae350 0x7f66fc0ae7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52098/0 (socket says 172.21.15.16:52098) 2023-12-08T08:38:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.170+0000 7f66fad9d640 1 -- 172.21.15.16:0/1369648126 learned_addr learned my addr 172.21.15.16:0/1369648126 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.170+0000 7f66fa59c640 1 -- 172.21.15.16:0/1369648126 >> v1:172.21.15.112:6789/0 conn(0x7f66fc0afeb0 legacy=0x7f66fc0cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.170+0000 7f66fa59c640 1 -- 172.21.15.16:0/1369648126 >> v1:172.21.15.16:6789/0 conn(0x7f66fc0ae350 legacy=0x7f66fc0ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.170+0000 7f66f959a640 1 -- 172.21.15.16:0/1369648126 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900485884 0 0) 0x7f66ec001490 con 0x7f66fc0ae350 2023-12-08T08:38:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.170+0000 7f66fa59c640 1 -- 172.21.15.16:0/1369648126 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66fc06c810 con 0x7f66fc15d590 2023-12-08T08:38:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.171+0000 7f66fa59c640 1 --2- 172.21.15.16:0/1369648126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc15d590 0x7f66fc0ced10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f66f40097a0 tx=0x7f66f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866e1447303486f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.171+0000 7f66f959a640 1 -- 172.21.15.16:0/1369648126 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4031830 con 0x7f66fc15d590 2023-12-08T08:38:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.171+0000 7f66f959a640 1 -- 172.21.15.16:0/1369648126 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f4031990 con 0x7f66fc15d590 2023-12-08T08:38:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.171+0000 7f66f959a640 1 -- 172.21.15.16:0/1369648126 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f4031d20 con 0x7f66fc15d590 2023-12-08T08:38:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.172+0000 7f6700b68640 1 -- 172.21.15.16:0/1369648126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc15d590 msgr2=0x7f66fc0ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.172+0000 7f6700b68640 1 --2- 172.21.15.16:0/1369648126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc15d590 0x7f66fc0ced10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f66f40097a0 tx=0x7f66f402b8d0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.172+0000 7f6700b68640 1 -- 172.21.15.16:0/1369648126 shutdown_connections 2023-12-08T08:38:30.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.172+0000 7f6700b68640 1 --2- 172.21.15.16:0/1369648126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc15d590 0x7f66fc0ced10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.172+0000 7f6700b68640 1 -- 172.21.15.16:0/1369648126 >> 172.21.15.16:0/1369648126 conn(0x7f66fc0d0e10 msgr2=0x7f66fc0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:30.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.172+0000 7f6700b68640 1 -- 172.21.15.16:0/1369648126 shutdown_connections 2023-12-08T08:38:30.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.172+0000 7f6700b68640 1 -- 172.21.15.16:0/1369648126 wait complete. 2023-12-08T08:38:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.173+0000 7f6700b68640 1 Processor -- start 2023-12-08T08:38:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.173+0000 7f6700b68640 1 -- start start 2023-12-08T08:38:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.174+0000 7f6700b68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66fc0ae350 0x7f66fc0cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.174+0000 7f6700b68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc0afeb0 0x7f66fc1580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.174+0000 7f6700b68640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66fc15d590 0x7f66fc1605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.174+0000 7f6700b68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0cc6a0 con 0x7f66fc0afeb0 2023-12-08T08:38:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.174+0000 7f6700b68640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc160b00 con 0x7f66fc0ae350 2023-12-08T08:38:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.174+0000 7f6700b68640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc0cba90 con 0x7f66fc15d590 2023-12-08T08:38:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.174+0000 7f66f9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc0afeb0 0x7f66fc1580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.174+0000 7f66f9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc0afeb0 0x7f66fc1580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39360/0 (socket says 172.21.15.16:39360) 2023-12-08T08:38:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.174+0000 7f66f9d9b640 1 -- 172.21.15.16:0/1684589567 learned_addr learned my addr 172.21.15.16:0/1684589567 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.174+0000 7f66fa59c640 1 --2- 172.21.15.16:0/1684589567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66fc0ae350 0x7f66fc0cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.174+0000 7f66fad9d640 1 --2- 172.21.15.16:0/1684589567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66fc15d590 0x7f66fc1605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.175+0000 7f66f9d9b640 1 -- 172.21.15.16:0/1684589567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66fc15d590 msgr2=0x7f66fc1605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.175+0000 7f66f9d9b640 1 --2- 172.21.15.16:0/1684589567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66fc15d590 0x7f66fc1605c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.175+0000 7f66f9d9b640 1 -- 172.21.15.16:0/1684589567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66fc0ae350 msgr2=0x7f66fc0cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.175+0000 7f66f9d9b640 1 --2- 172.21.15.16:0/1684589567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66fc0ae350 0x7f66fc0cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.175+0000 7f66f9d9b640 1 -- 172.21.15.16:0/1684589567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4009400 con 0x7f66fc0afeb0 2023-12-08T08:38:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.175+0000 7f66f9d9b640 1 --2- 172.21.15.16:0/1684589567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc0afeb0 0x7f66fc1580e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f66e400c450 tx=0x7f66e400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.175+0000 7f66f2ffd640 1 -- 172.21.15.16:0/1684589567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e40052d0 con 0x7f66fc0afeb0 2023-12-08T08:38:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.175+0000 7f66f2ffd640 1 -- 172.21.15.16:0/1684589567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e4005430 con 0x7f66fc0afeb0 2023-12-08T08:38:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.175+0000 7f66f2ffd640 1 -- 172.21.15.16:0/1684589567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4012870 con 0x7f66fc0afeb0 2023-12-08T08:38:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.175+0000 7f6700b68640 1 -- 172.21.15.16:0/1684589567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc0cbd70 con 0x7f66fc0afeb0 2023-12-08T08:38:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.175+0000 7f6700b68640 1 -- 172.21.15.16:0/1684589567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc0cc330 con 0x7f66fc0afeb0 2023-12-08T08:38:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.176+0000 7f6700b68640 1 -- 172.21.15.16:0/1684589567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66fc0cc810 con 0x7f66fc0afeb0 2023-12-08T08:38:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.177+0000 7f66f2ffd640 1 -- 172.21.15.16:0/1684589567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66e40129d0 con 0x7f66fc0afeb0 2023-12-08T08:38:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.177+0000 7f66f2ffd640 1 --2- 172.21.15.16:0/1684589567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f66dc06ae40 0x7f66dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.177+0000 7f66f2ffd640 1 -- 172.21.15.16:0/1684589567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f66e40892e0 con 0x7f66fc0afeb0 2023-12-08T08:38:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.177+0000 7f66fa59c640 1 --2- 172.21.15.16:0/1684589567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f66dc06ae40 0x7f66dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.178+0000 7f66fa59c640 1 --2- 172.21.15.16:0/1684589567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f66dc06ae40 0x7f66dc06d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f66f4036000 tx=0x7f66f403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:30.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.181+0000 7f66f2ffd640 1 -- 172.21.15.16:0/1684589567 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f66e4054150 con 0x7f66fc0afeb0 2023-12-08T08:38:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.354+0000 7f6700b68640 1 -- 172.21.15.16:0/1684589567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f66fc0b1490 con 0x7f66fc0afeb0 2023-12-08T08:38:30.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.355+0000 7f66f2ffd640 1 -- 172.21.15.16:0/1684589567 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f66e4057770 con 0x7f66fc0afeb0 2023-12-08T08:38:30.357 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:38:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.357+0000 7f6700b68640 1 -- 172.21.15.16:0/1684589567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f66dc06ae40 msgr2=0x7f66dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.357+0000 7f6700b68640 1 --2- 172.21.15.16:0/1684589567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f66dc06ae40 0x7f66dc06d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f66f4036000 tx=0x7f66f403c040 comp rx=0 tx=0).stop 2023-12-08T08:38:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.357+0000 7f6700b68640 1 -- 172.21.15.16:0/1684589567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc0afeb0 msgr2=0x7f66fc1580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.357+0000 7f6700b68640 1 --2- 172.21.15.16:0/1684589567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc0afeb0 0x7f66fc1580e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f66e400c450 tx=0x7f66e400c910 comp rx=0 tx=0).stop 2023-12-08T08:38:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.357+0000 7f6700b68640 1 -- 172.21.15.16:0/1684589567 shutdown_connections 2023-12-08T08:38:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.357+0000 7f6700b68640 1 --2- 172.21.15.16:0/1684589567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f66dc06ae40 0x7f66dc06d300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.357+0000 7f6700b68640 1 --2- 172.21.15.16:0/1684589567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66fc15d590 0x7f66fc1605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.357+0000 7f6700b68640 1 --2- 172.21.15.16:0/1684589567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66fc0afeb0 0x7f66fc1580e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.357+0000 7f6700b68640 1 --2- 172.21.15.16:0/1684589567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66fc0ae350 0x7f66fc0cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.357+0000 7f6700b68640 1 -- 172.21.15.16:0/1684589567 >> 172.21.15.16:0/1684589567 conn(0x7f66fc0d0e10 msgr2=0x7f66fc0b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.357+0000 7f6700b68640 1 -- 172.21.15.16:0/1684589567 shutdown_connections 2023-12-08T08:38:30.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.357+0000 7f6700b68640 1 -- 172.21.15.16:0/1684589567 wait complete. 2023-12-08T08:38:30.369 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:38:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.476+0000 7ff66bfd7640 1 Processor -- start 2023-12-08T08:38:30.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.476+0000 7ff66bfd7640 1 -- start start 2023-12-08T08:38:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff66bfd7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff66415d4f0 0x7ff6640b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff66bfd7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6640b2280 con 0x7ff6640b0920 2023-12-08T08:38:30.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff66bfd7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6640b1430 con 0x7ff6640cef90 2023-12-08T08:38:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff66bfd7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66406bb40 con 0x7ff66415d4f0 2023-12-08T08:38:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff669572640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff6640b0920 0x7ff6640b0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52114/0 (socket says 172.21.15.16:52114) 2023-12-08T08:38:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff669572640 1 -- 172.21.15.16:0/227866734 learned_addr learned my addr 172.21.15.16:0/227866734 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff669d73640 1 --2- 172.21.15.16:0/227866734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff66415d4f0 0x7ff6640b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407820571 0 0) 0x7ff6640b2280 con 0x7ff6640b0920 2023-12-08T08:38:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff650003520 con 0x7ff6640b0920 2023-12-08T08:38:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159518250 0 0) 0x7ff6640b1430 con 0x7ff6640cef90 2023-12-08T08:38:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6640b2280 con 0x7ff6640cef90 2023-12-08T08:38:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2526574779 0 0) 0x7ff650003520 con 0x7ff6640b0920 2023-12-08T08:38:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff66415d4f0 msgr2=0x7ff6640b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 --2- 172.21.15.16:0/227866734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff66415d4f0 0x7ff6640b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 >> v1:172.21.15.112:6789/0 conn(0x7ff6640cef90 legacy=0x7ff6640cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6640b2710 con 0x7ff6640b0920 2023-12-08T08:38:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7ff654001670 con 0x7ff6640b0920 2023-12-08T08:38:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff669d73640 1 --2- 172.21.15.16:0/227866734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff66415d4f0 0x7ff6640b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 >> v1:172.21.15.16:6789/0 conn(0x7ff6640b0920 legacy=0x7ff6640b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 --2- 172.21.15.16:0/227866734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff66415d4f0 0x7ff650003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 --2- 172.21.15.16:0/227866734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650004070 0x7ff6500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 --2- 172.21.15.16:0/227866734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650004d00 0x7ff650005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66406bb40 con 0x7ff650004070 2023-12-08T08:38:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff65000ba30 con 0x7ff650004d00 2023-12-08T08:38:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff65000bbf0 con 0x7ff66415d4f0 2023-12-08T08:38:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.477+0000 7ff669572640 1 --2- 172.21.15.16:0/227866734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650004070 0x7ff6500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.478+0000 7ff669d73640 1 --2- 172.21.15.16:0/227866734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff66415d4f0 0x7ff650003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.478+0000 7ff66a574640 1 --2- 172.21.15.16:0/227866734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650004d00 0x7ff650005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.478+0000 7ff669572640 1 -- 172.21.15.16:0/227866734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff66415d4f0 msgr2=0x7ff650003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.478+0000 7ff669572640 1 --2- 172.21.15.16:0/227866734 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff66415d4f0 0x7ff650003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.478+0000 7ff669572640 1 -- 172.21.15.16:0/227866734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650004d00 msgr2=0x7ff650005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.478+0000 7ff669572640 1 --2- 172.21.15.16:0/227866734 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650004d00 0x7ff650005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.478+0000 7ff669572640 1 -- 172.21.15.16:0/227866734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff660009400 con 0x7ff650004070 2023-12-08T08:38:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.479+0000 7ff669572640 1 --2- 172.21.15.16:0/227866734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650004070 0x7ff6500047c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff654007fd0 tx=0x7ff6500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16121e81a6d96e39 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.479+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff654005ad0 con 0x7ff650004070 2023-12-08T08:38:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.479+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff654005c50 con 0x7ff650004070 2023-12-08T08:38:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.479+0000 7ff668d71640 1 -- 172.21.15.16:0/227866734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65402c830 con 0x7ff650004070 2023-12-08T08:38:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.479+0000 7ff66bfd7640 1 -- 172.21.15.16:0/227866734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650004070 msgr2=0x7ff6500047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.479+0000 7ff66bfd7640 1 --2- 172.21.15.16:0/227866734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff650004070 0x7ff6500047c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff654007fd0 tx=0x7ff6500057f0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.479+0000 7ff66a574640 1 -- 172.21.15.16:0/227866734 reap_dead start 2023-12-08T08:38:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.479+0000 7ff66bfd7640 1 -- 172.21.15.16:0/227866734 shutdown_connections 2023-12-08T08:38:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.479+0000 7ff66bfd7640 1 -- 172.21.15.16:0/227866734 >> 172.21.15.16:0/227866734 conn(0x7ff6640d0e00 msgr2=0x7ff6640d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.479+0000 7ff66bfd7640 1 -- 172.21.15.16:0/227866734 shutdown_connections 2023-12-08T08:38:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.479+0000 7ff66bfd7640 1 -- 172.21.15.16:0/227866734 wait complete. 2023-12-08T08:38:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.480+0000 7ff66bfd7640 1 Processor -- start 2023-12-08T08:38:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.481+0000 7ff66bfd7640 1 -- start start 2023-12-08T08:38:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.481+0000 7ff66bfd7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650004070 0x7ff66414d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.481+0000 7ff66bfd7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6640c9930 0x7ff664147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.481+0000 7ff66bfd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6641f2230 0x7ff66406e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.481+0000 7ff66bfd7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6640ca0a0 con 0x7ff6641f2230 2023-12-08T08:38:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.481+0000 7ff66bfd7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff664147b00 con 0x7ff650004070 2023-12-08T08:38:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.481+0000 7ff66bfd7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6640ccab0 con 0x7ff6640c9930 2023-12-08T08:38:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.481+0000 7ff66a574640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6641f2230 0x7ff66406e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.481+0000 7ff66a574640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6641f2230 0x7ff66406e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39380/0 (socket says 172.21.15.16:39380) 2023-12-08T08:38:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.481+0000 7ff66a574640 1 -- 172.21.15.16:0/3018388583 learned_addr learned my addr 172.21.15.16:0/3018388583 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.481+0000 7ff669d73640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650004070 0x7ff66414d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.482+0000 7ff669572640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6640c9930 0x7ff664147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.482+0000 7ff66a574640 1 -- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6640c9930 msgr2=0x7ff664147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.482+0000 7ff66a574640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6640c9930 0x7ff664147d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.482+0000 7ff66a574640 1 -- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650004070 msgr2=0x7ff66414d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.482+0000 7ff66a574640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650004070 0x7ff66414d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.482+0000 7ff66a574640 1 -- 172.21.15.16:0/3018388583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6640ccd50 con 0x7ff6641f2230 2023-12-08T08:38:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.482+0000 7ff669572640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6640c9930 0x7ff664147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.482+0000 7ff66a574640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6641f2230 0x7ff66406e930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff6640727a0 tx=0x7ff65c00fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.482+0000 7ff65a7fc640 1 -- 172.21.15.16:0/3018388583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c006e20 con 0x7ff6641f2230 2023-12-08T08:38:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.482+0000 7ff65a7fc640 1 -- 172.21.15.16:0/3018388583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65c003970 con 0x7ff6641f2230 2023-12-08T08:38:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.482+0000 7ff65a7fc640 1 -- 172.21.15.16:0/3018388583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c003d20 con 0x7ff6641f2230 2023-12-08T08:38:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.482+0000 7ff66bfd7640 1 -- 172.21.15.16:0/3018388583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6640cd030 con 0x7ff6641f2230 2023-12-08T08:38:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.482+0000 7ff66bfd7640 1 -- 172.21.15.16:0/3018388583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6640cd5f0 con 0x7ff6641f2230 2023-12-08T08:38:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.483+0000 7ff66bfd7640 1 -- 172.21.15.16:0/3018388583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff664067430 con 0x7ff6641f2230 2023-12-08T08:38:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.483+0000 7ff65a7fc640 1 -- 172.21.15.16:0/3018388583 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff65c005030 con 0x7ff6641f2230 2023-12-08T08:38:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.484+0000 7ff65a7fc640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff64806ae70 0x7ff64806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.484+0000 7ff65a7fc640 1 -- 172.21.15.16:0/3018388583 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff65c088910 con 0x7ff6641f2230 2023-12-08T08:38:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.484+0000 7ff669d73640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff64806ae70 0x7ff64806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.485+0000 7ff669d73640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff64806ae70 0x7ff64806d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff660009c60 tx=0x7ff660009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:30.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.488+0000 7ff65a7fc640 1 -- 172.21.15.16:0/3018388583 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff65c053780 con 0x7ff6641f2230 2023-12-08T08:38:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.664+0000 7ff66bfd7640 1 -- 172.21.15.16:0/3018388583 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff66406bdd0 con 0x7ff6641f2230 2023-12-08T08:38:30.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.664+0000 7ff65a7fc640 1 -- 172.21.15.16:0/3018388583 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff65c0190a0 con 0x7ff6641f2230 2023-12-08T08:38:30.666 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:38:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.666+0000 7ff66bfd7640 1 -- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff64806ae70 msgr2=0x7ff64806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.666+0000 7ff66bfd7640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff64806ae70 0x7ff64806d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff660009c60 tx=0x7ff660009190 comp rx=0 tx=0).stop 2023-12-08T08:38:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.666+0000 7ff66bfd7640 1 -- 172.21.15.16:0/3018388583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6641f2230 msgr2=0x7ff66406e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.666+0000 7ff66bfd7640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6641f2230 0x7ff66406e930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff6640727a0 tx=0x7ff65c00fd60 comp rx=0 tx=0).stop 2023-12-08T08:38:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.666+0000 7ff66bfd7640 1 -- 172.21.15.16:0/3018388583 shutdown_connections 2023-12-08T08:38:30.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.666+0000 7ff66bfd7640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff64806ae70 0x7ff64806d330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.666+0000 7ff66bfd7640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6641f2230 0x7ff66406e930 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.666+0000 7ff66bfd7640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff6640c9930 0x7ff664147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.666+0000 7ff66bfd7640 1 --2- 172.21.15.16:0/3018388583 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff650004070 0x7ff66414d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.666+0000 7ff66bfd7640 1 -- 172.21.15.16:0/3018388583 >> 172.21.15.16:0/3018388583 conn(0x7ff6640d0e00 msgr2=0x7ff66415f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.666+0000 7ff66bfd7640 1 -- 172.21.15.16:0/3018388583 shutdown_connections 2023-12-08T08:38:30.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.666+0000 7ff66bfd7640 1 -- 172.21.15.16:0/3018388583 wait complete. 2023-12-08T08:38:30.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-08T08:38:30.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.784+0000 7f5090d41640 1 Processor -- start 2023-12-08T08:38:30.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.784+0000 7f5090d41640 1 -- start start 2023-12-08T08:38:30.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.785+0000 7f5090d41640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f508c0d1d60 0x7f508c0d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.785+0000 7f5090d41640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f508c15f890 0x7f508c15d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.785+0000 7f5090d41640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c15ee90 con 0x7f508c15d580 2023-12-08T08:38:30.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.785+0000 7f5090d41640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c0d0fe0 con 0x7f508c15f890 2023-12-08T08:38:30.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.785+0000 7f5090d41640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c0d1280 con 0x7f508c0d1d60 2023-12-08T08:38:30.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.785+0000 7f508ad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f508c15f890 0x7f508c15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.785+0000 7f508a59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f508c15d580 0x7f508c15f180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52124/0 (socket says 172.21.15.16:52124) 2023-12-08T08:38:30.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.785+0000 7f508a59c640 1 -- 172.21.15.16:0/2973769313 learned_addr learned my addr 172.21.15.16:0/2973769313 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:30.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.785+0000 7f5089d9b640 1 --2- 172.21.15.16:0/2973769313 >> v2:172.21.15.112:3300/0 conn(0x7f508c0d1d60 0x7f508c0d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.785+0000 7f508ad9d640 1 -- 172.21.15.16:0/2973769313 >> v2:172.21.15.112:3300/0 conn(0x7f508c0d1d60 msgr2=0x7f508c0d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:30.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.785+0000 7f508ad9d640 1 --2- 172.21.15.16:0/2973769313 >> v2:172.21.15.112:3300/0 conn(0x7f508c0d1d60 0x7f508c0d3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.785+0000 7f508ad9d640 1 -- 172.21.15.16:0/2973769313 >> v1:172.21.15.16:6789/0 conn(0x7f508c15d580 legacy=0x7f508c15f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.785+0000 7f508959a640 1 -- 172.21.15.16:0/2973769313 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716837565 0 0) 0x7f508c15ee90 con 0x7f508c15d580 2023-12-08T08:38:30.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.785+0000 7f508ad9d640 1 -- 172.21.15.16:0/2973769313 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c0d1560 con 0x7f508c15f890 2023-12-08T08:38:30.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.786+0000 7f508ad9d640 1 --2- 172.21.15.16:0/2973769313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c15f890 0x7f508c15d960 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f507c0097a0 tx=0x7f507c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f932a138f13ded4 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:30.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.786+0000 7f508959a640 1 -- 172.21.15.16:0/2973769313 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c02e070 con 0x7f508c15f890 2023-12-08T08:38:30.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.786+0000 7f508959a640 1 -- 172.21.15.16:0/2973769313 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c0315a0 con 0x7f508c15f890 2023-12-08T08:38:30.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.786+0000 7f508959a640 1 -- 172.21.15.16:0/2973769313 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c031930 con 0x7f508c15f890 2023-12-08T08:38:30.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.787+0000 7f5090d41640 1 -- 172.21.15.16:0/2973769313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c15f890 msgr2=0x7f508c15d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.787+0000 7f5090d41640 1 --2- 172.21.15.16:0/2973769313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c15f890 0x7f508c15d960 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f507c0097a0 tx=0x7f507c02b840 comp rx=0 tx=0).stop 2023-12-08T08:38:30.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.787+0000 7f5090d41640 1 -- 172.21.15.16:0/2973769313 shutdown_connections 2023-12-08T08:38:30.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.787+0000 7f5090d41640 1 --2- 172.21.15.16:0/2973769313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c15f890 0x7f508c15d960 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.787+0000 7f5090d41640 1 --2- 172.21.15.16:0/2973769313 >> v2:172.21.15.112:3300/0 conn(0x7f508c0d1d60 0x7f508c0d3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.787+0000 7f5090d41640 1 -- 172.21.15.16:0/2973769313 >> 172.21.15.16:0/2973769313 conn(0x7f508c0d2fb0 msgr2=0x7f508c0d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:30.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.787+0000 7f5090d41640 1 -- 172.21.15.16:0/2973769313 shutdown_connections 2023-12-08T08:38:30.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.787+0000 7f5090d41640 1 -- 172.21.15.16:0/2973769313 wait complete. 2023-12-08T08:38:30.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.788+0000 7f5090d41640 1 Processor -- start 2023-12-08T08:38:30.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.788+0000 7f5090d41640 1 -- start start 2023-12-08T08:38:30.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.789+0000 7f5090d41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c0d1d60 0x7f508c1f2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.789+0000 7f5090d41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f508c15d580 0x7f508c0c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.789+0000 7f5090d41640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508c15f890 0x7f508c1f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.789+0000 7f5090d41640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0c8ea0 con 0x7f508c0d1d60 2023-12-08T08:38:30.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.789+0000 7f5090d41640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0afce0 con 0x7f508c15d580 2023-12-08T08:38:30.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.789+0000 7f5090d41640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0c90f0 con 0x7f508c15f890 2023-12-08T08:38:30.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.789+0000 7f508a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c0d1d60 0x7f508c1f2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.789+0000 7f508a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c0d1d60 0x7f508c1f2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39410/0 (socket says 172.21.15.16:39410) 2023-12-08T08:38:30.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.789+0000 7f508a59c640 1 -- 172.21.15.16:0/1800074557 learned_addr learned my addr 172.21.15.16:0/1800074557 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:30.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.789+0000 7f5089d9b640 1 --2- 172.21.15.16:0/1800074557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f508c15d580 0x7f508c0c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.789+0000 7f508ad9d640 1 --2- 172.21.15.16:0/1800074557 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508c15f890 0x7f508c1f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.790+0000 7f508a59c640 1 -- 172.21.15.16:0/1800074557 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508c15f890 msgr2=0x7f508c1f63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.790+0000 7f508a59c640 1 --2- 172.21.15.16:0/1800074557 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508c15f890 0x7f508c1f63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.790+0000 7f508a59c640 1 -- 172.21.15.16:0/1800074557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f508c15d580 msgr2=0x7f508c0c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.790+0000 7f508a59c640 1 --2- 172.21.15.16:0/1800074557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f508c15d580 0x7f508c0c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.790+0000 7f508a59c640 1 -- 172.21.15.16:0/1800074557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c009400 con 0x7f508c0d1d60 2023-12-08T08:38:30.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.790+0000 7f508a59c640 1 --2- 172.21.15.16:0/1800074557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c0d1d60 0x7f508c1f2840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f508400c890 tx=0x7f508400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:30.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.790+0000 7f507affd640 1 -- 172.21.15.16:0/1800074557 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508400a640 con 0x7f508c0d1d60 2023-12-08T08:38:30.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.790+0000 7f507affd640 1 -- 172.21.15.16:0/1800074557 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508400a7c0 con 0x7f508c0d1d60 2023-12-08T08:38:30.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.790+0000 7f5090d41640 1 -- 172.21.15.16:0/1800074557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c0c93d0 con 0x7f508c0d1d60 2023-12-08T08:38:30.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.790+0000 7f507affd640 1 -- 172.21.15.16:0/1800074557 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508400ab70 con 0x7f508c0d1d60 2023-12-08T08:38:30.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.790+0000 7f5090d41640 1 -- 172.21.15.16:0/1800074557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c0c9990 con 0x7f508c0d1d60 2023-12-08T08:38:30.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.791+0000 7f5090d41640 1 -- 172.21.15.16:0/1800074557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c0afee0 con 0x7f508c0d1d60 2023-12-08T08:38:30.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.791+0000 7f507affd640 1 -- 172.21.15.16:0/1800074557 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f508400acd0 con 0x7f508c0d1d60 2023-12-08T08:38:30.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.792+0000 7f507affd640 1 --2- 172.21.15.16:0/1800074557 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f506c06ae40 0x7f506c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:30.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.792+0000 7f507affd640 1 -- 172.21.15.16:0/1800074557 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f50840891d0 con 0x7f508c0d1d60 2023-12-08T08:38:30.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.792+0000 7f5089d9b640 1 --2- 172.21.15.16:0/1800074557 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f506c06ae40 0x7f506c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:30.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.793+0000 7f5089d9b640 1 --2- 172.21.15.16:0/1800074557 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f506c06ae40 0x7f506c06d300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5080006fd0 tx=0x7f5080008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:30.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.796+0000 7f507affd640 1 -- 172.21.15.16:0/1800074557 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f508c0afee0 con 0x7f508c0d1d60 2023-12-08T08:38:30.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.985+0000 7f5090d41640 1 -- 172.21.15.16:0/1800074557 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f508c0d09d0 con 0x7f508c0d1d60 2023-12-08T08:38:30.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.985+0000 7f507affd640 1 -- 172.21.15.16:0/1800074557 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5084053f90 con 0x7f508c0d1d60 2023-12-08T08:38:30.987 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-12-08T08:38:30.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.987+0000 7f5090d41640 1 -- 172.21.15.16:0/1800074557 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f506c06ae40 msgr2=0x7f506c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.987+0000 7f5090d41640 1 --2- 172.21.15.16:0/1800074557 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f506c06ae40 0x7f506c06d300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5080006fd0 tx=0x7f5080008040 comp rx=0 tx=0).stop 2023-12-08T08:38:30.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.987+0000 7f5090d41640 1 -- 172.21.15.16:0/1800074557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c0d1d60 msgr2=0x7f508c1f2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:30.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.987+0000 7f5090d41640 1 --2- 172.21.15.16:0/1800074557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c0d1d60 0x7f508c1f2840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f508400c890 tx=0x7f508400cd50 comp rx=0 tx=0).stop 2023-12-08T08:38:30.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.987+0000 7f5090d41640 1 -- 172.21.15.16:0/1800074557 shutdown_connections 2023-12-08T08:38:30.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.987+0000 7f5090d41640 1 --2- 172.21.15.16:0/1800074557 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f506c06ae40 0x7f506c06d300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.987+0000 7f5090d41640 1 --2- 172.21.15.16:0/1800074557 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f508c15f890 0x7f508c1f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.987+0000 7f5090d41640 1 --2- 172.21.15.16:0/1800074557 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f508c15d580 0x7f508c0c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.987+0000 7f5090d41640 1 --2- 172.21.15.16:0/1800074557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508c0d1d60 0x7f508c1f2840 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:30.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.987+0000 7f5090d41640 1 -- 172.21.15.16:0/1800074557 >> 172.21.15.16:0/1800074557 conn(0x7f508c0d2fb0 msgr2=0x7f508c0d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:30.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.987+0000 7f5090d41640 1 -- 172.21.15.16:0/1800074557 shutdown_connections 2023-12-08T08:38:30.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:30.987+0000 7f5090d41640 1 -- 172.21.15.16:0/1800074557 wait complete. 2023-12-08T08:38:31.000 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:38:31.000 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T08:38:31.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:31.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.159+0000 7fa7b32c9640 1 Processor -- start 2023-12-08T08:38:31.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.159+0000 7fa7b32c9640 1 -- start start 2023-12-08T08:38:31.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.160+0000 7fa7b32c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac0d1d60 0x7fa7ac0d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.160+0000 7fa7b32c9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa7ac15f890 0x7fa7ac15d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.160+0000 7fa7b32c9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7ac15ee90 con 0x7fa7ac15d580 2023-12-08T08:38:31.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.160+0000 7fa7b32c9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7ac0d0fe0 con 0x7fa7ac15f890 2023-12-08T08:38:31.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.160+0000 7fa7b32c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac0d1280 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.160+0000 7fa7b1065640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa7ac15d580 0x7fa7ac15f180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52132/0 (socket says 172.21.15.16:52132) 2023-12-08T08:38:31.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.160+0000 7fa7b1065640 1 -- 172.21.15.16:0/1483646390 learned_addr learned my addr 172.21.15.16:0/1483646390 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:31.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.160+0000 7fa7b0864640 1 --2- 172.21.15.16:0/1483646390 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac0d1d60 0x7fa7ac0d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.160+0000 7fa7b1866640 1 --2- 172.21.15.16:0/1483646390 >> v2:172.21.15.112:3300/0 conn(0x7fa7ac15f890 0x7fa7ac15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.161+0000 7fa7abfff640 1 -- 172.21.15.16:0/1483646390 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3869333447 0 0) 0x7fa7ac15ee90 con 0x7fa7ac15d580 2023-12-08T08:38:31.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.161+0000 7fa7abfff640 1 -- 172.21.15.16:0/1483646390 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa794003520 con 0x7fa7ac15d580 2023-12-08T08:38:31.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.161+0000 7fa7abfff640 1 -- 172.21.15.16:0/1483646390 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 81546381 0 0) 0x7fa794003520 con 0x7fa7ac15d580 2023-12-08T08:38:31.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.161+0000 7fa7b0864640 1 -- 172.21.15.16:0/1483646390 >> v2:172.21.15.112:3300/0 conn(0x7fa7ac15f890 msgr2=0x7fa7ac15d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:31.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.161+0000 7fa7b0864640 1 --2- 172.21.15.16:0/1483646390 >> v2:172.21.15.112:3300/0 conn(0x7fa7ac15f890 0x7fa7ac15d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.161+0000 7fa7b0864640 1 -- 172.21.15.16:0/1483646390 >> v1:172.21.15.16:6789/0 conn(0x7fa7ac15d580 legacy=0x7fa7ac15f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.161+0000 7fa7b0864640 1 -- 172.21.15.16:0/1483646390 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac0d1560 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.161+0000 7fa7b1866640 1 --2- 172.21.15.16:0/1483646390 >> v2:172.21.15.112:3300/0 conn(0x7fa7ac15f890 0x7fa7ac15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:31.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.162+0000 7fa7b0864640 1 --2- 172.21.15.16:0/1483646390 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac0d1d60 0x7fa7ac0d3ad0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00097a0 tx=0x7fa7a002b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2333bfc52817e55a server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:31.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.162+0000 7fa7abfff640 1 -- 172.21.15.16:0/1483646390 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4001670 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.162+0000 7fa7abfff640 1 -- 172.21.15.16:0/1483646390 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a002e050 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.162+0000 7fa7abfff640 1 -- 172.21.15.16:0/1483646390 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a0030730 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.162+0000 7fa7b32c9640 1 -- 172.21.15.16:0/1483646390 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac0d1d60 msgr2=0x7fa7ac0d3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.162+0000 7fa7b32c9640 1 --2- 172.21.15.16:0/1483646390 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac0d1d60 0x7fa7ac0d3ad0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa7a00097a0 tx=0x7fa7a002b710 comp rx=0 tx=0).stop 2023-12-08T08:38:31.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.162+0000 7fa7b32c9640 1 -- 172.21.15.16:0/1483646390 shutdown_connections 2023-12-08T08:38:31.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.162+0000 7fa7b32c9640 1 --2- 172.21.15.16:0/1483646390 >> v2:172.21.15.112:3300/0 conn(0x7fa7ac15f890 0x7fa7ac15d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.162+0000 7fa7b32c9640 1 --2- 172.21.15.16:0/1483646390 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac0d1d60 0x7fa7ac0d3ad0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.162+0000 7fa7b32c9640 1 -- 172.21.15.16:0/1483646390 >> 172.21.15.16:0/1483646390 conn(0x7fa7ac0d2fb0 msgr2=0x7fa7ac0d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.162+0000 7fa7b32c9640 1 -- 172.21.15.16:0/1483646390 shutdown_connections 2023-12-08T08:38:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.162+0000 7fa7b32c9640 1 -- 172.21.15.16:0/1483646390 wait complete. 2023-12-08T08:38:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.164+0000 7fa7b32c9640 1 Processor -- start 2023-12-08T08:38:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.164+0000 7fa7b32c9640 1 -- start start 2023-12-08T08:38:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.164+0000 7fa7b32c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ac0d1d60 0x7fa7ac1f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.164+0000 7fa7b32c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7ac15d580 0x7fa7ac0c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.164+0000 7fa7b32c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac15f890 0x7fa7ac15c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.164+0000 7fa7b32c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac0c7450 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.164+0000 7fa7b32c9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac0c7730 con 0x7fa7ac15d580 2023-12-08T08:38:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.164+0000 7fa7b32c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ac1f6200 con 0x7fa7ac15f890 2023-12-08T08:38:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.164+0000 7fa7b1065640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ac0d1d60 0x7fa7ac1f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.164+0000 7fa7b1065640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ac0d1d60 0x7fa7ac1f6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39426/0 (socket says 172.21.15.16:39426) 2023-12-08T08:38:31.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.164+0000 7fa7b1065640 1 -- 172.21.15.16:0/12489754 learned_addr learned my addr 172.21.15.16:0/12489754 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.165+0000 7fa7b0864640 1 --2- 172.21.15.16:0/12489754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7ac15d580 0x7fa7ac0c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.165+0000 7fa7b1866640 1 --2- 172.21.15.16:0/12489754 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac15f890 0x7fa7ac15c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.165+0000 7fa7b1065640 1 -- 172.21.15.16:0/12489754 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac15f890 msgr2=0x7fa7ac15c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.165+0000 7fa7b1065640 1 --2- 172.21.15.16:0/12489754 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac15f890 0x7fa7ac15c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.165+0000 7fa7b1065640 1 -- 172.21.15.16:0/12489754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7ac15d580 msgr2=0x7fa7ac0c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.165+0000 7fa7b1065640 1 --2- 172.21.15.16:0/12489754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7ac15d580 0x7fa7ac0c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.165+0000 7fa7b1065640 1 -- 172.21.15.16:0/12489754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c009a70 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.165+0000 7fa7b1065640 1 --2- 172.21.15.16:0/12489754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ac0d1d60 0x7fa7ac1f6ce0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400c710 tx=0x7fa7a400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.165+0000 7fa7a9ffb640 1 -- 172.21.15.16:0/12489754 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4005ad0 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.165+0000 7fa7a9ffb640 1 -- 172.21.15.16:0/12489754 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a4005c30 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.165+0000 7fa7a9ffb640 1 -- 172.21.15.16:0/12489754 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4007070 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.165+0000 7fa7b32c9640 1 -- 172.21.15.16:0/12489754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a0009400 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.165+0000 7fa7b32c9640 1 -- 172.21.15.16:0/12489754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ac1f67c0 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.166+0000 7fa7b32c9640 1 -- 172.21.15.16:0/12489754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ac0d09b0 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.167+0000 7fa7a9ffb640 1 -- 172.21.15.16:0/12489754 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa7a40071d0 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.167+0000 7fa7a9ffb640 1 --2- 172.21.15.16:0/12489754 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa79806ae40 0x7fa79806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.167+0000 7fa7a9ffb640 1 -- 172.21.15.16:0/12489754 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa7a4089590 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.167+0000 7fa7b0864640 1 --2- 172.21.15.16:0/12489754 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa79806ae40 0x7fa79806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.168+0000 7fa7b0864640 1 --2- 172.21.15.16:0/12489754 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa79806ae40 0x7fa79806d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0034000 tx=0x7fa7a0008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:31.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.171+0000 7fa7a9ffb640 1 -- 172.21.15.16:0/12489754 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa7a40542d0 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.348+0000 7fa7b32c9640 1 -- 172.21.15.16:0/12489754 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7ac0b4e60 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.349+0000 7fa7a9ffb640 1 -- 172.21.15.16:0/12489754 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fa7a40578f0 con 0x7fa7ac0d1d60 2023-12-08T08:38:31.352 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:31.352 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:24.030625+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.352+0000 7fa7b32c9640 1 -- 172.21.15.16:0/12489754 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa79806ae40 msgr2=0x7fa79806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.352+0000 7fa7b32c9640 1 --2- 172.21.15.16:0/12489754 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa79806ae40 0x7fa79806d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa7a0034000 tx=0x7fa7a0008030 comp rx=0 tx=0).stop 2023-12-08T08:38:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.352+0000 7fa7b32c9640 1 -- 172.21.15.16:0/12489754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ac0d1d60 msgr2=0x7fa7ac1f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.352+0000 7fa7b32c9640 1 --2- 172.21.15.16:0/12489754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ac0d1d60 0x7fa7ac1f6ce0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400c710 tx=0x7fa7a400cb10 comp rx=0 tx=0).stop 2023-12-08T08:38:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.352+0000 7fa7b32c9640 1 -- 172.21.15.16:0/12489754 shutdown_connections 2023-12-08T08:38:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.352+0000 7fa7b32c9640 1 --2- 172.21.15.16:0/12489754 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa79806ae40 0x7fa79806d300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.352+0000 7fa7b32c9640 1 --2- 172.21.15.16:0/12489754 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa7ac15f890 0x7fa7ac15c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.352+0000 7fa7b32c9640 1 --2- 172.21.15.16:0/12489754 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa7ac15d580 0x7fa7ac0c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.352+0000 7fa7b32c9640 1 --2- 172.21.15.16:0/12489754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7ac0d1d60 0x7fa7ac1f6ce0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.352+0000 7fa7b32c9640 1 -- 172.21.15.16:0/12489754 >> 172.21.15.16:0/12489754 conn(0x7fa7ac0d2fb0 msgr2=0x7fa7ac15e140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:31.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.352+0000 7fa7b32c9640 1 -- 172.21.15.16:0/12489754 shutdown_connections 2023-12-08T08:38:31.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.352+0000 7fa7b32c9640 1 -- 172.21.15.16:0/12489754 wait complete. 2023-12-08T08:38:31.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:38:31.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:38:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.472+0000 7f950452c640 1 Processor -- start 2023-12-08T08:38:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.472+0000 7f950452c640 1 -- start start 2023-12-08T08:38:31.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f950452c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94fc0b8950 0x7f94fc0ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f950452c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f94fc0cfc80 0x7f94fc0b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f950452c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94fc0b5f00 con 0x7f94fc0cebc0 2023-12-08T08:38:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f950452c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94fc0b6170 con 0x7f94fc0cfc80 2023-12-08T08:38:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f950452c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0b6450 con 0x7f94fc0b8950 2023-12-08T08:38:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f9502ac9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f94fc0cfc80 0x7f94fc0b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f9502ac9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f94fc0cfc80 0x7f94fc0b6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39438/0 (socket says 172.21.15.16:39438) 2023-12-08T08:38:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f9502ac9640 1 -- 172.21.15.16:0/3743592451 learned_addr learned my addr 172.21.15.16:0/3743592451 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f9501ac7640 1 --1- 172.21.15.16:0/3743592451 >> v1:172.21.15.16:6789/0 conn(0x7f94fc0cebc0 0x7f94fc0cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52134/0 (socket says 172.21.15.16:52134) 2023-12-08T08:38:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f95022c8640 1 --2- 172.21.15.16:0/3743592451 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94fc0b8950 0x7f94fc0ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f9502ac9640 1 -- 172.21.15.16:0/3743592451 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94fc0b8950 msgr2=0x7f94fc0ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f9502ac9640 1 --2- 172.21.15.16:0/3743592451 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94fc0b8950 0x7f94fc0ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f9502ac9640 1 -- 172.21.15.16:0/3743592451 >> v1:172.21.15.16:6789/0 conn(0x7f94fc0cebc0 legacy=0x7f94fc0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f9502ac9640 1 -- 172.21.15.16:0/3743592451 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc0b6730 con 0x7f94fc0cfc80 2023-12-08T08:38:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f95012c6640 1 -- 172.21.15.16:0/3743592451 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375630340 0 0) 0x7f94fc0b5f00 con 0x7f94fc0cebc0 2023-12-08T08:38:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.473+0000 7f95022c8640 1 --2- 172.21.15.16:0/3743592451 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94fc0b8950 0x7f94fc0ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.474+0000 7f9502ac9640 1 --2- 172.21.15.16:0/3743592451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94fc0cfc80 0x7f94fc0b6c50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009e50 tx=0x7f94f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188802e7b0cde9e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.474+0000 7f95012c6640 1 -- 172.21.15.16:0/3743592451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f402ca70 con 0x7f94fc0cfc80 2023-12-08T08:38:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.474+0000 7f95012c6640 1 -- 172.21.15.16:0/3743592451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f402cbd0 con 0x7f94fc0cfc80 2023-12-08T08:38:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.474+0000 7f95012c6640 1 -- 172.21.15.16:0/3743592451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f40343f0 con 0x7f94fc0cfc80 2023-12-08T08:38:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.475+0000 7f950452c640 1 -- 172.21.15.16:0/3743592451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94fc0cfc80 msgr2=0x7f94fc0b6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.475+0000 7f950452c640 1 --2- 172.21.15.16:0/3743592451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94fc0cfc80 0x7f94fc0b6c50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009e50 tx=0x7f94f402bed0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.475+0000 7f950452c640 1 -- 172.21.15.16:0/3743592451 shutdown_connections 2023-12-08T08:38:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.475+0000 7f950452c640 1 --2- 172.21.15.16:0/3743592451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94fc0cfc80 0x7f94fc0b6c50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.475+0000 7f950452c640 1 --2- 172.21.15.16:0/3743592451 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94fc0b8950 0x7f94fc0ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.475+0000 7f950452c640 1 -- 172.21.15.16:0/3743592451 >> 172.21.15.16:0/3743592451 conn(0x7f94fc0d3010 msgr2=0x7f94fc0d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.475+0000 7f950452c640 1 -- 172.21.15.16:0/3743592451 shutdown_connections 2023-12-08T08:38:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.475+0000 7f950452c640 1 -- 172.21.15.16:0/3743592451 wait complete. 2023-12-08T08:38:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.476+0000 7f950452c640 1 Processor -- start 2023-12-08T08:38:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.476+0000 7f950452c640 1 -- start start 2023-12-08T08:38:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f950452c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94fc0b8950 0x7f94fc1f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f950452c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94fc0cebc0 0x7f94fc1f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f950452c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94fc0cfc80 0x7f94fc0b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f950452c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0b1980 con 0x7f94fc0cfc80 2023-12-08T08:38:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f950452c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc14fb30 con 0x7f94fc0b8950 2023-12-08T08:38:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f950452c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc14efb0 con 0x7f94fc0cebc0 2023-12-08T08:38:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f9502ac9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94fc0cfc80 0x7f94fc0b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f9502ac9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94fc0cfc80 0x7f94fc0b6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39450/0 (socket says 172.21.15.16:39450) 2023-12-08T08:38:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f9502ac9640 1 -- 172.21.15.16:0/3898782093 learned_addr learned my addr 172.21.15.16:0/3898782093 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f95022c8640 1 --2- 172.21.15.16:0/3898782093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94fc0b8950 0x7f94fc1f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f9501ac7640 1 --2- 172.21.15.16:0/3898782093 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94fc0cebc0 0x7f94fc1f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f9502ac9640 1 -- 172.21.15.16:0/3898782093 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94fc0cebc0 msgr2=0x7f94fc1f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f9502ac9640 1 --2- 172.21.15.16:0/3898782093 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94fc0cebc0 0x7f94fc1f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f9502ac9640 1 -- 172.21.15.16:0/3898782093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94fc0b8950 msgr2=0x7f94fc1f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f9502ac9640 1 --2- 172.21.15.16:0/3898782093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94fc0b8950 0x7f94fc1f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f9502ac9640 1 -- 172.21.15.16:0/3898782093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f4009b30 con 0x7f94fc0cfc80 2023-12-08T08:38:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.477+0000 7f9502ac9640 1 --2- 172.21.15.16:0/3898782093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94fc0cfc80 0x7f94fc0b6b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009e50 tx=0x7f94f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.478+0000 7f94faffd640 1 -- 172.21.15.16:0/3898782093 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f4009510 con 0x7f94fc0cfc80 2023-12-08T08:38:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.478+0000 7f94faffd640 1 -- 172.21.15.16:0/3898782093 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f403e450 con 0x7f94fc0cfc80 2023-12-08T08:38:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.478+0000 7f950452c640 1 -- 172.21.15.16:0/3898782093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94fc14f290 con 0x7f94fc0cfc80 2023-12-08T08:38:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.478+0000 7f94faffd640 1 -- 172.21.15.16:0/3898782093 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f403e800 con 0x7f94fc0cfc80 2023-12-08T08:38:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.478+0000 7f950452c640 1 -- 172.21.15.16:0/3898782093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc14f850 con 0x7f94fc0cfc80 2023-12-08T08:38:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.479+0000 7f950452c640 1 -- 172.21.15.16:0/3898782093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94fc08b150 con 0x7f94fc0cfc80 2023-12-08T08:38:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.479+0000 7f94faffd640 1 -- 172.21.15.16:0/3898782093 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f94f403e960 con 0x7f94fc0cfc80 2023-12-08T08:38:31.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.479+0000 7f94faffd640 1 --2- 172.21.15.16:0/3898782093 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f94e806ae40 0x7f94e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.480+0000 7f95022c8640 1 --2- 172.21.15.16:0/3898782093 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f94e806ae40 0x7f94e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.480+0000 7f94faffd640 1 -- 172.21.15.16:0/3898782093 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f94f40ac610 con 0x7f94fc0cfc80 2023-12-08T08:38:31.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.480+0000 7f95022c8640 1 --2- 172.21.15.16:0/3898782093 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f94e806ae40 0x7f94e806d300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f94ec006fd0 tx=0x7f94ec008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:31.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.484+0000 7f94faffd640 1 -- 172.21.15.16:0/3898782093 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94f4077460 con 0x7f94fc0cfc80 2023-12-08T08:38:31.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.665+0000 7f950452c640 1 -- 172.21.15.16:0/3898782093 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94fc067430 con 0x7f94fc0cfc80 2023-12-08T08:38:31.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.665+0000 7f94faffd640 1 -- 172.21.15.16:0/3898782093 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94f407aa80 con 0x7f94fc0cfc80 2023-12-08T08:38:31.667 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:38:31.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.667+0000 7f950452c640 1 -- 172.21.15.16:0/3898782093 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f94e806ae40 msgr2=0x7f94e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.667+0000 7f950452c640 1 --2- 172.21.15.16:0/3898782093 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f94e806ae40 0x7f94e806d300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f94ec006fd0 tx=0x7f94ec008040 comp rx=0 tx=0).stop 2023-12-08T08:38:31.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.667+0000 7f950452c640 1 -- 172.21.15.16:0/3898782093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94fc0cfc80 msgr2=0x7f94fc0b6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.667+0000 7f950452c640 1 --2- 172.21.15.16:0/3898782093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94fc0cfc80 0x7f94fc0b6b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009e50 tx=0x7f94f4004a40 comp rx=0 tx=0).stop 2023-12-08T08:38:31.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.667+0000 7f950452c640 1 -- 172.21.15.16:0/3898782093 shutdown_connections 2023-12-08T08:38:31.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.667+0000 7f950452c640 1 --2- 172.21.15.16:0/3898782093 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f94e806ae40 0x7f94e806d300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.667+0000 7f950452c640 1 --2- 172.21.15.16:0/3898782093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94fc0cfc80 0x7f94fc0b6b50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.667+0000 7f950452c640 1 --2- 172.21.15.16:0/3898782093 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f94fc0cebc0 0x7f94fc1f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.667+0000 7f950452c640 1 --2- 172.21.15.16:0/3898782093 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f94fc0b8950 0x7f94fc1f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.667+0000 7f950452c640 1 -- 172.21.15.16:0/3898782093 >> 172.21.15.16:0/3898782093 conn(0x7f94fc0d3010 msgr2=0x7f94fc06c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:31.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.667+0000 7f950452c640 1 -- 172.21.15.16:0/3898782093 shutdown_connections 2023-12-08T08:38:31.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.667+0000 7f950452c640 1 -- 172.21.15.16:0/3898782093 wait complete. 2023-12-08T08:38:31.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:38:31.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.788+0000 7fa195c89640 1 Processor -- start 2023-12-08T08:38:31.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.789+0000 7fa195c89640 1 -- start start 2023-12-08T08:38:31.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.789+0000 7fa195c89640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa19015d590 0x7fa1900ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.789+0000 7fa195c89640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1900afeb0 0x7fa1900cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.789+0000 7fa195c89640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa19006d650 con 0x7fa1900cf790 2023-12-08T08:38:31.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.789+0000 7fa195c89640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1900adbe0 con 0x7fa1900afeb0 2023-12-08T08:38:31.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.789+0000 7fa195c89640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa19015d970 con 0x7fa19015d590 2023-12-08T08:38:31.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.789+0000 7fa18effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1900afeb0 0x7fa1900cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.789+0000 7fa18effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1900afeb0 0x7fa1900cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39464/0 (socket says 172.21.15.16:39464) 2023-12-08T08:38:31.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.789+0000 7fa18effd640 1 -- 172.21.15.16:0/724376062 learned_addr learned my addr 172.21.15.16:0/724376062 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:31.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.789+0000 7fa18ffff640 1 --1- 172.21.15.16:0/724376062 >> v1:172.21.15.16:6789/0 conn(0x7fa1900cf790 0x7fa190161070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52142/0 (socket says 172.21.15.16:52142) 2023-12-08T08:38:31.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.790+0000 7fa18f7fe640 1 --2- 172.21.15.16:0/724376062 >> v2:172.21.15.112:3300/0 conn(0x7fa19015d590 0x7fa1900ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.790+0000 7fa18effd640 1 -- 172.21.15.16:0/724376062 >> v2:172.21.15.112:3300/0 conn(0x7fa19015d590 msgr2=0x7fa1900ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:31.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.790+0000 7fa18effd640 1 --2- 172.21.15.16:0/724376062 >> v2:172.21.15.112:3300/0 conn(0x7fa19015d590 0x7fa1900ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.790+0000 7fa18effd640 1 -- 172.21.15.16:0/724376062 >> v1:172.21.15.16:6789/0 conn(0x7fa1900cf790 legacy=0x7fa190161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.790+0000 7fa18e7fc640 1 -- 172.21.15.16:0/724376062 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698794674 0 0) 0x7fa188001490 con 0x7fa1900cf790 2023-12-08T08:38:31.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.790+0000 7fa18effd640 1 -- 172.21.15.16:0/724376062 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19006c810 con 0x7fa1900afeb0 2023-12-08T08:38:31.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.791+0000 7fa18effd640 1 --2- 172.21.15.16:0/724376062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1900afeb0 0x7fa1900cf250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa19006e0c0 tx=0x7fa18402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fffe7f633ad99d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:31.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.791+0000 7fa18e7fc640 1 -- 172.21.15.16:0/724376062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa184030860 con 0x7fa1900afeb0 2023-12-08T08:38:31.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.791+0000 7fa18e7fc640 1 -- 172.21.15.16:0/724376062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1840309c0 con 0x7fa1900afeb0 2023-12-08T08:38:31.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.791+0000 7fa18e7fc640 1 -- 172.21.15.16:0/724376062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa184030d50 con 0x7fa1900afeb0 2023-12-08T08:38:31.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.791+0000 7fa195c89640 1 -- 172.21.15.16:0/724376062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1900afeb0 msgr2=0x7fa1900cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.791+0000 7fa195c89640 1 --2- 172.21.15.16:0/724376062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1900afeb0 0x7fa1900cf250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa19006e0c0 tx=0x7fa18402b9a0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.791+0000 7fa195c89640 1 -- 172.21.15.16:0/724376062 shutdown_connections 2023-12-08T08:38:31.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.791+0000 7fa195c89640 1 --2- 172.21.15.16:0/724376062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1900afeb0 0x7fa1900cf250 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.791+0000 7fa195c89640 1 --2- 172.21.15.16:0/724376062 >> v2:172.21.15.112:3300/0 conn(0x7fa19015d590 0x7fa1900ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.791+0000 7fa195c89640 1 -- 172.21.15.16:0/724376062 >> 172.21.15.16:0/724376062 conn(0x7fa1900d0e10 msgr2=0x7fa1900d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:31.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.791+0000 7fa195c89640 1 -- 172.21.15.16:0/724376062 shutdown_connections 2023-12-08T08:38:31.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.791+0000 7fa195c89640 1 -- 172.21.15.16:0/724376062 wait complete. 2023-12-08T08:38:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.793+0000 7fa195c89640 1 Processor -- start 2023-12-08T08:38:31.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.793+0000 7fa195c89640 1 -- start start 2023-12-08T08:38:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.793+0000 7fa195c89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1900afeb0 0x7fa19014b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.793+0000 7fa195c89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1900cf790 0x7fa19014abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.793+0000 7fa195c89640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa19015d590 0x7fa1900cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.793+0000 7fa195c89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1900ce5d0 con 0x7fa1900afeb0 2023-12-08T08:38:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.793+0000 7fa195c89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19015e5b0 con 0x7fa1900cf790 2023-12-08T08:38:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.793+0000 7fa195c89640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa19015e890 con 0x7fa19015d590 2023-12-08T08:38:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa18f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1900afeb0 0x7fa19014b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa18f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1900afeb0 0x7fa19014b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39478/0 (socket says 172.21.15.16:39478) 2023-12-08T08:38:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa18f7fe640 1 -- 172.21.15.16:0/4069503373 learned_addr learned my addr 172.21.15.16:0/4069503373 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa18effd640 1 --2- 172.21.15.16:0/4069503373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1900cf790 0x7fa19014abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa18ffff640 1 --2- 172.21.15.16:0/4069503373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa19015d590 0x7fa1900cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa18f7fe640 1 -- 172.21.15.16:0/4069503373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa19015d590 msgr2=0x7fa1900cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa18f7fe640 1 --2- 172.21.15.16:0/4069503373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa19015d590 0x7fa1900cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa18f7fe640 1 -- 172.21.15.16:0/4069503373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1900cf790 msgr2=0x7fa19014abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa18f7fe640 1 --2- 172.21.15.16:0/4069503373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1900cf790 0x7fa19014abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa18f7fe640 1 -- 172.21.15.16:0/4069503373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1840094d0 con 0x7fa1900afeb0 2023-12-08T08:38:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa18f7fe640 1 --2- 172.21.15.16:0/4069503373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1900afeb0 0x7fa19014b500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa18000d5f0 tx=0x7fa18000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa177fff640 1 -- 172.21.15.16:0/4069503373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1800125f0 con 0x7fa1900afeb0 2023-12-08T08:38:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa177fff640 1 -- 172.21.15.16:0/4069503373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa180012750 con 0x7fa1900afeb0 2023-12-08T08:38:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa195c89640 1 -- 172.21.15.16:0/4069503373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19015eb70 con 0x7fa1900afeb0 2023-12-08T08:38:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa177fff640 1 -- 172.21.15.16:0/4069503373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa180012b00 con 0x7fa1900afeb0 2023-12-08T08:38:31.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.794+0000 7fa195c89640 1 -- 172.21.15.16:0/4069503373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19014b110 con 0x7fa1900afeb0 2023-12-08T08:38:31.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.795+0000 7fa195c89640 1 -- 172.21.15.16:0/4069503373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1900af150 con 0x7fa1900afeb0 2023-12-08T08:38:31.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.796+0000 7fa177fff640 1 -- 172.21.15.16:0/4069503373 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa180012c60 con 0x7fa1900afeb0 2023-12-08T08:38:31.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.796+0000 7fa177fff640 1 --2- 172.21.15.16:0/4069503373 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa17c06ae40 0x7fa17c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:31.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.796+0000 7fa177fff640 1 -- 172.21.15.16:0/4069503373 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa180089e80 con 0x7fa1900afeb0 2023-12-08T08:38:31.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.796+0000 7fa18effd640 1 --2- 172.21.15.16:0/4069503373 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa17c06ae40 0x7fa17c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:31.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.797+0000 7fa18effd640 1 --2- 172.21.15.16:0/4069503373 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa17c06ae40 0x7fa17c06d300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa184036000 tx=0x7fa184008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:31.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.800+0000 7fa177fff640 1 -- 172.21.15.16:0/4069503373 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa180054ba0 con 0x7fa1900afeb0 2023-12-08T08:38:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.979+0000 7fa195c89640 1 -- 172.21.15.16:0/4069503373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1900af330 con 0x7fa1900afeb0 2023-12-08T08:38:31.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.980+0000 7fa177fff640 1 -- 172.21.15.16:0/4069503373 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1800581c0 con 0x7fa1900afeb0 2023-12-08T08:38:31.982 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:38:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.982+0000 7fa195c89640 1 -- 172.21.15.16:0/4069503373 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa17c06ae40 msgr2=0x7fa17c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.982+0000 7fa195c89640 1 --2- 172.21.15.16:0/4069503373 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa17c06ae40 0x7fa17c06d300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa184036000 tx=0x7fa184008030 comp rx=0 tx=0).stop 2023-12-08T08:38:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.982+0000 7fa195c89640 1 -- 172.21.15.16:0/4069503373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1900afeb0 msgr2=0x7fa19014b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.982+0000 7fa195c89640 1 --2- 172.21.15.16:0/4069503373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1900afeb0 0x7fa19014b500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa18000d5f0 tx=0x7fa18000dab0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.982+0000 7fa195c89640 1 -- 172.21.15.16:0/4069503373 shutdown_connections 2023-12-08T08:38:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.982+0000 7fa195c89640 1 --2- 172.21.15.16:0/4069503373 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa17c06ae40 0x7fa17c06d300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.982+0000 7fa195c89640 1 --2- 172.21.15.16:0/4069503373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa19015d590 0x7fa1900cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.982+0000 7fa195c89640 1 --2- 172.21.15.16:0/4069503373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1900cf790 0x7fa19014abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.982+0000 7fa195c89640 1 --2- 172.21.15.16:0/4069503373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1900afeb0 0x7fa19014b500 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.982+0000 7fa195c89640 1 -- 172.21.15.16:0/4069503373 >> 172.21.15.16:0/4069503373 conn(0x7fa1900d0e10 msgr2=0x7fa19015c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.982+0000 7fa195c89640 1 -- 172.21.15.16:0/4069503373 shutdown_connections 2023-12-08T08:38:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:31.982+0000 7fa195c89640 1 -- 172.21.15.16:0/4069503373 wait complete. 2023-12-08T08:38:31.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:38:32.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.102+0000 7f4a0d143640 1 Processor -- start 2023-12-08T08:38:32.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.102+0000 7f4a0d143640 1 -- start start 2023-12-08T08:38:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.102+0000 7f4a0d143640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4a080cf570 0x7f4a080cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.102+0000 7f4a0d143640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a080b6c50 0x7f4a080b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.102+0000 7f4a0d143640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a080cfef0 con 0x7f4a080b8950 2023-12-08T08:38:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.102+0000 7f4a0d143640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a080b5f00 con 0x7f4a080b6c50 2023-12-08T08:38:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.102+0000 7f4a0d143640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a080b6170 con 0x7f4a080cf570 2023-12-08T08:38:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.103+0000 7f4a0759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a080b6c50 0x7f4a080b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.103+0000 7f4a06d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4a080b8950 0x7f4a080ce680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52146/0 (socket says 172.21.15.16:52146) 2023-12-08T08:38:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.103+0000 7f4a0759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a080b6c50 0x7f4a080b70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39492/0 (socket says 172.21.15.16:39492) 2023-12-08T08:38:32.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.103+0000 7f4a06d9d640 1 -- 172.21.15.16:0/2820538999 learned_addr learned my addr 172.21.15.16:0/2820538999 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.103+0000 7f4a0659c640 1 --2- 172.21.15.16:0/2820538999 >> v2:172.21.15.112:3300/0 conn(0x7f4a080cf570 0x7f4a080cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.103+0000 7f4a0759e640 1 -- 172.21.15.16:0/2820538999 >> v2:172.21.15.112:3300/0 conn(0x7f4a080cf570 msgr2=0x7f4a080cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.103+0000 7f4a0759e640 1 --2- 172.21.15.16:0/2820538999 >> v2:172.21.15.112:3300/0 conn(0x7f4a080cf570 0x7f4a080cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.103+0000 7f4a0759e640 1 -- 172.21.15.16:0/2820538999 >> v1:172.21.15.16:6789/0 conn(0x7f4a080b8950 legacy=0x7f4a080ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.103+0000 7f4a05d9b640 1 -- 172.21.15.16:0/2820538999 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209217442 0 0) 0x7f4a080cfef0 con 0x7f4a080b8950 2023-12-08T08:38:32.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.103+0000 7f4a0759e640 1 -- 172.21.15.16:0/2820538999 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a080b6450 con 0x7f4a080b6c50 2023-12-08T08:38:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.104+0000 7f4a0759e640 1 --2- 172.21.15.16:0/2820538999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a080b6c50 0x7f4a080b70a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4a0806a800 tx=0x7f4a0002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90188bba369af11d server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.104+0000 7f4a05d9b640 1 -- 172.21.15.16:0/2820538999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0002cbf0 con 0x7f4a080b6c50 2023-12-08T08:38:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.104+0000 7f4a05d9b640 1 -- 172.21.15.16:0/2820538999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0002cd50 con 0x7f4a080b6c50 2023-12-08T08:38:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.104+0000 7f4a05d9b640 1 -- 172.21.15.16:0/2820538999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00034660 con 0x7f4a080b6c50 2023-12-08T08:38:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.104+0000 7f4a0d143640 1 -- 172.21.15.16:0/2820538999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a080b6c50 msgr2=0x7f4a080b70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.104+0000 7f4a0d143640 1 --2- 172.21.15.16:0/2820538999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a080b6c50 0x7f4a080b70a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4a0806a800 tx=0x7f4a0002be40 comp rx=0 tx=0).stop 2023-12-08T08:38:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.104+0000 7f4a0d143640 1 -- 172.21.15.16:0/2820538999 shutdown_connections 2023-12-08T08:38:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.104+0000 7f4a0d143640 1 --2- 172.21.15.16:0/2820538999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a080b6c50 0x7f4a080b70a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.104+0000 7f4a0d143640 1 --2- 172.21.15.16:0/2820538999 >> v2:172.21.15.112:3300/0 conn(0x7f4a080cf570 0x7f4a080cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.104+0000 7f4a0d143640 1 -- 172.21.15.16:0/2820538999 >> 172.21.15.16:0/2820538999 conn(0x7f4a080d3010 msgr2=0x7f4a080d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:32.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.104+0000 7f4a0d143640 1 -- 172.21.15.16:0/2820538999 shutdown_connections 2023-12-08T08:38:32.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.104+0000 7f4a0d143640 1 -- 172.21.15.16:0/2820538999 wait complete. 2023-12-08T08:38:32.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.106+0000 7f4a0d143640 1 Processor -- start 2023-12-08T08:38:32.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.106+0000 7f4a0d143640 1 -- start start 2023-12-08T08:38:32.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0d143640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080b6c50 0x7f4a080c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0d143640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a080b8950 0x7f4a080ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0d143640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a080cf570 0x7f4a080cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0d143640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a080cbf50 con 0x7f4a080cf570 2023-12-08T08:38:32.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0d143640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a080cb2d0 con 0x7f4a080b6c50 2023-12-08T08:38:32.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0d143640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a080cb5b0 con 0x7f4a080b8950 2023-12-08T08:38:32.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a080cf570 0x7f4a080cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a080cf570 0x7f4a080cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39498/0 (socket says 172.21.15.16:39498) 2023-12-08T08:38:32.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0759e640 1 -- 172.21.15.16:0/251917611 learned_addr learned my addr 172.21.15.16:0/251917611 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:32.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0659c640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a080b8950 0x7f4a080ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a06d9d640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080b6c50 0x7f4a080c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0759e640 1 -- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a080b8950 msgr2=0x7f4a080ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0759e640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a080b8950 0x7f4a080ccc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0759e640 1 -- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080b6c50 msgr2=0x7f4a080c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0759e640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080b6c50 0x7f4a080c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0759e640 1 -- 172.21.15.16:0/251917611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a00009a70 con 0x7f4a080cf570 2023-12-08T08:38:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0659c640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a080b8950 0x7f4a080ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a06d9d640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080b6c50 0x7f4a080c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0759e640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a080cf570 0x7f4a080cd170 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009e80 tx=0x7f4a00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f49ff7fe640 1 -- 172.21.15.16:0/251917611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00040650 con 0x7f4a080cf570 2023-12-08T08:38:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f49ff7fe640 1 -- 172.21.15.16:0/251917611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a000407b0 con 0x7f4a080cf570 2023-12-08T08:38:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f4a0d143640 1 -- 172.21.15.16:0/251917611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a080cb890 con 0x7f4a080cf570 2023-12-08T08:38:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.107+0000 7f49ff7fe640 1 -- 172.21.15.16:0/251917611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a00040b60 con 0x7f4a080cf570 2023-12-08T08:38:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.108+0000 7f4a0d143640 1 -- 172.21.15.16:0/251917611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a080afca0 con 0x7f4a080cf570 2023-12-08T08:38:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.109+0000 7f4a0d143640 1 -- 172.21.15.16:0/251917611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0808b150 con 0x7f4a080cf570 2023-12-08T08:38:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.109+0000 7f49ff7fe640 1 -- 172.21.15.16:0/251917611 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a00040cc0 con 0x7f4a080cf570 2023-12-08T08:38:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.109+0000 7f49ff7fe640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f49e806ae40 0x7f49e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.109+0000 7f49ff7fe640 1 -- 172.21.15.16:0/251917611 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4a000ac6b0 con 0x7f4a080cf570 2023-12-08T08:38:32.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.110+0000 7f4a06d9d640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f49e806ae40 0x7f49e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.110+0000 7f4a06d9d640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f49e806ae40 0x7f49e806d300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f49f0000f60 tx=0x7f49f0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.114+0000 7f49ff7fe640 1 -- 172.21.15.16:0/251917611 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a00077470 con 0x7f4a080cf570 2023-12-08T08:38:32.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.292+0000 7f4a0d143640 1 -- 172.21.15.16:0/251917611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a080cc090 con 0x7f4a080cf570 2023-12-08T08:38:32.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.292+0000 7f49ff7fe640 1 -- 172.21.15.16:0/251917611 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a080cc090 con 0x7f4a080cf570 2023-12-08T08:38:32.294 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:38:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.294+0000 7f4a0d143640 1 -- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f49e806ae40 msgr2=0x7f49e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.294+0000 7f4a0d143640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f49e806ae40 0x7f49e806d300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f49f0000f60 tx=0x7f49f0009040 comp rx=0 tx=0).stop 2023-12-08T08:38:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.294+0000 7f4a0d143640 1 -- 172.21.15.16:0/251917611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a080cf570 msgr2=0x7f4a080cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.294+0000 7f4a0d143640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a080cf570 0x7f4a080cd170 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009e80 tx=0x7f4a00004a40 comp rx=0 tx=0).stop 2023-12-08T08:38:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.294+0000 7f4a0d143640 1 -- 172.21.15.16:0/251917611 shutdown_connections 2023-12-08T08:38:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.294+0000 7f4a0d143640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f49e806ae40 0x7f49e806d300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.294+0000 7f4a0d143640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a080cf570 0x7f4a080cd170 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.294+0000 7f4a0d143640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a080b8950 0x7f4a080ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.294+0000 7f4a0d143640 1 --2- 172.21.15.16:0/251917611 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a080b6c50 0x7f4a080c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.294+0000 7f4a0d143640 1 -- 172.21.15.16:0/251917611 >> 172.21.15.16:0/251917611 conn(0x7f4a080d3010 msgr2=0x7f4a080d33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:32.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.294+0000 7f4a0d143640 1 -- 172.21.15.16:0/251917611 shutdown_connections 2023-12-08T08:38:32.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.294+0000 7f4a0d143640 1 -- 172.21.15.16:0/251917611 wait complete. 2023-12-08T08:38:32.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:38:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.413+0000 7fb8990ad640 1 Processor -- start 2023-12-08T08:38:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.413+0000 7fb8990ad640 1 -- start start 2023-12-08T08:38:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb8990ad640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb89415d4f0 0x7fb8940b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb8990ad640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8940cef90 0x7fb8940cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb8990ad640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8940b2280 con 0x7fb8940b0920 2023-12-08T08:38:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb8990ad640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb89406bb40 con 0x7fb8940cef90 2023-12-08T08:38:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb8990ad640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940b1430 con 0x7fb89415d4f0 2023-12-08T08:38:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb89359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8940cef90 0x7fb8940cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb89259c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb8940b0920 0x7fb8940b0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52150/0 (socket says 172.21.15.16:52150) 2023-12-08T08:38:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb89259c640 1 -- 172.21.15.16:0/3892154124 learned_addr learned my addr 172.21.15.16:0/3892154124 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb892d9d640 1 --2- 172.21.15.16:0/3892154124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb89415d4f0 0x7fb8940b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb89359e640 1 -- 172.21.15.16:0/3892154124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb89415d4f0 msgr2=0x7fb8940b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb89359e640 1 --2- 172.21.15.16:0/3892154124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb89415d4f0 0x7fb8940b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb89359e640 1 -- 172.21.15.16:0/3892154124 >> v1:172.21.15.16:6789/0 conn(0x7fb8940b0920 legacy=0x7fb8940b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb89359e640 1 -- 172.21.15.16:0/3892154124 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8940b2710 con 0x7fb8940cef90 2023-12-08T08:38:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb891d9b640 1 -- 172.21.15.16:0/3892154124 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3994084729 0 0) 0x7fb8940b2280 con 0x7fb8940b0920 2023-12-08T08:38:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.414+0000 7fb892d9d640 1 --2- 172.21.15.16:0/3892154124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb89415d4f0 0x7fb8940b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:32.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.415+0000 7fb89359e640 1 --2- 172.21.15.16:0/3892154124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8940cef90 0x7fb8940cf420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0097a0 tx=0x7fb88c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abb34af8615737d server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.415+0000 7fb891d9b640 1 -- 172.21.15.16:0/3892154124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c031830 con 0x7fb8940cef90 2023-12-08T08:38:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.415+0000 7fb891d9b640 1 -- 172.21.15.16:0/3892154124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c031990 con 0x7fb8940cef90 2023-12-08T08:38:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.415+0000 7fb891d9b640 1 -- 172.21.15.16:0/3892154124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c031d20 con 0x7fb8940cef90 2023-12-08T08:38:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.416+0000 7fb8990ad640 1 -- 172.21.15.16:0/3892154124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8940cef90 msgr2=0x7fb8940cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.416+0000 7fb8990ad640 1 --2- 172.21.15.16:0/3892154124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8940cef90 0x7fb8940cf420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0097a0 tx=0x7fb88c02b8d0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.416+0000 7fb8990ad640 1 -- 172.21.15.16:0/3892154124 shutdown_connections 2023-12-08T08:38:32.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.416+0000 7fb8990ad640 1 --2- 172.21.15.16:0/3892154124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8940cef90 0x7fb8940cf420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.416+0000 7fb8990ad640 1 --2- 172.21.15.16:0/3892154124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb89415d4f0 0x7fb8940b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.416+0000 7fb8990ad640 1 -- 172.21.15.16:0/3892154124 >> 172.21.15.16:0/3892154124 conn(0x7fb8940d0e00 msgr2=0x7fb8940d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.416+0000 7fb8990ad640 1 -- 172.21.15.16:0/3892154124 shutdown_connections 2023-12-08T08:38:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.416+0000 7fb8990ad640 1 -- 172.21.15.16:0/3892154124 wait complete. 2023-12-08T08:38:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.417+0000 7fb8990ad640 1 Processor -- start 2023-12-08T08:38:32.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.417+0000 7fb8990ad640 1 -- start start 2023-12-08T08:38:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.418+0000 7fb8990ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8940b0920 0x7fb8940cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.418+0000 7fb8990ad640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8940cef90 0x7fb8941f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.418+0000 7fb8990ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb89415d4f0 0x7fb89406e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.418+0000 7fb8990ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940c61d0 con 0x7fb89415d4f0 2023-12-08T08:38:32.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.418+0000 7fb8990ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8941f7430 con 0x7fb8940b0920 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.418+0000 7fb8990ad640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8940cdf60 con 0x7fb8940cef90 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.418+0000 7fb89359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb89415d4f0 0x7fb89406e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.418+0000 7fb89359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb89415d4f0 0x7fb89406e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39512/0 (socket says 172.21.15.16:39512) 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.418+0000 7fb89359e640 1 -- 172.21.15.16:0/1783449623 learned_addr learned my addr 172.21.15.16:0/1783449623 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.419+0000 7fb89259c640 1 --2- 172.21.15.16:0/1783449623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8940cef90 0x7fb8941f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.419+0000 7fb892d9d640 1 --2- 172.21.15.16:0/1783449623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8940b0920 0x7fb8940cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.419+0000 7fb89359e640 1 -- 172.21.15.16:0/1783449623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8940cef90 msgr2=0x7fb8941f6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.419+0000 7fb89359e640 1 --2- 172.21.15.16:0/1783449623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8940cef90 0x7fb8941f6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.419+0000 7fb89359e640 1 -- 172.21.15.16:0/1783449623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8940b0920 msgr2=0x7fb8940cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.419+0000 7fb89359e640 1 --2- 172.21.15.16:0/1783449623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8940b0920 0x7fb8940cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.419+0000 7fb89359e640 1 -- 172.21.15.16:0/1783449623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c009400 con 0x7fb89415d4f0 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.419+0000 7fb89359e640 1 --2- 172.21.15.16:0/1783449623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb89415d4f0 0x7fb89406e600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0097a0 tx=0x7fb88c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.419+0000 7fb87b7fe640 1 -- 172.21.15.16:0/1783449623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c004230 con 0x7fb89415d4f0 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.419+0000 7fb87b7fe640 1 -- 172.21.15.16:0/1783449623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c0043b0 con 0x7fb89415d4f0 2023-12-08T08:38:32.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.419+0000 7fb8990ad640 1 -- 172.21.15.16:0/1783449623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8940ce240 con 0x7fb89415d4f0 2023-12-08T08:38:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.419+0000 7fb87b7fe640 1 -- 172.21.15.16:0/1783449623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c004520 con 0x7fb89415d4f0 2023-12-08T08:38:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.419+0000 7fb8990ad640 1 -- 172.21.15.16:0/1783449623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8940c6730 con 0x7fb89415d4f0 2023-12-08T08:38:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.420+0000 7fb8990ad640 1 -- 172.21.15.16:0/1783449623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb894067430 con 0x7fb89415d4f0 2023-12-08T08:38:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.421+0000 7fb87b7fe640 1 -- 172.21.15.16:0/1783449623 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb88c03a960 con 0x7fb89415d4f0 2023-12-08T08:38:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.421+0000 7fb87b7fe640 1 --2- 172.21.15.16:0/1783449623 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb88006ae40 0x7fb88006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.421+0000 7fb87b7fe640 1 -- 172.21.15.16:0/1783449623 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb88c0abe50 con 0x7fb89415d4f0 2023-12-08T08:38:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.421+0000 7fb892d9d640 1 --2- 172.21.15.16:0/1783449623 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb88006ae40 0x7fb88006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.422+0000 7fb892d9d640 1 --2- 172.21.15.16:0/1783449623 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb88006ae40 0x7fb88006d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb88400f500 tx=0x7fb884007770 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:32.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.426+0000 7fb87b7fe640 1 -- 172.21.15.16:0/1783449623 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb88c076e70 con 0x7fb89415d4f0 2023-12-08T08:38:32.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.601+0000 7fb8990ad640 1 -- 172.21.15.16:0/1783449623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb89406bdd0 con 0x7fb89415d4f0 2023-12-08T08:38:32.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.602+0000 7fb87b7fe640 1 -- 172.21.15.16:0/1783449623 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb88c07a490 con 0x7fb89415d4f0 2023-12-08T08:38:32.604 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:38:32.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.604+0000 7fb8990ad640 1 -- 172.21.15.16:0/1783449623 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb88006ae40 msgr2=0x7fb88006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.604+0000 7fb8990ad640 1 --2- 172.21.15.16:0/1783449623 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb88006ae40 0x7fb88006d300 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb88400f500 tx=0x7fb884007770 comp rx=0 tx=0).stop 2023-12-08T08:38:32.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.604+0000 7fb8990ad640 1 -- 172.21.15.16:0/1783449623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb89415d4f0 msgr2=0x7fb89406e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.604+0000 7fb8990ad640 1 --2- 172.21.15.16:0/1783449623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb89415d4f0 0x7fb89406e600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0097a0 tx=0x7fb88c0046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.604+0000 7fb8990ad640 1 -- 172.21.15.16:0/1783449623 shutdown_connections 2023-12-08T08:38:32.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.604+0000 7fb8990ad640 1 --2- 172.21.15.16:0/1783449623 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb88006ae40 0x7fb88006d300 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.604+0000 7fb8990ad640 1 --2- 172.21.15.16:0/1783449623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb89415d4f0 0x7fb89406e600 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.604+0000 7fb8990ad640 1 --2- 172.21.15.16:0/1783449623 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb8940cef90 0x7fb8941f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.604+0000 7fb8990ad640 1 --2- 172.21.15.16:0/1783449623 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb8940b0920 0x7fb8940cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.604+0000 7fb8990ad640 1 -- 172.21.15.16:0/1783449623 >> 172.21.15.16:0/1783449623 conn(0x7fb8940d0e00 msgr2=0x7fb89406faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.604+0000 7fb8990ad640 1 -- 172.21.15.16:0/1783449623 shutdown_connections 2023-12-08T08:38:32.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.604+0000 7fb8990ad640 1 -- 172.21.15.16:0/1783449623 wait complete. 2023-12-08T08:38:32.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-08T08:38:32.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.723+0000 7ff954539640 1 Processor -- start 2023-12-08T08:38:32.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.723+0000 7ff954539640 1 -- start start 2023-12-08T08:38:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff954539640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c0cfb30 0x7ff94c0cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff954539640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff94c0cc1f0 0x7ff94c0cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff954539640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff94c0d02b0 con 0x7ff94c0b6ac0 2023-12-08T08:38:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff954539640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94c160cb0 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff954539640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c0cb430 con 0x7ff94c0cfb30 2023-12-08T08:38:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff9522d5640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff94c0b6ac0 0x7ff94c0cd660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52160/0 (socket says 172.21.15.16:52160) 2023-12-08T08:38:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff9522d5640 1 -- 172.21.15.16:0/3786718252 learned_addr learned my addr 172.21.15.16:0/3786718252 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:32.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff952ad6640 1 --2- 172.21.15.16:0/3786718252 >> v2:172.21.15.16:3300/0 conn(0x7ff94c0cc1f0 0x7ff94c0cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff951ad4640 1 --2- 172.21.15.16:0/3786718252 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c0cfb30 0x7ff94c0cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff952ad6640 1 -- 172.21.15.16:0/3786718252 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c0cfb30 msgr2=0x7ff94c0cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:32.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff952ad6640 1 --2- 172.21.15.16:0/3786718252 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c0cfb30 0x7ff94c0cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff952ad6640 1 -- 172.21.15.16:0/3786718252 >> v1:172.21.15.16:6789/0 conn(0x7ff94c0b6ac0 legacy=0x7ff94c0cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff952ad6640 1 -- 172.21.15.16:0/3786718252 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c0cb710 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff9512d3640 1 -- 172.21.15.16:0/3786718252 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118123737 0 0) 0x7ff94c0d02b0 con 0x7ff94c0b6ac0 2023-12-08T08:38:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.724+0000 7ff951ad4640 1 --2- 172.21.15.16:0/3786718252 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c0cfb30 0x7ff94c0cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.725+0000 7ff952ad6640 1 --2- 172.21.15.16:0/3786718252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c0cc1f0 0x7ff94c0cc600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff9440097a0 tx=0x7ff94402b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f699d61d010c8a78 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.725+0000 7ff9512d3640 1 -- 172.21.15.16:0/3786718252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94402f050 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.725+0000 7ff9512d3640 1 -- 172.21.15.16:0/3786718252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff944031700 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.725+0000 7ff9512d3640 1 -- 172.21.15.16:0/3786718252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff944031a90 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.726+0000 7ff954539640 1 -- 172.21.15.16:0/3786718252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c0cc1f0 msgr2=0x7ff94c0cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.726+0000 7ff954539640 1 --2- 172.21.15.16:0/3786718252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c0cc1f0 0x7ff94c0cc600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff9440097a0 tx=0x7ff94402b7a0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.726+0000 7ff954539640 1 -- 172.21.15.16:0/3786718252 shutdown_connections 2023-12-08T08:38:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.726+0000 7ff954539640 1 --2- 172.21.15.16:0/3786718252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c0cc1f0 0x7ff94c0cc600 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.726+0000 7ff954539640 1 --2- 172.21.15.16:0/3786718252 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c0cfb30 0x7ff94c0cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.726+0000 7ff954539640 1 -- 172.21.15.16:0/3786718252 >> 172.21.15.16:0/3786718252 conn(0x7ff94c0d0de0 msgr2=0x7ff94c0d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.726+0000 7ff954539640 1 -- 172.21.15.16:0/3786718252 shutdown_connections 2023-12-08T08:38:32.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.726+0000 7ff954539640 1 -- 172.21.15.16:0/3786718252 wait complete. 2023-12-08T08:38:32.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.727+0000 7ff954539640 1 Processor -- start 2023-12-08T08:38:32.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.727+0000 7ff954539640 1 -- start start 2023-12-08T08:38:32.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff954539640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c0b6ac0 0x7ff94c154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff954539640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c0cc1f0 0x7ff94c1f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff954539640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c0cfb30 0x7ff94c09cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff954539640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c09d290 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff954539640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c09d4f0 con 0x7ff94c0cfb30 2023-12-08T08:38:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff954539640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c09ac20 con 0x7ff94c0b6ac0 2023-12-08T08:38:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff951ad4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c0cc1f0 0x7ff94c1f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff951ad4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c0cc1f0 0x7ff94c1f37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39520/0 (socket says 172.21.15.16:39520) 2023-12-08T08:38:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff951ad4640 1 -- 172.21.15.16:0/1815073556 learned_addr learned my addr 172.21.15.16:0/1815073556 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff9522d5640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c0b6ac0 0x7ff94c154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff952ad6640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c0cfb30 0x7ff94c09cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff951ad4640 1 -- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c0b6ac0 msgr2=0x7ff94c154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff951ad4640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c0b6ac0 0x7ff94c154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff951ad4640 1 -- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c0cfb30 msgr2=0x7ff94c09cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff951ad4640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c0cfb30 0x7ff94c09cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff951ad4640 1 -- 172.21.15.16:0/1815073556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff944009400 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.728+0000 7ff9522d5640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c0b6ac0 0x7ff94c154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.729+0000 7ff951ad4640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c0cc1f0 0x7ff94c1f37b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff94000c7f0 tx=0x7ff94000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.729+0000 7ff94affd640 1 -- 172.21.15.16:0/1815073556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940007a30 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.729+0000 7ff94affd640 1 -- 172.21.15.16:0/1815073556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff940007b90 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.729+0000 7ff954539640 1 -- 172.21.15.16:0/1815073556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94c09af00 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.729+0000 7ff94affd640 1 -- 172.21.15.16:0/1815073556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94000ce40 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.729+0000 7ff954539640 1 -- 172.21.15.16:0/1815073556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94c09d7d0 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.730+0000 7ff954539640 1 -- 172.21.15.16:0/1815073556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff94c067430 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.730+0000 7ff94affd640 1 -- 172.21.15.16:0/1815073556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff9400113f0 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.731+0000 7ff94affd640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff93806ae70 0x7ff93806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:32.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.731+0000 7ff94affd640 1 -- 172.21.15.16:0/1815073556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff940089ae0 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.731+0000 7ff9522d5640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff93806ae70 0x7ff93806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:32.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.731+0000 7ff9522d5640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff93806ae70 0x7ff93806d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0012e0 tx=0x7ff93c005bb0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:32.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.735+0000 7ff94affd640 1 -- 172.21.15.16:0/1815073556 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff940054890 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.925+0000 7ff954539640 1 -- 172.21.15.16:0/1815073556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff94c0b1e50 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.925+0000 7ff94affd640 1 -- 172.21.15.16:0/1815073556 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff94001f080 con 0x7ff94c0cc1f0 2023-12-08T08:38:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-12-08T08:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.927+0000 7ff954539640 1 -- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff93806ae70 msgr2=0x7ff93806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.927+0000 7ff954539640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff93806ae70 0x7ff93806d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0012e0 tx=0x7ff93c005bb0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.927+0000 7ff954539640 1 -- 172.21.15.16:0/1815073556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c0cc1f0 msgr2=0x7ff94c1f37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.927+0000 7ff954539640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c0cc1f0 0x7ff94c1f37b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff94000c7f0 tx=0x7ff94000cbf0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.927+0000 7ff954539640 1 -- 172.21.15.16:0/1815073556 shutdown_connections 2023-12-08T08:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.927+0000 7ff954539640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff93806ae70 0x7ff93806d330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.927+0000 7ff954539640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff94c0cfb30 0x7ff94c09cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.927+0000 7ff954539640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff94c0cc1f0 0x7ff94c1f37b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.927+0000 7ff954539640 1 --2- 172.21.15.16:0/1815073556 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff94c0b6ac0 0x7ff94c154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.928+0000 7ff954539640 1 -- 172.21.15.16:0/1815073556 >> 172.21.15.16:0/1815073556 conn(0x7ff94c0d0de0 msgr2=0x7ff94c06f770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.928+0000 7ff954539640 1 -- 172.21.15.16:0/1815073556 shutdown_connections 2023-12-08T08:38:32.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:32.928+0000 7ff954539640 1 -- 172.21.15.16:0/1815073556 wait complete. 2023-12-08T08:38:32.940 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-12-08T08:38:32.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:32.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-12-08T08:38:32.969 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi016.stderr:ceph-fuse[95582]: fuse finished with error 0 and tester_r 0 2023-12-08T08:38:32.970 INFO:teuthology.orchestra.run:waiting for 300 2023-12-08T08:38:39.072 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-12-08T08:38:39.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:39.073 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-12-08T08:38:39.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.257+0000 7fe5354c9640 1 Processor -- start 2023-12-08T08:38:39.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.257+0000 7fe5354c9640 1 -- start start 2023-12-08T08:38:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.258+0000 7fe5354c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5300cd240 0x7fe5300cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.258+0000 7fe5354c9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe5300cf730 0x7fe53015edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.258+0000 7fe5354c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe53015f310 0x7fe53015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.258+0000 7fe5354c9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5300cd620 con 0x7fe53015f310 2023-12-08T08:38:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.258+0000 7fe5354c9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5300cfb90 con 0x7fe5300cf730 2023-12-08T08:38:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.258+0000 7fe5354c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53015c150 con 0x7fe5300cd240 2023-12-08T08:38:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.258+0000 7fe52f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe53015f310 0x7fe53015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.258+0000 7fe52f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe53015f310 0x7fe53015a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52220/0 (socket says 172.21.15.16:52220) 2023-12-08T08:38:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.258+0000 7fe52f7fe640 1 -- 172.21.15.16:0/3714473851 learned_addr learned my addr 172.21.15.16:0/3714473851 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.258+0000 7fe52e7fc640 1 --2- 172.21.15.16:0/3714473851 >> v2:172.21.15.112:3300/0 conn(0x7fe5300cf730 0x7fe53015edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.258+0000 7fe52effd640 1 --2- 172.21.15.16:0/3714473851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5300cd240 0x7fe5300cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.259+0000 7fe52f7fe640 1 -- 172.21.15.16:0/3714473851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5300cd240 msgr2=0x7fe5300cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.259+0000 7fe52f7fe640 1 --2- 172.21.15.16:0/3714473851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5300cd240 0x7fe5300cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.259+0000 7fe52f7fe640 1 -- 172.21.15.16:0/3714473851 >> v2:172.21.15.112:3300/0 conn(0x7fe5300cf730 msgr2=0x7fe53015edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.259+0000 7fe52f7fe640 1 --2- 172.21.15.16:0/3714473851 >> v2:172.21.15.112:3300/0 conn(0x7fe5300cf730 0x7fe53015edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.259+0000 7fe52f7fe640 1 -- 172.21.15.16:0/3714473851 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe53015fb00 con 0x7fe53015f310 2023-12-08T08:38:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.259+0000 7fe52f7fe640 1 --2- 172.21.15.16:0/3714473851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe53015f310 0x7fe53015a9f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe5300b3930 tx=0x7fe52002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa98c2a97c006a0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.260+0000 7fe52dffb640 1 -- 172.21.15.16:0/3714473851 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52002ca70 con 0x7fe53015f310 2023-12-08T08:38:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.260+0000 7fe52dffb640 1 -- 172.21.15.16:0/3714473851 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52002cbd0 con 0x7fe53015f310 2023-12-08T08:38:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.260+0000 7fe52dffb640 1 -- 172.21.15.16:0/3714473851 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5200343f0 con 0x7fe53015f310 2023-12-08T08:38:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.260+0000 7fe5354c9640 1 -- 172.21.15.16:0/3714473851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe53015f310 msgr2=0x7fe53015a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.260+0000 7fe5354c9640 1 --2- 172.21.15.16:0/3714473851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe53015f310 0x7fe53015a9f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe5300b3930 tx=0x7fe52002bed0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.260+0000 7fe5354c9640 1 -- 172.21.15.16:0/3714473851 shutdown_connections 2023-12-08T08:38:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.260+0000 7fe5354c9640 1 --2- 172.21.15.16:0/3714473851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe53015f310 0x7fe53015a9f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.260+0000 7fe5354c9640 1 --2- 172.21.15.16:0/3714473851 >> v2:172.21.15.112:3300/0 conn(0x7fe5300cf730 0x7fe53015edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.260+0000 7fe5354c9640 1 --2- 172.21.15.16:0/3714473851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5300cd240 0x7fe5300cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.260+0000 7fe5354c9640 1 -- 172.21.15.16:0/3714473851 >> 172.21.15.16:0/3714473851 conn(0x7fe530099a10 msgr2=0x7fe530099e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.260+0000 7fe5354c9640 1 -- 172.21.15.16:0/3714473851 shutdown_connections 2023-12-08T08:38:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.260+0000 7fe5354c9640 1 -- 172.21.15.16:0/3714473851 wait complete. 2023-12-08T08:38:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.262+0000 7fe5354c9640 1 Processor -- start 2023-12-08T08:38:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.262+0000 7fe5354c9640 1 -- start start 2023-12-08T08:38:39.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.262+0000 7fe5354c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5300cd240 0x7fe5300c7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.262+0000 7fe5354c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5300cf730 0x7fe5300b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.262+0000 7fe5354c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe53015f310 0x7fe5300b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.262+0000 7fe5354c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53014fb50 con 0x7fe5300cd240 2023-12-08T08:38:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.262+0000 7fe5354c9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe530157070 con 0x7fe53015f310 2023-12-08T08:38:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.262+0000 7fe5354c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53014ef40 con 0x7fe5300cf730 2023-12-08T08:38:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.262+0000 7fe52effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5300cd240 0x7fe5300c7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.262+0000 7fe52effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5300cd240 0x7fe5300c7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52234/0 (socket says 172.21.15.16:52234) 2023-12-08T08:38:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.262+0000 7fe52effd640 1 -- 172.21.15.16:0/4182137552 learned_addr learned my addr 172.21.15.16:0/4182137552 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.263+0000 7fe52f7fe640 1 --2- 172.21.15.16:0/4182137552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe53015f310 0x7fe5300b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.263+0000 7fe52e7fc640 1 --2- 172.21.15.16:0/4182137552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5300cf730 0x7fe5300b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.263+0000 7fe52effd640 1 -- 172.21.15.16:0/4182137552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5300cf730 msgr2=0x7fe5300b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.263+0000 7fe52effd640 1 --2- 172.21.15.16:0/4182137552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5300cf730 0x7fe5300b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.263+0000 7fe52effd640 1 -- 172.21.15.16:0/4182137552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe53015f310 msgr2=0x7fe5300b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.263+0000 7fe52effd640 1 --2- 172.21.15.16:0/4182137552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe53015f310 0x7fe5300b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.263+0000 7fe52effd640 1 -- 172.21.15.16:0/4182137552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe520009b30 con 0x7fe5300cd240 2023-12-08T08:38:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.263+0000 7fe52effd640 1 --2- 172.21.15.16:0/4182137552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5300cd240 0x7fe5300c7140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe52800d800 tx=0x7fe52800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.263+0000 7fe5277fe640 1 -- 172.21.15.16:0/4182137552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5280096a0 con 0x7fe5300cd240 2023-12-08T08:38:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.263+0000 7fe5277fe640 1 -- 172.21.15.16:0/4182137552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe528009800 con 0x7fe5300cd240 2023-12-08T08:38:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.263+0000 7fe5354c9640 1 -- 172.21.15.16:0/4182137552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53014f220 con 0x7fe5300cd240 2023-12-08T08:38:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.263+0000 7fe5277fe640 1 -- 172.21.15.16:0/4182137552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe528009990 con 0x7fe5300cd240 2023-12-08T08:38:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.263+0000 7fe5354c9640 1 -- 172.21.15.16:0/4182137552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53014f7e0 con 0x7fe5300cd240 2023-12-08T08:38:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.264+0000 7fe5354c9640 1 -- 172.21.15.16:0/4182137552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe53014f9a0 con 0x7fe5300cd240 2023-12-08T08:38:39.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.265+0000 7fe5277fe640 1 -- 172.21.15.16:0/4182137552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe528009af0 con 0x7fe5300cd240 2023-12-08T08:38:39.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.265+0000 7fe5277fe640 1 --2- 172.21.15.16:0/4182137552 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe51006ae70 0x7fe51006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.265+0000 7fe5277fe640 1 -- 172.21.15.16:0/4182137552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe528089bc0 con 0x7fe5300cd240 2023-12-08T08:38:39.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.265+0000 7fe52e7fc640 1 --2- 172.21.15.16:0/4182137552 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe51006ae70 0x7fe51006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.266+0000 7fe52e7fc640 1 --2- 172.21.15.16:0/4182137552 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe51006ae70 0x7fe51006d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe5300cb8b0 tx=0x7fe518008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:39.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.270+0000 7fe5277fe640 1 -- 172.21.15.16:0/4182137552 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe528054a30 con 0x7fe5300cd240 2023-12-08T08:38:39.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.446+0000 7fe5354c9640 1 -- 172.21.15.16:0/4182137552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe530067ad0 con 0x7fe5300cd240 2023-12-08T08:38:39.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.447+0000 7fe5277fe640 1 -- 172.21.15.16:0/4182137552 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fe528058050 con 0x7fe5300cd240 2023-12-08T08:38:39.450 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:39.451 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:24.030625+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.449+0000 7fe5354c9640 1 -- 172.21.15.16:0/4182137552 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe51006ae70 msgr2=0x7fe51006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.449+0000 7fe5354c9640 1 --2- 172.21.15.16:0/4182137552 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe51006ae70 0x7fe51006d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe5300cb8b0 tx=0x7fe518008040 comp rx=0 tx=0).stop 2023-12-08T08:38:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.449+0000 7fe5354c9640 1 -- 172.21.15.16:0/4182137552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5300cd240 msgr2=0x7fe5300c7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.449+0000 7fe5354c9640 1 --2- 172.21.15.16:0/4182137552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5300cd240 0x7fe5300c7140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe52800d800 tx=0x7fe52800dcc0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.449+0000 7fe5354c9640 1 -- 172.21.15.16:0/4182137552 shutdown_connections 2023-12-08T08:38:39.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.449+0000 7fe5354c9640 1 --2- 172.21.15.16:0/4182137552 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe51006ae70 0x7fe51006d330 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.449+0000 7fe5354c9640 1 --2- 172.21.15.16:0/4182137552 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe53015f310 0x7fe5300b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.449+0000 7fe5354c9640 1 --2- 172.21.15.16:0/4182137552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe5300cf730 0x7fe5300b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.449+0000 7fe5354c9640 1 --2- 172.21.15.16:0/4182137552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5300cd240 0x7fe5300c7140 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.449+0000 7fe5354c9640 1 -- 172.21.15.16:0/4182137552 >> 172.21.15.16:0/4182137552 conn(0x7fe530099a10 msgr2=0x7fe5300cbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:39.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.449+0000 7fe5354c9640 1 -- 172.21.15.16:0/4182137552 shutdown_connections 2023-12-08T08:38:39.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.449+0000 7fe5354c9640 1 -- 172.21.15.16:0/4182137552 wait complete. 2023-12-08T08:38:39.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:39.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.567+0000 7f096d8e7640 1 Processor -- start 2023-12-08T08:38:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.567+0000 7f096d8e7640 1 -- start start 2023-12-08T08:38:39.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.568+0000 7f096d8e7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f096806e330 0x7f096806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.568+0000 7f096d8e7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09680d0ce0 0x7f09680d1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.568+0000 7f096d8e7640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09680d1690 con 0x7f096815d490 2023-12-08T08:38:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.568+0000 7f096d8e7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09680ce270 con 0x7f096806e330 2023-12-08T08:38:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.568+0000 7f096d8e7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09680ce550 con 0x7f09680d0ce0 2023-12-08T08:38:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.569+0000 7f09667fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f096806e330 0x7f096806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.569+0000 7f09677fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09680d0ce0 0x7f09680d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.569+0000 7f09667fc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f096806e330 0x7f096806e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:45768/0 (socket says 172.21.15.16:45768) 2023-12-08T08:38:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.569+0000 7f0966ffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f096815d490 0x7f096806c400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:33888/0 (socket says 172.21.15.16:33888) 2023-12-08T08:38:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.569+0000 7f09667fc640 1 -- 172.21.15.16:0/4009062201 learned_addr learned my addr 172.21.15.16:0/4009062201 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.569+0000 7f09667fc640 1 -- 172.21.15.16:0/4009062201 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09680d0ce0 msgr2=0x7f09680d1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.569+0000 7f09667fc640 1 --2- 172.21.15.16:0/4009062201 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09680d0ce0 0x7f09680d1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.569+0000 7f09667fc640 1 -- 172.21.15.16:0/4009062201 >> v1:172.21.15.112:6789/0 conn(0x7f096815d490 legacy=0x7f096806c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.569+0000 7f09667fc640 1 -- 172.21.15.16:0/4009062201 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096806b920 con 0x7f096806e330 2023-12-08T08:38:39.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.570+0000 7f09667fc640 1 --2- 172.21.15.16:0/4009062201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f096806e330 0x7f096806e7c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0950009870 tx=0x7f095002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=870b6554de27ddfd server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.570+0000 7f0965ffb640 1 -- 172.21.15.16:0/4009062201 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950030840 con 0x7f096806e330 2023-12-08T08:38:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.570+0000 7f0965ffb640 1 -- 172.21.15.16:0/4009062201 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09500309a0 con 0x7f096806e330 2023-12-08T08:38:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.570+0000 7f0965ffb640 1 -- 172.21.15.16:0/4009062201 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950030d30 con 0x7f096806e330 2023-12-08T08:38:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.570+0000 7f096d8e7640 1 -- 172.21.15.16:0/4009062201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f096806e330 msgr2=0x7f096806e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.570+0000 7f096d8e7640 1 --2- 172.21.15.16:0/4009062201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f096806e330 0x7f096806e7c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0950009870 tx=0x7f095002b980 comp rx=0 tx=0).stop 2023-12-08T08:38:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.570+0000 7f096d8e7640 1 -- 172.21.15.16:0/4009062201 shutdown_connections 2023-12-08T08:38:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.570+0000 7f096d8e7640 1 --2- 172.21.15.16:0/4009062201 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09680d0ce0 0x7f09680d1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.570+0000 7f096d8e7640 1 --2- 172.21.15.16:0/4009062201 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f096806e330 0x7f096806e7c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.570+0000 7f096d8e7640 1 -- 172.21.15.16:0/4009062201 >> 172.21.15.16:0/4009062201 conn(0x7f0968099a10 msgr2=0x7f0968099e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.570+0000 7f096d8e7640 1 -- 172.21.15.16:0/4009062201 shutdown_connections 2023-12-08T08:38:39.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.570+0000 7f096d8e7640 1 -- 172.21.15.16:0/4009062201 wait complete. 2023-12-08T08:38:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.571+0000 7f096d8e7640 1 Processor -- start 2023-12-08T08:38:39.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.572+0000 7f096d8e7640 1 -- start start 2023-12-08T08:38:39.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.572+0000 7f096d8e7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f096806e330 0x7f09680adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.572+0000 7f096d8e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09680d0ce0 0x7f096815c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.572+0000 7f096d8e7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09680afb50 0x7f096814c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.572+0000 7f096d8e7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09680c9b70 con 0x7f09680d0ce0 2023-12-08T08:38:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.572+0000 7f096d8e7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09681566f0 con 0x7f096806e330 2023-12-08T08:38:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.572+0000 7f096d8e7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096815cca0 con 0x7f09680afb50 2023-12-08T08:38:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f09667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09680d0ce0 0x7f096815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f09667fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09680d0ce0 0x7f096815c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52248/0 (socket says 172.21.15.16:52248) 2023-12-08T08:38:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f09667fc640 1 -- 172.21.15.16:0/3763366808 learned_addr learned my addr 172.21.15.16:0/3763366808 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f0966ffd640 1 --2- 172.21.15.16:0/3763366808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f096806e330 0x7f09680adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f09677fe640 1 --2- 172.21.15.16:0/3763366808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09680afb50 0x7f096814c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f09667fc640 1 -- 172.21.15.16:0/3763366808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09680afb50 msgr2=0x7f096814c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f09667fc640 1 --2- 172.21.15.16:0/3763366808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09680afb50 0x7f096814c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f09667fc640 1 -- 172.21.15.16:0/3763366808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f096806e330 msgr2=0x7f09680adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f09667fc640 1 --2- 172.21.15.16:0/3763366808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f096806e330 0x7f09680adb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f09667fc640 1 -- 172.21.15.16:0/3763366808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500094d0 con 0x7f09680d0ce0 2023-12-08T08:38:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f09667fc640 1 --2- 172.21.15.16:0/3763366808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09680d0ce0 0x7f096815c760 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0950000c00 tx=0x7f0950004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f095ffff640 1 -- 172.21.15.16:0/3763366808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950039c90 con 0x7f09680d0ce0 2023-12-08T08:38:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f095ffff640 1 -- 172.21.15.16:0/3763366808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0950039df0 con 0x7f09680d0ce0 2023-12-08T08:38:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f096d8e7640 1 -- 172.21.15.16:0/3763366808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09680c97d0 con 0x7f09680d0ce0 2023-12-08T08:38:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f095ffff640 1 -- 172.21.15.16:0/3763366808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950043640 con 0x7f09680d0ce0 2023-12-08T08:38:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.573+0000 7f096d8e7640 1 -- 172.21.15.16:0/3763366808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09680b00c0 con 0x7f09680d0ce0 2023-12-08T08:38:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.574+0000 7f096d8e7640 1 -- 172.21.15.16:0/3763366808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09680c9cb0 con 0x7f09680d0ce0 2023-12-08T08:38:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.575+0000 7f095ffff640 1 -- 172.21.15.16:0/3763366808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f09500437a0 con 0x7f09680d0ce0 2023-12-08T08:38:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.575+0000 7f095ffff640 1 --2- 172.21.15.16:0/3763366808 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f094806ae40 0x7f094806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.575+0000 7f095ffff640 1 -- 172.21.15.16:0/3763366808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f095003e070 con 0x7f09680d0ce0 2023-12-08T08:38:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.575+0000 7f0966ffd640 1 --2- 172.21.15.16:0/3763366808 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f094806ae40 0x7f094806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.576+0000 7f0966ffd640 1 --2- 172.21.15.16:0/3763366808 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f094806ae40 0x7f094806d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0960000f60 tx=0x7f0960000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.579+0000 7f095ffff640 1 -- 172.21.15.16:0/3763366808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f09680c9cb0 con 0x7f09680d0ce0 2023-12-08T08:38:39.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.759+0000 7f096d8e7640 1 -- 172.21.15.16:0/3763366808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09680c9cb0 con 0x7f09680d0ce0 2023-12-08T08:38:39.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.760+0000 7f095ffff640 1 -- 172.21.15.16:0/3763366808 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f09680c9cb0 con 0x7f09680d0ce0 2023-12-08T08:38:39.763 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:39.763 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:24.030625+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.762+0000 7f096d8e7640 1 -- 172.21.15.16:0/3763366808 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f094806ae40 msgr2=0x7f094806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.762+0000 7f096d8e7640 1 --2- 172.21.15.16:0/3763366808 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f094806ae40 0x7f094806d300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0960000f60 tx=0x7f0960000ed0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.762+0000 7f096d8e7640 1 -- 172.21.15.16:0/3763366808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09680d0ce0 msgr2=0x7f096815c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.762+0000 7f096d8e7640 1 --2- 172.21.15.16:0/3763366808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09680d0ce0 0x7f096815c760 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0950000c00 tx=0x7f0950004730 comp rx=0 tx=0).stop 2023-12-08T08:38:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.762+0000 7f096d8e7640 1 -- 172.21.15.16:0/3763366808 shutdown_connections 2023-12-08T08:38:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.762+0000 7f096d8e7640 1 --2- 172.21.15.16:0/3763366808 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f094806ae40 0x7f094806d300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.762+0000 7f096d8e7640 1 --2- 172.21.15.16:0/3763366808 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f09680afb50 0x7f096814c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.762+0000 7f096d8e7640 1 --2- 172.21.15.16:0/3763366808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f09680d0ce0 0x7f096815c760 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.762+0000 7f096d8e7640 1 --2- 172.21.15.16:0/3763366808 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f096806e330 0x7f09680adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.762+0000 7f096d8e7640 1 -- 172.21.15.16:0/3763366808 >> 172.21.15.16:0/3763366808 conn(0x7f0968099a10 msgr2=0x7f09680cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.762+0000 7f096d8e7640 1 -- 172.21.15.16:0/3763366808 shutdown_connections 2023-12-08T08:38:39.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.762+0000 7f096d8e7640 1 -- 172.21.15.16:0/3763366808 wait complete. 2023-12-08T08:38:39.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-08T08:38:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.883+0000 7fb7b9953640 1 Processor -- start 2023-12-08T08:38:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.884+0000 7fb7b9953640 1 -- start start 2023-12-08T08:38:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.884+0000 7fb7b9953640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7b40cfa30 0x7fb7b40b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.884+0000 7fb7b9953640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7b40b4150 0x7fb7b40b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.884+0000 7fb7b9953640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b40c6be0 con 0x7fb7b40cfe10 2023-12-08T08:38:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.884+0000 7fb7b9953640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b40d3ff0 con 0x7fb7b40b4150 2023-12-08T08:38:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.884+0000 7fb7b9953640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40b4860 con 0x7fb7b40cfa30 2023-12-08T08:38:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.884+0000 7fb7b27fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7b40b4150 0x7fb7b40b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.885+0000 7fb7b27fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7b40b4150 0x7fb7b40b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52264/0 (socket says 172.21.15.16:52264) 2023-12-08T08:38:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.885+0000 7fb7b27fc640 1 -- 172.21.15.16:0/890457368 learned_addr learned my addr 172.21.15.16:0/890457368 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.885+0000 7fb7b2ffd640 1 --2- 172.21.15.16:0/890457368 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7b40cfa30 0x7fb7b40b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.885+0000 7fb7b27fc640 1 -- 172.21.15.16:0/890457368 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7b40cfa30 msgr2=0x7fb7b40b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.885+0000 7fb7b27fc640 1 --2- 172.21.15.16:0/890457368 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7b40cfa30 0x7fb7b40b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.885+0000 7fb7b27fc640 1 -- 172.21.15.16:0/890457368 >> v1:172.21.15.112:6789/0 conn(0x7fb7b40cfe10 legacy=0x7fb7b4161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.885+0000 7fb7b27fc640 1 -- 172.21.15.16:0/890457368 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b40d3d50 con 0x7fb7b40b4150 2023-12-08T08:38:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.886+0000 7fb7b27fc640 1 --2- 172.21.15.16:0/890457368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40b4150 0x7fb7b40b3460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009870 tx=0x7fb7a802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fbccd4429c5330 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.886+0000 7fb7b1ffb640 1 -- 172.21.15.16:0/890457368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8031640 con 0x7fb7b40b4150 2023-12-08T08:38:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.886+0000 7fb7b1ffb640 1 -- 172.21.15.16:0/890457368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a80317a0 con 0x7fb7b40b4150 2023-12-08T08:38:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.886+0000 7fb7b1ffb640 1 -- 172.21.15.16:0/890457368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8031b30 con 0x7fb7b40b4150 2023-12-08T08:38:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.886+0000 7fb7b9953640 1 -- 172.21.15.16:0/890457368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40b4150 msgr2=0x7fb7b40b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.886+0000 7fb7b9953640 1 --2- 172.21.15.16:0/890457368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40b4150 0x7fb7b40b3460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009870 tx=0x7fb7a802b8f0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.886+0000 7fb7b9953640 1 -- 172.21.15.16:0/890457368 shutdown_connections 2023-12-08T08:38:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.886+0000 7fb7b9953640 1 --2- 172.21.15.16:0/890457368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40b4150 0x7fb7b40b3460 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.886+0000 7fb7b9953640 1 --2- 172.21.15.16:0/890457368 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7b40cfa30 0x7fb7b40b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.886+0000 7fb7b9953640 1 -- 172.21.15.16:0/890457368 >> 172.21.15.16:0/890457368 conn(0x7fb7b40d3090 msgr2=0x7fb7b40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.886+0000 7fb7b9953640 1 -- 172.21.15.16:0/890457368 shutdown_connections 2023-12-08T08:38:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.886+0000 7fb7b9953640 1 -- 172.21.15.16:0/890457368 wait complete. 2023-12-08T08:38:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.887+0000 7fb7b9953640 1 Processor -- start 2023-12-08T08:38:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.888+0000 7fb7b9953640 1 -- start start 2023-12-08T08:38:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.888+0000 7fb7b9953640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b40b4150 0x7fb7b41526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.888+0000 7fb7b9953640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cfa30 0x7fb7b40b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.888+0000 7fb7b9953640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7b40cfe10 0x7fb7b40aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.888+0000 7fb7b9953640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40cd210 con 0x7fb7b40cfa30 2023-12-08T08:38:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.888+0000 7fb7b9953640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40b0460 con 0x7fb7b40b4150 2023-12-08T08:38:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.888+0000 7fb7b9953640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b40cc870 con 0x7fb7b40cfe10 2023-12-08T08:38:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.888+0000 7fb7b27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cfa30 0x7fb7b40b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb7b27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cfa30 0x7fb7b40b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52270/0 (socket says 172.21.15.16:52270) 2023-12-08T08:38:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb7b27fc640 1 -- 172.21.15.16:0/2691442703 learned_addr learned my addr 172.21.15.16:0/2691442703 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb7b2ffd640 1 --2- 172.21.15.16:0/2691442703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b40b4150 0x7fb7b41526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb7b37fe640 1 --2- 172.21.15.16:0/2691442703 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7b40cfe10 0x7fb7b40aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb7b27fc640 1 -- 172.21.15.16:0/2691442703 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7b40cfe10 msgr2=0x7fb7b40aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb7b27fc640 1 --2- 172.21.15.16:0/2691442703 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7b40cfe10 0x7fb7b40aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb7b27fc640 1 -- 172.21.15.16:0/2691442703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b40b4150 msgr2=0x7fb7b41526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb7b27fc640 1 --2- 172.21.15.16:0/2691442703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b40b4150 0x7fb7b41526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb7b27fc640 1 -- 172.21.15.16:0/2691442703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a80094d0 con 0x7fb7b40cfa30 2023-12-08T08:38:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb7b27fc640 1 --2- 172.21.15.16:0/2691442703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cfa30 0x7fb7b40b29b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8000c00 tx=0x7fb7a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb79bfff640 1 -- 172.21.15.16:0/2691442703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a803a930 con 0x7fb7b40cfa30 2023-12-08T08:38:39.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb79bfff640 1 -- 172.21.15.16:0/2691442703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a803aa90 con 0x7fb7b40cfa30 2023-12-08T08:38:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb7b9953640 1 -- 172.21.15.16:0/2691442703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b40ccb50 con 0x7fb7b40cfa30 2023-12-08T08:38:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb79bfff640 1 -- 172.21.15.16:0/2691442703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a803ae40 con 0x7fb7b40cfa30 2023-12-08T08:38:39.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.889+0000 7fb7b9953640 1 -- 172.21.15.16:0/2691442703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b40cbac0 con 0x7fb7b40cfa30 2023-12-08T08:38:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.890+0000 7fb7b9953640 1 -- 172.21.15.16:0/2691442703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b4067430 con 0x7fb7b40cfa30 2023-12-08T08:38:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.891+0000 7fb79bfff640 1 -- 172.21.15.16:0/2691442703 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb7a802f050 con 0x7fb7b40cfa30 2023-12-08T08:38:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.891+0000 7fb79bfff640 1 --2- 172.21.15.16:0/2691442703 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb7a006ae40 0x7fb7a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.891+0000 7fb79bfff640 1 -- 172.21.15.16:0/2691442703 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb7a80ab600 con 0x7fb7b40cfa30 2023-12-08T08:38:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.892+0000 7fb7b2ffd640 1 --2- 172.21.15.16:0/2691442703 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb7a006ae40 0x7fb7a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.892+0000 7fb7b2ffd640 1 --2- 172.21.15.16:0/2691442703 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb7a006ae40 0x7fb7a006d300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009970 tx=0x7fb7a4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:39.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:39.896+0000 7fb79bfff640 1 -- 172.21.15.16:0/2691442703 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb7a8076450 con 0x7fb7b40cfa30 2023-12-08T08:38:40.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.115+0000 7fb7b9953640 1 -- 172.21.15.16:0/2691442703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb7b40743e0 con 0x7fb7b40cfa30 2023-12-08T08:38:40.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:38:40.295 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:38:40.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.718+0000 7fb79bfff640 1 -- 172.21.15.16:0/2691442703 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb7a8079a70 con 0x7fb7b40cfa30 2023-12-08T08:38:40.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.721+0000 7fb7b9953640 1 -- 172.21.15.16:0/2691442703 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb7a006ae40 msgr2=0x7fb7a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:40.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.721+0000 7fb7b9953640 1 --2- 172.21.15.16:0/2691442703 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb7a006ae40 0x7fb7a006d300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb7a4009970 tx=0x7fb7a4008040 comp rx=0 tx=0).stop 2023-12-08T08:38:40.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.721+0000 7fb7b9953640 1 -- 172.21.15.16:0/2691442703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cfa30 msgr2=0x7fb7b40b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:40.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.721+0000 7fb7b9953640 1 --2- 172.21.15.16:0/2691442703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cfa30 0x7fb7b40b29b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8000c00 tx=0x7fb7a80046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:40.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.721+0000 7fb7b9953640 1 -- 172.21.15.16:0/2691442703 shutdown_connections 2023-12-08T08:38:40.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.721+0000 7fb7b9953640 1 --2- 172.21.15.16:0/2691442703 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb7a006ae40 0x7fb7a006d300 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:40.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.721+0000 7fb7b9953640 1 --2- 172.21.15.16:0/2691442703 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb7b40cfe10 0x7fb7b40aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:40.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.721+0000 7fb7b9953640 1 --2- 172.21.15.16:0/2691442703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7b40cfa30 0x7fb7b40b29b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:40.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.721+0000 7fb7b9953640 1 --2- 172.21.15.16:0/2691442703 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb7b40b4150 0x7fb7b41526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:40.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.721+0000 7fb7b9953640 1 -- 172.21.15.16:0/2691442703 >> 172.21.15.16:0/2691442703 conn(0x7fb7b40d3090 msgr2=0x7fb7b415c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:40.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.721+0000 7fb7b9953640 1 -- 172.21.15.16:0/2691442703 shutdown_connections 2023-12-08T08:38:40.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.721+0000 7fb7b9953640 1 -- 172.21.15.16:0/2691442703 wait complete. 2023-12-08T08:38:40.736 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-12-08T08:38:40.737 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-08T08:38:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.848+0000 7fef8ec7c640 1 Processor -- start 2023-12-08T08:38:40.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.848+0000 7fef8ec7c640 1 -- start start 2023-12-08T08:38:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.848+0000 7fef8ec7c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef880b8950 0x7fef880ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef8ec7c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fef880cfc80 0x7fef880b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef8ec7c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef880b5f00 con 0x7fef880cebc0 2023-12-08T08:38:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef8ec7c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef880b6170 con 0x7fef880cfc80 2023-12-08T08:38:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef8ec7c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef880b6450 con 0x7fef880b8950 2023-12-08T08:38:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef87fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fef880cebc0 0x7fef880cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59660/0 (socket says 172.21.15.16:59660) 2023-12-08T08:38:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef87fff640 1 -- 172.21.15.16:0/1963457246 learned_addr learned my addr 172.21.15.16:0/1963457246 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef8ca18640 1 --2- 172.21.15.16:0/1963457246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef880b8950 0x7fef880ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 -- 172.21.15.16:0/1963457246 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800751633 0 0) 0x7fef880b5f00 con 0x7fef880cebc0 2023-12-08T08:38:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 -- 172.21.15.16:0/1963457246 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef78003520 con 0x7fef880cebc0 2023-12-08T08:38:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef8d219640 1 --2- 172.21.15.16:0/1963457246 >> v2:172.21.15.112:3300/0 conn(0x7fef880cfc80 0x7fef880b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 -- 172.21.15.16:0/1963457246 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 82646188 0 0) 0x7fef78003520 con 0x7fef880cebc0 2023-12-08T08:38:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 -- 172.21.15.16:0/1963457246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef880b8950 msgr2=0x7fef880ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 --2- 172.21.15.16:0/1963457246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef880b8950 0x7fef880ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:40.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 -- 172.21.15.16:0/1963457246 >> v2:172.21.15.112:3300/0 conn(0x7fef880cfc80 msgr2=0x7fef880b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 --2- 172.21.15.16:0/1963457246 >> v2:172.21.15.112:3300/0 conn(0x7fef880cfc80 0x7fef880b6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 -- 172.21.15.16:0/1963457246 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef880b6730 con 0x7fef880cebc0 2023-12-08T08:38:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 -- 172.21.15.16:0/1963457246 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7fef74001670 con 0x7fef880cebc0 2023-12-08T08:38:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 -- 172.21.15.16:0/1963457246 >> v1:172.21.15.16:6789/0 conn(0x7fef880cebc0 legacy=0x7fef880cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 --2- 172.21.15.16:0/1963457246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef78003970 0x7fef78003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 --2- 172.21.15.16:0/1963457246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef780047c0 0x7fef78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 --2- 172.21.15.16:0/1963457246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef880b8950 0x7fef78005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 -- 172.21.15.16:0/1963457246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef880b6170 con 0x7fef78003970 2023-12-08T08:38:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 -- 172.21.15.16:0/1963457246 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef880b6450 con 0x7fef780047c0 2023-12-08T08:38:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef877fe640 1 -- 172.21.15.16:0/1963457246 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7800b980 con 0x7fef880b8950 2023-12-08T08:38:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.849+0000 7fef8ca18640 1 --2- 172.21.15.16:0/1963457246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef78003970 0x7fef78003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.850+0000 7fef87fff640 1 --2- 172.21.15.16:0/1963457246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef780047c0 0x7fef78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.850+0000 7fef8d219640 1 --2- 172.21.15.16:0/1963457246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef880b8950 0x7fef78005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:40.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.850+0000 7fef8ca18640 1 -- 172.21.15.16:0/1963457246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef880b8950 msgr2=0x7fef78005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.850+0000 7fef8ca18640 1 --2- 172.21.15.16:0/1963457246 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef880b8950 0x7fef78005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.850+0000 7fef8ca18640 1 -- 172.21.15.16:0/1963457246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef780047c0 msgr2=0x7fef78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.850+0000 7fef8ca18640 1 --2- 172.21.15.16:0/1963457246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef780047c0 0x7fef78004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.850+0000 7fef8ca18640 1 -- 172.21.15.16:0/1963457246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef78003520 con 0x7fef78003970 2023-12-08T08:38:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.850+0000 7fef87fff640 1 --2- 172.21.15.16:0/1963457246 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef780047c0 0x7fef78004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.851+0000 7fef8ca18640 1 --2- 172.21.15.16:0/1963457246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef78003970 0x7fef78003d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009af0 tx=0x7fef780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974c173fabe92e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.851+0000 7fef877fe640 1 -- 172.21.15.16:0/1963457246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c02c6b0 con 0x7fef78003970 2023-12-08T08:38:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.851+0000 7fef877fe640 1 -- 172.21.15.16:0/1963457246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7c02c810 con 0x7fef78003970 2023-12-08T08:38:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.851+0000 7fef877fe640 1 -- 172.21.15.16:0/1963457246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c02cba0 con 0x7fef78003970 2023-12-08T08:38:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.851+0000 7fef8ec7c640 1 -- 172.21.15.16:0/1963457246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef78003970 msgr2=0x7fef78003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.851+0000 7fef8ec7c640 1 --2- 172.21.15.16:0/1963457246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef78003970 0x7fef78003d90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009af0 tx=0x7fef780057f0 comp rx=0 tx=0).stop 2023-12-08T08:38:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.851+0000 7fef8d219640 1 -- 172.21.15.16:0/1963457246 reap_dead start 2023-12-08T08:38:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.851+0000 7fef8ec7c640 1 -- 172.21.15.16:0/1963457246 shutdown_connections 2023-12-08T08:38:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.851+0000 7fef8ec7c640 1 -- 172.21.15.16:0/1963457246 >> 172.21.15.16:0/1963457246 conn(0x7fef880d3010 msgr2=0x7fef880d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:40.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.851+0000 7fef8ec7c640 1 -- 172.21.15.16:0/1963457246 shutdown_connections 2023-12-08T08:38:40.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.851+0000 7fef8ec7c640 1 -- 172.21.15.16:0/1963457246 wait complete. 2023-12-08T08:38:40.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.852+0000 7fef8ec7c640 1 Processor -- start 2023-12-08T08:38:40.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.852+0000 7fef8ec7c640 1 -- start start 2023-12-08T08:38:40.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.853+0000 7fef8ec7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef880c98d0 0x7fef880a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.853+0000 7fef8ec7c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef88151900 0x7fef88150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.853+0000 7fef8ec7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef880a0970 0x7fef88151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.853+0000 7fef8ec7c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8809f010 con 0x7fef880a0970 2023-12-08T08:38:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.853+0000 7fef8ec7c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef8809e770 con 0x7fef880c98d0 2023-12-08T08:38:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.853+0000 7fef8ec7c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8809ea50 con 0x7fef88151900 2023-12-08T08:38:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.853+0000 7fef8d219640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef880a0970 0x7fef88151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.853+0000 7fef8d219640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef880a0970 0x7fef88151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52290/0 (socket says 172.21.15.16:52290) 2023-12-08T08:38:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.853+0000 7fef8d219640 1 -- 172.21.15.16:0/578928567 learned_addr learned my addr 172.21.15.16:0/578928567 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef87fff640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef88151900 0x7fef88150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef8ca18640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef880c98d0 0x7fef880a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef8d219640 1 -- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef88151900 msgr2=0x7fef88150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef8d219640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef88151900 0x7fef88150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef8d219640 1 -- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef880c98d0 msgr2=0x7fef880a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef8d219640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef880c98d0 0x7fef880a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef8d219640 1 -- 172.21.15.16:0/578928567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8809ed30 con 0x7fef880a0970 2023-12-08T08:38:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef87fff640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef88151900 0x7fef88150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef8ca18640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef880c98d0 0x7fef880a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef8d219640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef880a0970 0x7fef88151150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fef881610d0 tx=0x7fef8000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef857fa640 1 -- 172.21.15.16:0/578928567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8000e040 con 0x7fef880a0970 2023-12-08T08:38:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef857fa640 1 -- 172.21.15.16:0/578928567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8000f040 con 0x7fef880a0970 2023-12-08T08:38:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef857fa640 1 -- 172.21.15.16:0/578928567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef80015430 con 0x7fef880a0970 2023-12-08T08:38:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef8ec7c640 1 -- 172.21.15.16:0/578928567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef880c79b0 con 0x7fef880a0970 2023-12-08T08:38:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.854+0000 7fef8ec7c640 1 -- 172.21.15.16:0/578928567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef8809ccb0 con 0x7fef880a0970 2023-12-08T08:38:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.855+0000 7fef8ec7c640 1 -- 172.21.15.16:0/578928567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef880c7810 con 0x7fef880a0970 2023-12-08T08:38:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.856+0000 7fef857fa640 1 -- 172.21.15.16:0/578928567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef80015590 con 0x7fef880a0970 2023-12-08T08:38:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.856+0000 7fef857fa640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fef6c06ae70 0x7fef6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.856+0000 7fef857fa640 1 -- 172.21.15.16:0/578928567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fef80089ff0 con 0x7fef880a0970 2023-12-08T08:38:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.856+0000 7fef8ca18640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fef6c06ae70 0x7fef6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.857+0000 7fef8ca18640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fef6c06ae70 0x7fef6c06d330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fef7c036000 tx=0x7fef7c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:40.861+0000 7fef857fa640 1 -- 172.21.15.16:0/578928567 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef880c7810 con 0x7fef880a0970 2023-12-08T08:38:41.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.073+0000 7fef8ec7c640 1 -- 172.21.15.16:0/578928567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fef880c7810 con 0x7fef880a0970 2023-12-08T08:38:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.719+0000 7fef857fa640 1 -- 172.21.15.16:0/578928567 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fef880c7810 con 0x7fef880a0970 2023-12-08T08:38:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.722+0000 7fef8ec7c640 1 -- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fef6c06ae70 msgr2=0x7fef6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.722+0000 7fef8ec7c640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fef6c06ae70 0x7fef6c06d330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fef7c036000 tx=0x7fef7c030000 comp rx=0 tx=0).stop 2023-12-08T08:38:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.722+0000 7fef8ec7c640 1 -- 172.21.15.16:0/578928567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef880a0970 msgr2=0x7fef88151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.722+0000 7fef8ec7c640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef880a0970 0x7fef88151150 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fef881610d0 tx=0x7fef8000f580 comp rx=0 tx=0).stop 2023-12-08T08:38:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.722+0000 7fef8ec7c640 1 -- 172.21.15.16:0/578928567 shutdown_connections 2023-12-08T08:38:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.722+0000 7fef8ec7c640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fef6c06ae70 0x7fef6c06d330 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.722+0000 7fef8ec7c640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef880a0970 0x7fef88151150 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.722+0000 7fef8ec7c640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef88151900 0x7fef88150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:41.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.722+0000 7fef8ec7c640 1 --2- 172.21.15.16:0/578928567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef880c98d0 0x7fef880a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.722+0000 7fef8ec7c640 1 -- 172.21.15.16:0/578928567 >> 172.21.15.16:0/578928567 conn(0x7fef880d3010 msgr2=0x7fef8806c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.722+0000 7fef8ec7c640 1 -- 172.21.15.16:0/578928567 shutdown_connections 2023-12-08T08:38:41.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.722+0000 7fef8ec7c640 1 -- 172.21.15.16:0/578928567 wait complete. 2023-12-08T08:38:41.738 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-08T08:38:41.738 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-08T08:38:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.849+0000 7fb6c6f1d640 1 Processor -- start 2023-12-08T08:38:41.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.849+0000 7fb6c6f1d640 1 -- start start 2023-12-08T08:38:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6c6f1d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb6c015d490 0x7fb6c006c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6c6f1d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6c006ea40 0x7fb6c00d0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6c6f1d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c00ce270 con 0x7fb6c006c940 2023-12-08T08:38:41.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6c6f1d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c00ce550 con 0x7fb6c015d490 2023-12-08T08:38:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6c6f1d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c006b920 con 0x7fb6c006ea40 2023-12-08T08:38:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6bffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb6c006c940 0x7fb6c006e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59668/0 (socket says 172.21.15.16:59668) 2023-12-08T08:38:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6bffff640 1 -- 172.21.15.16:0/2813824041 learned_addr learned my addr 172.21.15.16:0/2813824041 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6c54ba640 1 --2- 172.21.15.16:0/2813824041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6c006ea40 0x7fb6c00d0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6c4cb9640 1 --2- 172.21.15.16:0/2813824041 >> v2:172.21.15.112:3300/0 conn(0x7fb6c015d490 0x7fb6c006c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6bf7fe640 1 -- 172.21.15.16:0/2813824041 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689776908 0 0) 0x7fb6c00ce270 con 0x7fb6c006c940 2023-12-08T08:38:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6bf7fe640 1 -- 172.21.15.16:0/2813824041 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6b0003520 con 0x7fb6c006c940 2023-12-08T08:38:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6c54ba640 1 -- 172.21.15.16:0/2813824041 >> v2:172.21.15.112:3300/0 conn(0x7fb6c015d490 msgr2=0x7fb6c006c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6c54ba640 1 --2- 172.21.15.16:0/2813824041 >> v2:172.21.15.112:3300/0 conn(0x7fb6c015d490 0x7fb6c006c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6c54ba640 1 -- 172.21.15.16:0/2813824041 >> v1:172.21.15.16:6789/0 conn(0x7fb6c006c940 legacy=0x7fb6c006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6bf7fe640 1 -- 172.21.15.16:0/2813824041 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4262602713 0 0) 0x7fb6b0003520 con 0x7fb6c006c940 2023-12-08T08:38:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6c54ba640 1 -- 172.21.15.16:0/2813824041 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c006bc00 con 0x7fb6c006ea40 2023-12-08T08:38:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.850+0000 7fb6c4cb9640 1 --2- 172.21.15.16:0/2813824041 >> v2:172.21.15.112:3300/0 conn(0x7fb6c015d490 0x7fb6c006c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:41.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.851+0000 7fb6c54ba640 1 --2- 172.21.15.16:0/2813824041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6c006ea40 0x7fb6c00d0ce0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009d90 tx=0x7fb6b802bd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98d8aac58e06d5ea server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.851+0000 7fb6bf7fe640 1 -- 172.21.15.16:0/2813824041 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac001670 con 0x7fb6c006ea40 2023-12-08T08:38:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.851+0000 7fb6bf7fe640 1 -- 172.21.15.16:0/2813824041 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b802c8a0 con 0x7fb6c006ea40 2023-12-08T08:38:41.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.851+0000 7fb6bf7fe640 1 -- 172.21.15.16:0/2813824041 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b802cc10 con 0x7fb6c006ea40 2023-12-08T08:38:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.852+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/2813824041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6c006ea40 msgr2=0x7fb6c00d0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:41.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.852+0000 7fb6c6f1d640 1 --2- 172.21.15.16:0/2813824041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6c006ea40 0x7fb6c00d0ce0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009d90 tx=0x7fb6b802bd70 comp rx=0 tx=0).stop 2023-12-08T08:38:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.852+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/2813824041 shutdown_connections 2023-12-08T08:38:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.852+0000 7fb6c6f1d640 1 --2- 172.21.15.16:0/2813824041 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6c006ea40 0x7fb6c00d0ce0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.852+0000 7fb6c6f1d640 1 --2- 172.21.15.16:0/2813824041 >> v2:172.21.15.112:3300/0 conn(0x7fb6c015d490 0x7fb6c006c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.852+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/2813824041 >> 172.21.15.16:0/2813824041 conn(0x7fb6c0099a10 msgr2=0x7fb6c0099e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.852+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/2813824041 shutdown_connections 2023-12-08T08:38:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.852+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/2813824041 wait complete. 2023-12-08T08:38:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.853+0000 7fb6c6f1d640 1 Processor -- start 2023-12-08T08:38:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.853+0000 7fb6c6f1d640 1 -- start start 2023-12-08T08:38:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c6f1d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c006c940 0x7fb6c00c4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c6f1d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c00afa50 0x7fb6c01485d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c6f1d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6c00c6280 0x7fb6c00c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:41.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c6f1d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00af350 con 0x7fb6c006c940 2023-12-08T08:38:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c6f1d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00aed90 con 0x7fb6c00afa50 2023-12-08T08:38:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c6f1d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c00af070 con 0x7fb6c00c6280 2023-12-08T08:38:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c4cb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c006c940 0x7fb6c00c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c4cb9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c006c940 0x7fb6c00c4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52300/0 (socket says 172.21.15.16:52300) 2023-12-08T08:38:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c4cb9640 1 -- 172.21.15.16:0/1924162150 learned_addr learned my addr 172.21.15.16:0/1924162150 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6bffff640 1 --2- 172.21.15.16:0/1924162150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c00afa50 0x7fb6c01485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c54ba640 1 --2- 172.21.15.16:0/1924162150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6c00c6280 0x7fb6c00c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c4cb9640 1 -- 172.21.15.16:0/1924162150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6c00c6280 msgr2=0x7fb6c00c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c4cb9640 1 --2- 172.21.15.16:0/1924162150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6c00c6280 0x7fb6c00c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c4cb9640 1 -- 172.21.15.16:0/1924162150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c00afa50 msgr2=0x7fb6c01485d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c4cb9640 1 --2- 172.21.15.16:0/1924162150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c00afa50 0x7fb6c01485d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.854+0000 7fb6c4cb9640 1 -- 172.21.15.16:0/1924162150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b4009400 con 0x7fb6c006c940 2023-12-08T08:38:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.855+0000 7fb6c4cb9640 1 --2- 172.21.15.16:0/1924162150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c006c940 0x7fb6c00c4990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400e960 tx=0x7fb6b400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.855+0000 7fb6bd7fa640 1 -- 172.21.15.16:0/1924162150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b400ca00 con 0x7fb6c006c940 2023-12-08T08:38:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.855+0000 7fb6bd7fa640 1 -- 172.21.15.16:0/1924162150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b400cb60 con 0x7fb6c006c940 2023-12-08T08:38:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.855+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/1924162150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b8009a70 con 0x7fb6c006c940 2023-12-08T08:38:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.855+0000 7fb6bd7fa640 1 -- 172.21.15.16:0/1924162150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4013470 con 0x7fb6c006c940 2023-12-08T08:38:41.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.855+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/1924162150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c0148b10 con 0x7fb6c006c940 2023-12-08T08:38:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.856+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/1924162150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c015aa80 con 0x7fb6c006c940 2023-12-08T08:38:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.856+0000 7fb6bd7fa640 1 -- 172.21.15.16:0/1924162150 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb6b40135d0 con 0x7fb6c006c940 2023-12-08T08:38:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.857+0000 7fb6bd7fa640 1 --2- 172.21.15.16:0/1924162150 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb6a406ae70 0x7fb6a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.857+0000 7fb6bd7fa640 1 -- 172.21.15.16:0/1924162150 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb6b4089ab0 con 0x7fb6c006c940 2023-12-08T08:38:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.857+0000 7fb6bffff640 1 --2- 172.21.15.16:0/1924162150 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb6a406ae70 0x7fb6a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.857+0000 7fb6bffff640 1 --2- 172.21.15.16:0/1924162150 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb6a406ae70 0x7fb6a406d330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00d11f0 tx=0x7fb6ac001dc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:41.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:41.861+0000 7fb6bd7fa640 1 -- 172.21.15.16:0/1924162150 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6c015aa80 con 0x7fb6c006c940 2023-12-08T08:38:42.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.074+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/1924162150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6c015aa80 con 0x7fb6c006c940 2023-12-08T08:38:42.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.074+0000 7fb6bd7fa640 1 -- 172.21.15.16:0/1924162150 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7fb6c015aa80 con 0x7fb6c006c940 2023-12-08T08:38:42.078 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:42.078 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/307039451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":307039451},{"type":"v1","addr":"172.21.15.112:6836","nonce":307039451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1424152341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1424152341},{"type":"v1","addr":"172.21.15.16:6833","nonce":1424152341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3597811670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3597811670},{"type":"v1","addr":"172.21.15.16:6835","nonce":3597811670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-08T08:38:18.991090+0000","modified":"2023-12-08T08:38:41.075742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1082241707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1082241707},{"type":"v1","addr":"172.21.15.112:6837","nonce":1082241707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-08T08:38:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-12-08T08:38:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.076+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/1924162150 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb6a406ae70 msgr2=0x7fb6a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.076+0000 7fb6c6f1d640 1 --2- 172.21.15.16:0/1924162150 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb6a406ae70 0x7fb6a406d330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb6c00d11f0 tx=0x7fb6ac001dc0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.076+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/1924162150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c006c940 msgr2=0x7fb6c00c4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.076+0000 7fb6c6f1d640 1 --2- 172.21.15.16:0/1924162150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c006c940 0x7fb6c00c4990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb6b400e960 tx=0x7fb6b400ee20 comp rx=0 tx=0).stop 2023-12-08T08:38:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.076+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/1924162150 shutdown_connections 2023-12-08T08:38:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.076+0000 7fb6c6f1d640 1 --2- 172.21.15.16:0/1924162150 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb6a406ae70 0x7fb6a406d330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.076+0000 7fb6c6f1d640 1 --2- 172.21.15.16:0/1924162150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6c00c6280 0x7fb6c00c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.076+0000 7fb6c6f1d640 1 --2- 172.21.15.16:0/1924162150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6c00afa50 0x7fb6c01485d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.076+0000 7fb6c6f1d640 1 --2- 172.21.15.16:0/1924162150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c006c940 0x7fb6c00c4990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.076+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/1924162150 >> 172.21.15.16:0/1924162150 conn(0x7fb6c0099a10 msgr2=0x7fb6c015a220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.076+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/1924162150 shutdown_connections 2023-12-08T08:38:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.076+0000 7fb6c6f1d640 1 -- 172.21.15.16:0/1924162150 wait complete. 2023-12-08T08:38:42.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:42.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.194+0000 7f685c275640 1 Processor -- start 2023-12-08T08:38:42.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.194+0000 7f685c275640 1 -- start start 2023-12-08T08:38:42.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.194+0000 7f685c275640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f68540d1d60 0x7f68540d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.194+0000 7f685c275640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f685415f890 0x7f685415d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.194+0000 7f685c275640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685415ee90 con 0x7f685415d580 2023-12-08T08:38:42.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.194+0000 7f685c275640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68540d0fe0 con 0x7f685415f890 2023-12-08T08:38:42.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.194+0000 7f685c275640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68540d1280 con 0x7f68540d1d60 2023-12-08T08:38:42.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.195+0000 7f685a812640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f685415f890 0x7f685415d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.195+0000 7f685a812640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f685415f890 0x7f685415d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52304/0 (socket says 172.21.15.16:52304) 2023-12-08T08:38:42.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.195+0000 7f685a812640 1 -- 172.21.15.16:0/392587572 learned_addr learned my addr 172.21.15.16:0/392587572 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:42.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.195+0000 7f6859810640 1 --2- 172.21.15.16:0/392587572 >> v2:172.21.15.112:3300/0 conn(0x7f68540d1d60 0x7f68540d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.195+0000 7f685a812640 1 -- 172.21.15.16:0/392587572 >> v2:172.21.15.112:3300/0 conn(0x7f68540d1d60 msgr2=0x7f68540d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:42.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.195+0000 7f685a812640 1 --2- 172.21.15.16:0/392587572 >> v2:172.21.15.112:3300/0 conn(0x7f68540d1d60 0x7f68540d3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.195+0000 7f685a812640 1 -- 172.21.15.16:0/392587572 >> v1:172.21.15.112:6789/0 conn(0x7f685415d580 legacy=0x7f685415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.195+0000 7f685a812640 1 -- 172.21.15.16:0/392587572 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68540d1560 con 0x7f685415f890 2023-12-08T08:38:42.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.196+0000 7f685a812640 1 --2- 172.21.15.16:0/392587572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685415f890 0x7f685415d960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f684c009e50 tx=0x7f684c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4535c0e2033eb4eb server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:42.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.196+0000 7f685900f640 1 -- 172.21.15.16:0/392587572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c02ca80 con 0x7f685415f890 2023-12-08T08:38:42.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.196+0000 7f685900f640 1 -- 172.21.15.16:0/392587572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c02cbe0 con 0x7f685415f890 2023-12-08T08:38:42.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.196+0000 7f685900f640 1 -- 172.21.15.16:0/392587572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c0343f0 con 0x7f685415f890 2023-12-08T08:38:42.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.196+0000 7f685c275640 1 -- 172.21.15.16:0/392587572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685415f890 msgr2=0x7f685415d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.196+0000 7f685c275640 1 --2- 172.21.15.16:0/392587572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685415f890 0x7f685415d960 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f684c009e50 tx=0x7f684c02bee0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.196+0000 7f685c275640 1 -- 172.21.15.16:0/392587572 shutdown_connections 2023-12-08T08:38:42.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.196+0000 7f685c275640 1 --2- 172.21.15.16:0/392587572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685415f890 0x7f685415d960 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.196+0000 7f685c275640 1 --2- 172.21.15.16:0/392587572 >> v2:172.21.15.112:3300/0 conn(0x7f68540d1d60 0x7f68540d3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.196+0000 7f685c275640 1 -- 172.21.15.16:0/392587572 >> 172.21.15.16:0/392587572 conn(0x7f68540d2fb0 msgr2=0x7f68540d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.196+0000 7f685c275640 1 -- 172.21.15.16:0/392587572 shutdown_connections 2023-12-08T08:38:42.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.196+0000 7f685c275640 1 -- 172.21.15.16:0/392587572 wait complete. 2023-12-08T08:38:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.198+0000 7f685c275640 1 Processor -- start 2023-12-08T08:38:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.198+0000 7f685c275640 1 -- start start 2023-12-08T08:38:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.198+0000 7f685c275640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68540d1d60 0x7f68540b8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.198+0000 7f685c275640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685415d580 0x7f68540b8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.198+0000 7f685c275640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685415f890 0x7f68540c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.198+0000 7f685c275640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6854153d90 con 0x7f685415d580 2023-12-08T08:38:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.198+0000 7f685c275640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68540b5ec0 con 0x7f68540d1d60 2023-12-08T08:38:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.198+0000 7f685c275640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68540b5b90 con 0x7f685415f890 2023-12-08T08:38:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.198+0000 7f6859810640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685415d580 0x7f68540b8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.198+0000 7f6859810640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685415d580 0x7f68540b8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52320/0 (socket says 172.21.15.16:52320) 2023-12-08T08:38:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.199+0000 7f6859810640 1 -- 172.21.15.16:0/2212719716 learned_addr learned my addr 172.21.15.16:0/2212719716 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.199+0000 7f685a812640 1 --2- 172.21.15.16:0/2212719716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685415f890 0x7f68540c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.199+0000 7f685a011640 1 --2- 172.21.15.16:0/2212719716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68540d1d60 0x7f68540b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.199+0000 7f6859810640 1 -- 172.21.15.16:0/2212719716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685415f890 msgr2=0x7f68540c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.199+0000 7f6859810640 1 --2- 172.21.15.16:0/2212719716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685415f890 0x7f68540c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.199+0000 7f6859810640 1 -- 172.21.15.16:0/2212719716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68540d1d60 msgr2=0x7f68540b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.199+0000 7f6859810640 1 --2- 172.21.15.16:0/2212719716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68540d1d60 0x7f68540b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.199+0000 7f6859810640 1 -- 172.21.15.16:0/2212719716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c009b30 con 0x7f685415d580 2023-12-08T08:38:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.199+0000 7f6859810640 1 --2- 172.21.15.16:0/2212719716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685415d580 0x7f68540b8850 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f684400d5f0 tx=0x7f684400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:42.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.199+0000 7f684affd640 1 -- 172.21.15.16:0/2212719716 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68440125f0 con 0x7f685415d580 2023-12-08T08:38:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.199+0000 7f684affd640 1 -- 172.21.15.16:0/2212719716 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6844012750 con 0x7f685415d580 2023-12-08T08:38:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.199+0000 7f685c275640 1 -- 172.21.15.16:0/2212719716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68540cf110 con 0x7f685415d580 2023-12-08T08:38:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.199+0000 7f684affd640 1 -- 172.21.15.16:0/2212719716 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844012b00 con 0x7f685415d580 2023-12-08T08:38:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.199+0000 7f685c275640 1 -- 172.21.15.16:0/2212719716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68540cf6d0 con 0x7f685415d580 2023-12-08T08:38:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.200+0000 7f685c275640 1 -- 172.21.15.16:0/2212719716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68540d09d0 con 0x7f685415d580 2023-12-08T08:38:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.201+0000 7f684affd640 1 -- 172.21.15.16:0/2212719716 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6844012c60 con 0x7f685415d580 2023-12-08T08:38:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.201+0000 7f684affd640 1 --2- 172.21.15.16:0/2212719716 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f683806ae40 0x7f683806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.201+0000 7f684affd640 1 -- 172.21.15.16:0/2212719716 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6844089c40 con 0x7f685415d580 2023-12-08T08:38:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.201+0000 7f685a011640 1 --2- 172.21.15.16:0/2212719716 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f683806ae40 0x7f683806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.202+0000 7f685a011640 1 --2- 172.21.15.16:0/2212719716 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f683806ae40 0x7f683806d300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6850000f60 tx=0x7f6850009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:42.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.205+0000 7f684affd640 1 -- 172.21.15.16:0/2212719716 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6844054ab0 con 0x7f685415d580 2023-12-08T08:38:42.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.381+0000 7f685c275640 1 -- 172.21.15.16:0/2212719716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68540b6050 con 0x7f685415d580 2023-12-08T08:38:42.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.382+0000 7f684affd640 1 -- 172.21.15.16:0/2212719716 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f68440580d0 con 0x7f685415d580 2023-12-08T08:38:42.385 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:42.385 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:24.030625+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.384+0000 7f685c275640 1 -- 172.21.15.16:0/2212719716 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f683806ae40 msgr2=0x7f683806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.384+0000 7f685c275640 1 --2- 172.21.15.16:0/2212719716 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f683806ae40 0x7f683806d300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6850000f60 tx=0x7f6850009040 comp rx=0 tx=0).stop 2023-12-08T08:38:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.384+0000 7f685c275640 1 -- 172.21.15.16:0/2212719716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685415d580 msgr2=0x7f68540b8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.384+0000 7f685c275640 1 --2- 172.21.15.16:0/2212719716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685415d580 0x7f68540b8850 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f684400d5f0 tx=0x7f684400dab0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.384+0000 7f685c275640 1 -- 172.21.15.16:0/2212719716 shutdown_connections 2023-12-08T08:38:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.384+0000 7f685c275640 1 --2- 172.21.15.16:0/2212719716 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f683806ae40 0x7f683806d300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.384+0000 7f685c275640 1 --2- 172.21.15.16:0/2212719716 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f685415f890 0x7f68540c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.384+0000 7f685c275640 1 --2- 172.21.15.16:0/2212719716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685415d580 0x7f68540b8850 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.384+0000 7f685c275640 1 --2- 172.21.15.16:0/2212719716 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f68540d1d60 0x7f68540b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.384+0000 7f685c275640 1 -- 172.21.15.16:0/2212719716 >> 172.21.15.16:0/2212719716 conn(0x7f68540d2fb0 msgr2=0x7f685415c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.384+0000 7f685c275640 1 -- 172.21.15.16:0/2212719716 shutdown_connections 2023-12-08T08:38:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.384+0000 7f685c275640 1 -- 172.21.15.16:0/2212719716 wait complete. 2023-12-08T08:38:42.398 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-08T08:38:42.398 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-08T08:38:42.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.502+0000 7f33d43b0640 1 Processor -- start 2023-12-08T08:38:42.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.502+0000 7f33d43b0640 1 -- start start 2023-12-08T08:38:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.503+0000 7f33d43b0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33cc0b1cf0 0x7f33cc070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.503+0000 7f33d43b0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33cc06dae0 con 0x7f33cc071dc0 2023-12-08T08:38:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.503+0000 7f33d43b0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33cc0ae2e0 con 0x7f33cc159a10 2023-12-08T08:38:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.503+0000 7f33d43b0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33cc06dd30 con 0x7f33cc0b1cf0 2023-12-08T08:38:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.503+0000 7f33d194b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f33cc071dc0 0x7f33cc070750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59680/0 (socket says 172.21.15.16:59680) 2023-12-08T08:38:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.503+0000 7f33d294d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33cc0b1cf0 0x7f33cc070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.503+0000 7f33d194b640 1 -- 172.21.15.16:0/3460860732 learned_addr learned my addr 172.21.15.16:0/3460860732 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.503+0000 7f33d294d640 1 -- 172.21.15.16:0/3460860732 >> v1:172.21.15.112:6789/0 conn(0x7f33cc159a10 legacy=0x7f33cc0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.503+0000 7f33d294d640 1 -- 172.21.15.16:0/3460860732 >> v1:172.21.15.16:6789/0 conn(0x7f33cc071dc0 legacy=0x7f33cc070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.503+0000 7f33d294d640 1 -- 172.21.15.16:0/3460860732 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc06e010 con 0x7f33cc0b1cf0 2023-12-08T08:38:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.504+0000 7f33d294d640 1 --2- 172.21.15.16:0/3460860732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc0b1cf0 0x7f33cc070e60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009d90 tx=0x7f33c402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4600f508bb6e90 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.504+0000 7f33d114a640 1 -- 172.21.15.16:0/3460860732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c402cb40 con 0x7f33cc0b1cf0 2023-12-08T08:38:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.504+0000 7f33d114a640 1 -- 172.21.15.16:0/3460860732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c402cca0 con 0x7f33cc0b1cf0 2023-12-08T08:38:42.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.504+0000 7f33d114a640 1 -- 172.21.15.16:0/3460860732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4034660 con 0x7f33cc0b1cf0 2023-12-08T08:38:42.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.505+0000 7f33d43b0640 1 -- 172.21.15.16:0/3460860732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc0b1cf0 msgr2=0x7f33cc070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.505+0000 7f33d43b0640 1 --2- 172.21.15.16:0/3460860732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc0b1cf0 0x7f33cc070e60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f33c4009d90 tx=0x7f33c402bd90 comp rx=0 tx=0).stop 2023-12-08T08:38:42.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.505+0000 7f33d43b0640 1 -- 172.21.15.16:0/3460860732 shutdown_connections 2023-12-08T08:38:42.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.505+0000 7f33d43b0640 1 --2- 172.21.15.16:0/3460860732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc0b1cf0 0x7f33cc070e60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.505+0000 7f33d43b0640 1 -- 172.21.15.16:0/3460860732 >> 172.21.15.16:0/3460860732 conn(0x7f33cc0b09f0 msgr2=0x7f33cc0b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:42.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.505+0000 7f33d43b0640 1 -- 172.21.15.16:0/3460860732 shutdown_connections 2023-12-08T08:38:42.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.505+0000 7f33d43b0640 1 -- 172.21.15.16:0/3460860732 wait complete. 2023-12-08T08:38:42.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.506+0000 7f33d43b0640 1 Processor -- start 2023-12-08T08:38:42.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.506+0000 7f33d43b0640 1 -- start start 2023-12-08T08:38:42.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d43b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc071dc0 0x7f33cc13d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d43b0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33cc0b1cf0 0x7f33cc0afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d43b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33cc159a10 0x7f33cc13d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d43b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc099a10 con 0x7f33cc071dc0 2023-12-08T08:38:42.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d43b0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc0af820 con 0x7f33cc159a10 2023-12-08T08:38:42.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d43b0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc0af250 con 0x7f33cc0b1cf0 2023-12-08T08:38:42.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d214c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc071dc0 0x7f33cc13d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d214c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc071dc0 0x7f33cc13d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52336/0 (socket says 172.21.15.16:52336) 2023-12-08T08:38:42.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d214c640 1 -- 172.21.15.16:0/875603556 learned_addr learned my addr 172.21.15.16:0/875603556 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:42.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d294d640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33cc159a10 0x7f33cc13d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d194b640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33cc0b1cf0 0x7f33cc0afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d214c640 1 -- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33cc0b1cf0 msgr2=0x7f33cc0afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d214c640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33cc0b1cf0 0x7f33cc0afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d214c640 1 -- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33cc159a10 msgr2=0x7f33cc13d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d214c640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33cc159a10 0x7f33cc13d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d214c640 1 -- 172.21.15.16:0/875603556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c4009a70 con 0x7f33cc071dc0 2023-12-08T08:38:42.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d294d640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33cc159a10 0x7f33cc13d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:42.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33d214c640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc071dc0 0x7f33cc13d040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f33c800b3f0 tx=0x7f33c800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:42.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33c2ffd640 1 -- 172.21.15.16:0/875603556 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8011830 con 0x7f33cc071dc0 2023-12-08T08:38:42.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33c2ffd640 1 -- 172.21.15.16:0/875603556 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c8011990 con 0x7f33cc071dc0 2023-12-08T08:38:42.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.507+0000 7f33c2ffd640 1 -- 172.21.15.16:0/875603556 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8011d40 con 0x7f33cc071dc0 2023-12-08T08:38:42.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.508+0000 7f33d43b0640 1 -- 172.21.15.16:0/875603556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33cc0af4f0 con 0x7f33cc071dc0 2023-12-08T08:38:42.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.508+0000 7f33d43b0640 1 -- 172.21.15.16:0/875603556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33cc09a1e0 con 0x7f33cc071dc0 2023-12-08T08:38:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.509+0000 7f33d43b0640 1 -- 172.21.15.16:0/875603556 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33cc067430 con 0x7f33cc071dc0 2023-12-08T08:38:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.509+0000 7f33c2ffd640 1 -- 172.21.15.16:0/875603556 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33c801a400 con 0x7f33cc071dc0 2023-12-08T08:38:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.510+0000 7f33c2ffd640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f33b006ae40 0x7f33b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.510+0000 7f33c2ffd640 1 -- 172.21.15.16:0/875603556 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f33c8089760 con 0x7f33cc071dc0 2023-12-08T08:38:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.510+0000 7f33d194b640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f33b006ae40 0x7f33b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.510+0000 7f33d194b640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f33b006ae40 0x7f33b006d300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f33bc007fd0 tx=0x7f33bc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:42.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.514+0000 7f33c2ffd640 1 -- 172.21.15.16:0/875603556 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f33c8054520 con 0x7f33cc071dc0 2023-12-08T08:38:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.687+0000 7f33d43b0640 1 -- 172.21.15.16:0/875603556 --> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33cc0af960 con 0x7f33b006ae40 2023-12-08T08:38:42.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.697+0000 7f33c2ffd640 1 -- 172.21.15.16:0/875603556 <== mgr.4111 v2:172.21.15.112:6832/804067392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345284 (secure 0 0 0) 0x7f33bc001c90 con 0x7f33b006ae40 2023-12-08T08:38:42.704 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:42.706 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-12-08T08:38:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.701+0000 7f33d43b0640 1 -- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f33b006ae40 msgr2=0x7f33b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.701+0000 7f33d43b0640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f33b006ae40 0x7f33b006d300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f33bc007fd0 tx=0x7f33bc009040 comp rx=0 tx=0).stop 2023-12-08T08:38:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.701+0000 7f33d43b0640 1 -- 172.21.15.16:0/875603556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc071dc0 msgr2=0x7f33cc13d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.701+0000 7f33d43b0640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc071dc0 0x7f33cc13d040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f33c800b3f0 tx=0x7f33c800b8b0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.701+0000 7f33d43b0640 1 -- 172.21.15.16:0/875603556 shutdown_connections 2023-12-08T08:38:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.701+0000 7f33d43b0640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f33b006ae40 0x7f33b006d300 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.701+0000 7f33d43b0640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f33cc159a10 0x7f33cc13d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.701+0000 7f33d43b0640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f33cc0b1cf0 0x7f33cc0afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.701+0000 7f33d43b0640 1 --2- 172.21.15.16:0/875603556 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc071dc0 0x7f33cc13d040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.701+0000 7f33d43b0640 1 -- 172.21.15.16:0/875603556 >> 172.21.15.16:0/875603556 conn(0x7f33cc0b09f0 msgr2=0x7f33cc0ccbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.701+0000 7f33d43b0640 1 -- 172.21.15.16:0/875603556 shutdown_connections 2023-12-08T08:38:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.701+0000 7f33d43b0640 1 -- 172.21.15.16:0/875603556 wait complete. 2023-12-08T08:38:42.714 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-12-08T08:38:40.979329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231864,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742856,"statfs":{"total":773094113280,"available":772856684544,"internally_reserved":0,"allocated":3284992,"data_stored":1988346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003257"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659961+0000","last_change":"2023-12-08T08:38:18.634619+0000","last_active":"2023-12-08T08:38:23.659961+0000","last_peered":"2023-12-08T08:38:23.659961+0000","last_clean":"2023-12-08T08:38:23.659961+0000","last_became_active":"2023-12-08T08:38:18.634328+0000","last_became_peered":"2023-12-08T08:38:18.634328+0000","last_unstale":"2023-12-08T08:38:23.659961+0000","last_undegraded":"2023-12-08T08:38:23.659961+0000","last_fullsized":"2023-12-08T08:38:23.659961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:11.067589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656122+0000","last_change":"2023-12-08T08:38:19.627512+0000","last_active":"2023-12-08T08:38:23.656122+0000","last_peered":"2023-12-08T08:38:23.656122+0000","last_clean":"2023-12-08T08:38:23.656122+0000","last_became_active":"2023-12-08T08:38:19.625456+0000","last_became_peered":"2023-12-08T08:38:19.625456+0000","last_unstale":"2023-12-08T08:38:23.656122+0000","last_undegraded":"2023-12-08T08:38:23.656122+0000","last_fullsized":"2023-12-08T08:38:23.656122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576137+0000","last_change":"2023-12-08T08:38:23.856216+0000","last_active":"2023-12-08T08:38:25.576137+0000","last_peered":"2023-12-08T08:38:25.576137+0000","last_clean":"2023-12-08T08:38:25.576137+0000","last_became_active":"2023-12-08T08:38:23.855623+0000","last_became_peered":"2023-12-08T08:38:23.855623+0000","last_unstale":"2023-12-08T08:38:25.576137+0000","last_undegraded":"2023-12-08T08:38:25.576137+0000","last_fullsized":"2023-12-08T08:38:25.576137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:26:16.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656314+0000","last_change":"2023-12-08T08:38:18.631598+0000","last_active":"2023-12-08T08:38:23.656314+0000","last_peered":"2023-12-08T08:38:23.656314+0000","last_clean":"2023-12-08T08:38:23.656314+0000","last_became_active":"2023-12-08T08:38:18.631295+0000","last_became_peered":"2023-12-08T08:38:18.631295+0000","last_unstale":"2023-12-08T08:38:23.656314+0000","last_undegraded":"2023-12-08T08:38:23.656314+0000","last_fullsized":"2023-12-08T08:38:23.656314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659744+0000","last_change":"2023-12-08T08:38:19.633463+0000","last_active":"2023-12-08T08:38:23.659744+0000","last_peered":"2023-12-08T08:38:23.659744+0000","last_clean":"2023-12-08T08:38:23.659744+0000","last_became_active":"2023-12-08T08:38:19.633094+0000","last_became_peered":"2023-12-08T08:38:19.633094+0000","last_unstale":"2023-12-08T08:38:23.659744+0000","last_undegraded":"2023-12-08T08:38:23.659744+0000","last_fullsized":"2023-12-08T08:38:23.659744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608661+0000","last_change":"2023-12-08T08:38:23.674366+0000","last_active":"2023-12-08T08:38:24.608661+0000","last_peered":"2023-12-08T08:38:24.608661+0000","last_clean":"2023-12-08T08:38:24.608661+0000","last_became_active":"2023-12-08T08:38:23.673810+0000","last_became_peered":"2023-12-08T08:38:23.673810+0000","last_unstale":"2023-12-08T08:38:24.608661+0000","last_undegraded":"2023-12-08T08:38:24.608661+0000","last_fullsized":"2023-12-08T08:38:24.608661+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:47.374495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852405+0000","last_change":"2023-12-08T08:38:18.633940+0000","last_active":"2023-12-08T08:38:23.852405+0000","last_peered":"2023-12-08T08:38:23.852405+0000","last_clean":"2023-12-08T08:38:23.852405+0000","last_became_active":"2023-12-08T08:38:18.631964+0000","last_became_peered":"2023-12-08T08:38:18.631964+0000","last_unstale":"2023-12-08T08:38:23.852405+0000","last_undegraded":"2023-12-08T08:38:23.852405+0000","last_fullsized":"2023-12-08T08:38:23.852405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:37.317712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661342+0000","last_change":"2023-12-08T08:38:19.627445+0000","last_active":"2023-12-08T08:38:23.661342+0000","last_peered":"2023-12-08T08:38:23.661342+0000","last_clean":"2023-12-08T08:38:23.661342+0000","last_became_active":"2023-12-08T08:38:19.627174+0000","last_became_peered":"2023-12-08T08:38:19.627174+0000","last_unstale":"2023-12-08T08:38:23.661342+0000","last_undegraded":"2023-12-08T08:38:23.661342+0000","last_fullsized":"2023-12-08T08:38:23.661342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:32.116348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609171+0000","last_change":"2023-12-08T08:38:23.674338+0000","last_active":"2023-12-08T08:38:24.609171+0000","last_peered":"2023-12-08T08:38:24.609171+0000","last_clean":"2023-12-08T08:38:24.609171+0000","last_became_active":"2023-12-08T08:38:23.673614+0000","last_became_peered":"2023-12-08T08:38:23.673614+0000","last_unstale":"2023-12-08T08:38:24.609171+0000","last_undegraded":"2023-12-08T08:38:24.609171+0000","last_fullsized":"2023-12-08T08:38:24.609171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:59.405196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852122+0000","last_change":"2023-12-08T08:38:18.633561+0000","last_active":"2023-12-08T08:38:23.852122+0000","last_peered":"2023-12-08T08:38:23.852122+0000","last_clean":"2023-12-08T08:38:23.852122+0000","last_became_active":"2023-12-08T08:38:18.632646+0000","last_became_peered":"2023-12-08T08:38:18.632646+0000","last_unstale":"2023-12-08T08:38:23.852122+0000","last_undegraded":"2023-12-08T08:38:23.852122+0000","last_fullsized":"2023-12-08T08:38:23.852122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:36:34.339195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662141+0000","last_change":"2023-12-08T08:38:19.632216+0000","last_active":"2023-12-08T08:38:23.662141+0000","last_peered":"2023-12-08T08:38:23.662141+0000","last_clean":"2023-12-08T08:38:23.662141+0000","last_became_active":"2023-12-08T08:38:19.631599+0000","last_became_peered":"2023-12-08T08:38:19.631599+0000","last_unstale":"2023-12-08T08:38:23.662141+0000","last_undegraded":"2023-12-08T08:38:23.662141+0000","last_fullsized":"2023-12-08T08:38:23.662141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:11.797914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608922+0000","last_change":"2023-12-08T08:38:23.669631+0000","last_active":"2023-12-08T08:38:24.608922+0000","last_peered":"2023-12-08T08:38:24.608922+0000","last_clean":"2023-12-08T08:38:24.608922+0000","last_became_active":"2023-12-08T08:38:23.669222+0000","last_became_peered":"2023-12-08T08:38:23.669222+0000","last_unstale":"2023-12-08T08:38:24.608922+0000","last_undegraded":"2023-12-08T08:38:24.608922+0000","last_fullsized":"2023-12-08T08:38:24.608922+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:49:33.936211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664041+0000","last_change":"2023-12-08T08:38:19.620724+0000","last_active":"2023-12-08T08:38:23.664041+0000","last_peered":"2023-12-08T08:38:23.664041+0000","last_clean":"2023-12-08T08:38:23.664041+0000","last_became_active":"2023-12-08T08:38:19.620366+0000","last_became_peered":"2023-12-08T08:38:19.620366+0000","last_unstale":"2023-12-08T08:38:23.664041+0000","last_undegraded":"2023-12-08T08:38:23.664041+0000","last_fullsized":"2023-12-08T08:38:23.664041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:00:01.146409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852136+0000","last_change":"2023-12-08T08:38:18.632086+0000","last_active":"2023-12-08T08:38:23.852136+0000","last_peered":"2023-12-08T08:38:23.852136+0000","last_clean":"2023-12-08T08:38:23.852136+0000","last_became_active":"2023-12-08T08:38:18.631756+0000","last_became_peered":"2023-12-08T08:38:18.631756+0000","last_unstale":"2023-12-08T08:38:23.852136+0000","last_undegraded":"2023-12-08T08:38:23.852136+0000","last_fullsized":"2023-12-08T08:38:23.852136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:14.342540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609352+0000","last_change":"2023-12-08T08:38:23.673898+0000","last_active":"2023-12-08T08:38:24.609352+0000","last_peered":"2023-12-08T08:38:24.609352+0000","last_clean":"2023-12-08T08:38:24.609352+0000","last_became_active":"2023-12-08T08:38:23.673389+0000","last_became_peered":"2023-12-08T08:38:23.673389+0000","last_unstale":"2023-12-08T08:38:24.609352+0000","last_undegraded":"2023-12-08T08:38:24.609352+0000","last_fullsized":"2023-12-08T08:38:24.609352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:15.835775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655652+0000","last_change":"2023-12-08T08:38:19.627148+0000","last_active":"2023-12-08T08:38:23.655652+0000","last_peered":"2023-12-08T08:38:23.655652+0000","last_clean":"2023-12-08T08:38:23.655652+0000","last_became_active":"2023-12-08T08:38:19.626846+0000","last_became_peered":"2023-12-08T08:38:19.626846+0000","last_unstale":"2023-12-08T08:38:23.655652+0000","last_undegraded":"2023-12-08T08:38:23.655652+0000","last_fullsized":"2023-12-08T08:38:23.655652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.666654+0000","last_change":"2023-12-08T08:38:18.632191+0000","last_active":"2023-12-08T08:38:23.666654+0000","last_peered":"2023-12-08T08:38:23.666654+0000","last_clean":"2023-12-08T08:38:23.666654+0000","last_became_active":"2023-12-08T08:38:18.631927+0000","last_became_peered":"2023-12-08T08:38:18.631927+0000","last_unstale":"2023-12-08T08:38:23.666654+0000","last_undegraded":"2023-12-08T08:38:23.666654+0000","last_fullsized":"2023-12-08T08:38:23.666654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:10.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608968+0000","last_change":"2023-12-08T08:38:23.678875+0000","last_active":"2023-12-08T08:38:24.608968+0000","last_peered":"2023-12-08T08:38:24.608968+0000","last_clean":"2023-12-08T08:38:24.608968+0000","last_became_active":"2023-12-08T08:38:23.678332+0000","last_became_peered":"2023-12-08T08:38:23.678332+0000","last_unstale":"2023-12-08T08:38:24.608968+0000","last_undegraded":"2023-12-08T08:38:24.608968+0000","last_fullsized":"2023-12-08T08:38:24.608968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:36.154208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656075+0000","last_change":"2023-12-08T08:38:19.625657+0000","last_active":"2023-12-08T08:38:23.656075+0000","last_peered":"2023-12-08T08:38:23.656075+0000","last_clean":"2023-12-08T08:38:23.656075+0000","last_became_active":"2023-12-08T08:38:19.624790+0000","last_became_peered":"2023-12-08T08:38:19.624790+0000","last_unstale":"2023-12-08T08:38:23.656075+0000","last_undegraded":"2023-12-08T08:38:23.656075+0000","last_fullsized":"2023-12-08T08:38:23.656075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:32.116348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664683+0000","last_change":"2023-12-08T08:38:18.637627+0000","last_active":"2023-12-08T08:38:23.664683+0000","last_peered":"2023-12-08T08:38:23.664683+0000","last_clean":"2023-12-08T08:38:23.664683+0000","last_became_active":"2023-12-08T08:38:18.637335+0000","last_became_peered":"2023-12-08T08:38:18.637335+0000","last_unstale":"2023-12-08T08:38:23.664683+0000","last_undegraded":"2023-12-08T08:38:23.664683+0000","last_fullsized":"2023-12-08T08:38:23.664683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.828080+0000","last_change":"2023-12-08T08:38:23.852913+0000","last_active":"2023-12-08T08:38:25.828080+0000","last_peered":"2023-12-08T08:38:25.828080+0000","last_clean":"2023-12-08T08:38:25.828080+0000","last_became_active":"2023-12-08T08:38:23.852508+0000","last_became_peered":"2023-12-08T08:38:23.852508+0000","last_unstale":"2023-12-08T08:38:25.828080+0000","last_undegraded":"2023-12-08T08:38:25.828080+0000","last_fullsized":"2023-12-08T08:38:25.828080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:48.357070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658598+0000","last_change":"2023-12-08T08:38:19.623518+0000","last_active":"2023-12-08T08:38:23.658598+0000","last_peered":"2023-12-08T08:38:23.658598+0000","last_clean":"2023-12-08T08:38:23.658598+0000","last_became_active":"2023-12-08T08:38:19.623240+0000","last_became_peered":"2023-12-08T08:38:19.623240+0000","last_unstale":"2023-12-08T08:38:23.658598+0000","last_undegraded":"2023-12-08T08:38:23.658598+0000","last_fullsized":"2023-12-08T08:38:23.658598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:43.773215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851377+0000","last_change":"2023-12-08T08:38:18.633585+0000","last_active":"2023-12-08T08:38:23.851377+0000","last_peered":"2023-12-08T08:38:23.851377+0000","last_clean":"2023-12-08T08:38:23.851377+0000","last_became_active":"2023-12-08T08:38:18.633112+0000","last_became_peered":"2023-12-08T08:38:18.633112+0000","last_unstale":"2023-12-08T08:38:23.851377+0000","last_undegraded":"2023-12-08T08:38:23.851377+0000","last_fullsized":"2023-12-08T08:38:23.851377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:42.768579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609486+0000","last_change":"2023-12-08T08:38:23.678810+0000","last_active":"2023-12-08T08:38:24.609486+0000","last_peered":"2023-12-08T08:38:24.609486+0000","last_clean":"2023-12-08T08:38:24.609486+0000","last_became_active":"2023-12-08T08:38:23.678331+0000","last_became_peered":"2023-12-08T08:38:23.678331+0000","last_unstale":"2023-12-08T08:38:24.609486+0000","last_undegraded":"2023-12-08T08:38:24.609486+0000","last_fullsized":"2023-12-08T08:38:24.609486+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:17.394710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.666222+0000","last_change":"2023-12-08T08:38:19.619485+0000","last_active":"2023-12-08T08:38:23.666222+0000","last_peered":"2023-12-08T08:38:23.666222+0000","last_clean":"2023-12-08T08:38:23.666222+0000","last_became_active":"2023-12-08T08:38:19.618880+0000","last_became_peered":"2023-12-08T08:38:19.618880+0000","last_unstale":"2023-12-08T08:38:23.666222+0000","last_undegraded":"2023-12-08T08:38:23.666222+0000","last_fullsized":"2023-12-08T08:38:23.666222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:43.336635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665379+0000","last_change":"2023-12-08T08:38:18.633093+0000","last_active":"2023-12-08T08:38:23.665379+0000","last_peered":"2023-12-08T08:38:23.665379+0000","last_clean":"2023-12-08T08:38:23.665379+0000","last_became_active":"2023-12-08T08:38:18.632832+0000","last_became_peered":"2023-12-08T08:38:18.632832+0000","last_unstale":"2023-12-08T08:38:23.665379+0000","last_undegraded":"2023-12-08T08:38:23.665379+0000","last_fullsized":"2023-12-08T08:38:23.665379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:37.317712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608658+0000","last_change":"2023-12-08T08:38:23.853769+0000","last_active":"2023-12-08T08:38:24.608658+0000","last_peered":"2023-12-08T08:38:24.608658+0000","last_clean":"2023-12-08T08:38:24.608658+0000","last_became_active":"2023-12-08T08:38:23.853211+0000","last_became_peered":"2023-12-08T08:38:23.853211+0000","last_unstale":"2023-12-08T08:38:24.608658+0000","last_undegraded":"2023-12-08T08:38:24.608658+0000","last_fullsized":"2023-12-08T08:38:24.608658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:59:15.111403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656171+0000","last_change":"2023-12-08T08:38:19.626321+0000","last_active":"2023-12-08T08:38:23.656171+0000","last_peered":"2023-12-08T08:38:23.656171+0000","last_clean":"2023-12-08T08:38:23.656171+0000","last_became_active":"2023-12-08T08:38:19.624741+0000","last_became_peered":"2023-12-08T08:38:19.624741+0000","last_unstale":"2023-12-08T08:38:23.656171+0000","last_undegraded":"2023-12-08T08:38:23.656171+0000","last_fullsized":"2023-12-08T08:38:23.656171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:49:29.898351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664728+0000","last_change":"2023-12-08T08:38:18.636600+0000","last_active":"2023-12-08T08:38:23.664728+0000","last_peered":"2023-12-08T08:38:23.664728+0000","last_clean":"2023-12-08T08:38:23.664728+0000","last_became_active":"2023-12-08T08:38:18.633400+0000","last_became_peered":"2023-12-08T08:38:18.633400+0000","last_unstale":"2023-12-08T08:38:23.664728+0000","last_undegraded":"2023-12-08T08:38:23.664728+0000","last_fullsized":"2023-12-08T08:38:23.664728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:37.317712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576086+0000","last_change":"2023-12-08T08:38:23.856023+0000","last_active":"2023-12-08T08:38:25.576086+0000","last_peered":"2023-12-08T08:38:25.576086+0000","last_clean":"2023-12-08T08:38:25.576086+0000","last_became_active":"2023-12-08T08:38:23.855599+0000","last_became_peered":"2023-12-08T08:38:23.855599+0000","last_unstale":"2023-12-08T08:38:25.576086+0000","last_undegraded":"2023-12-08T08:38:25.576086+0000","last_fullsized":"2023-12-08T08:38:25.576086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T20:21:44.003574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662450+0000","last_change":"2023-12-08T08:38:19.622140+0000","last_active":"2023-12-08T08:38:23.662450+0000","last_peered":"2023-12-08T08:38:23.662450+0000","last_clean":"2023-12-08T08:38:23.662450+0000","last_became_active":"2023-12-08T08:38:19.621784+0000","last_became_peered":"2023-12-08T08:38:19.621784+0000","last_unstale":"2023-12-08T08:38:23.662450+0000","last_undegraded":"2023-12-08T08:38:23.662450+0000","last_fullsized":"2023-12-08T08:38:23.662450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852845+0000","last_change":"2023-12-08T08:38:18.634123+0000","last_active":"2023-12-08T08:38:23.852845+0000","last_peered":"2023-12-08T08:38:23.852845+0000","last_clean":"2023-12-08T08:38:23.852845+0000","last_became_active":"2023-12-08T08:38:18.632874+0000","last_became_peered":"2023-12-08T08:38:18.632874+0000","last_unstale":"2023-12-08T08:38:23.852845+0000","last_undegraded":"2023-12-08T08:38:23.852845+0000","last_fullsized":"2023-12-08T08:38:23.852845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:07.497079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.326786+0000","last_change":"2023-12-08T08:38:23.852386+0000","last_active":"2023-12-08T08:38:26.326786+0000","last_peered":"2023-12-08T08:38:26.326786+0000","last_clean":"2023-12-08T08:38:26.326786+0000","last_became_active":"2023-12-08T08:38:23.851973+0000","last_became_peered":"2023-12-08T08:38:23.851973+0000","last_unstale":"2023-12-08T08:38:26.326786+0000","last_undegraded":"2023-12-08T08:38:26.326786+0000","last_fullsized":"2023-12-08T08:38:26.326786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:22.372635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662330+0000","last_change":"2023-12-08T08:38:19.631928+0000","last_active":"2023-12-08T08:38:23.662330+0000","last_peered":"2023-12-08T08:38:23.662330+0000","last_clean":"2023-12-08T08:38:23.662330+0000","last_became_active":"2023-12-08T08:38:19.631447+0000","last_became_peered":"2023-12-08T08:38:19.631447+0000","last_unstale":"2023-12-08T08:38:23.662330+0000","last_undegraded":"2023-12-08T08:38:23.662330+0000","last_fullsized":"2023-12-08T08:38:23.662330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:13.356849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662548+0000","last_change":"2023-12-08T08:38:18.634530+0000","last_active":"2023-12-08T08:38:23.662548+0000","last_peered":"2023-12-08T08:38:23.662548+0000","last_clean":"2023-12-08T08:38:23.662548+0000","last_became_active":"2023-12-08T08:38:18.634239+0000","last_became_peered":"2023-12-08T08:38:18.634239+0000","last_unstale":"2023-12-08T08:38:23.662548+0000","last_undegraded":"2023-12-08T08:38:23.662548+0000","last_fullsized":"2023-12-08T08:38:23.662548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:42.768579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.827628+0000","last_change":"2023-12-08T08:38:23.676640+0000","last_active":"2023-12-08T08:38:25.827628+0000","last_peered":"2023-12-08T08:38:25.827628+0000","last_clean":"2023-12-08T08:38:25.827628+0000","last_became_active":"2023-12-08T08:38:23.675999+0000","last_became_peered":"2023-12-08T08:38:23.675999+0000","last_unstale":"2023-12-08T08:38:25.827628+0000","last_undegraded":"2023-12-08T08:38:25.827628+0000","last_fullsized":"2023-12-08T08:38:25.827628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:15.835775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658939+0000","last_change":"2023-12-08T08:38:19.615261+0000","last_active":"2023-12-08T08:38:23.658939+0000","last_peered":"2023-12-08T08:38:23.658939+0000","last_clean":"2023-12-08T08:38:23.658939+0000","last_became_active":"2023-12-08T08:38:19.614990+0000","last_became_peered":"2023-12-08T08:38:19.614990+0000","last_unstale":"2023-12-08T08:38:23.658939+0000","last_undegraded":"2023-12-08T08:38:23.658939+0000","last_fullsized":"2023-12-08T08:38:23.658939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:05:08.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662040+0000","last_change":"2023-12-08T08:38:18.630062+0000","last_active":"2023-12-08T08:38:23.662040+0000","last_peered":"2023-12-08T08:38:23.662040+0000","last_clean":"2023-12-08T08:38:23.662040+0000","last_became_active":"2023-12-08T08:38:18.629798+0000","last_became_peered":"2023-12-08T08:38:18.629798+0000","last_unstale":"2023-12-08T08:38:23.662040+0000","last_undegraded":"2023-12-08T08:38:23.662040+0000","last_fullsized":"2023-12-08T08:38:23.662040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:13.807850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.827791+0000","last_change":"2023-12-08T08:38:23.674624+0000","last_active":"2023-12-08T08:38:25.827791+0000","last_peered":"2023-12-08T08:38:25.827791+0000","last_clean":"2023-12-08T08:38:25.827791+0000","last_became_active":"2023-12-08T08:38:23.674121+0000","last_became_peered":"2023-12-08T08:38:23.674121+0000","last_unstale":"2023-12-08T08:38:25.827791+0000","last_undegraded":"2023-12-08T08:38:25.827791+0000","last_fullsized":"2023-12-08T08:38:25.827791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:17.394710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665266+0000","last_change":"2023-12-08T08:38:19.621679+0000","last_active":"2023-12-08T08:38:23.665266+0000","last_peered":"2023-12-08T08:38:23.665266+0000","last_clean":"2023-12-08T08:38:23.665266+0000","last_became_active":"2023-12-08T08:38:19.621388+0000","last_became_peered":"2023-12-08T08:38:19.621388+0000","last_unstale":"2023-12-08T08:38:23.665266+0000","last_undegraded":"2023-12-08T08:38:23.665266+0000","last_fullsized":"2023-12-08T08:38:23.665266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662810+0000","last_change":"2023-12-08T08:38:18.632407+0000","last_active":"2023-12-08T08:38:23.662810+0000","last_peered":"2023-12-08T08:38:23.662810+0000","last_clean":"2023-12-08T08:38:23.662810+0000","last_became_active":"2023-12-08T08:38:18.632115+0000","last_became_peered":"2023-12-08T08:38:18.632115+0000","last_unstale":"2023-12-08T08:38:23.662810+0000","last_undegraded":"2023-12-08T08:38:23.662810+0000","last_fullsized":"2023-12-08T08:38:23.662810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:05:14.591351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.828126+0000","last_change":"2023-12-08T08:38:23.675339+0000","last_active":"2023-12-08T08:38:25.828126+0000","last_peered":"2023-12-08T08:38:25.828126+0000","last_clean":"2023-12-08T08:38:25.828126+0000","last_became_active":"2023-12-08T08:38:23.674249+0000","last_became_peered":"2023-12-08T08:38:23.674249+0000","last_unstale":"2023-12-08T08:38:25.828126+0000","last_undegraded":"2023-12-08T08:38:25.828126+0000","last_fullsized":"2023-12-08T08:38:25.828126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:34:29.925547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.660253+0000","last_change":"2023-12-08T08:38:19.611646+0000","last_active":"2023-12-08T08:38:23.660253+0000","last_peered":"2023-12-08T08:38:23.660253+0000","last_clean":"2023-12-08T08:38:23.660253+0000","last_became_active":"2023-12-08T08:38:19.611315+0000","last_became_peered":"2023-12-08T08:38:19.611315+0000","last_unstale":"2023-12-08T08:38:23.660253+0000","last_undegraded":"2023-12-08T08:38:23.660253+0000","last_fullsized":"2023-12-08T08:38:23.660253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:11.612354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851317+0000","last_change":"2023-12-08T08:38:18.634473+0000","last_active":"2023-12-08T08:38:23.851317+0000","last_peered":"2023-12-08T08:38:23.851317+0000","last_clean":"2023-12-08T08:38:23.851317+0000","last_became_active":"2023-12-08T08:38:18.633755+0000","last_became_peered":"2023-12-08T08:38:18.633755+0000","last_unstale":"2023-12-08T08:38:23.851317+0000","last_undegraded":"2023-12-08T08:38:23.851317+0000","last_fullsized":"2023-12-08T08:38:23.851317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:11.067589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.327085+0000","last_change":"2023-12-08T08:38:23.852846+0000","last_active":"2023-12-08T08:38:26.327085+0000","last_peered":"2023-12-08T08:38:26.327085+0000","last_clean":"2023-12-08T08:38:26.327085+0000","last_became_active":"2023-12-08T08:38:23.852393+0000","last_became_peered":"2023-12-08T08:38:23.852393+0000","last_unstale":"2023-12-08T08:38:26.327085+0000","last_undegraded":"2023-12-08T08:38:26.327085+0000","last_fullsized":"2023-12-08T08:38:26.327085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:32:58.184058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655788+0000","last_change":"2023-12-08T08:38:19.634986+0000","last_active":"2023-12-08T08:38:23.655788+0000","last_peered":"2023-12-08T08:38:23.655788+0000","last_clean":"2023-12-08T08:38:23.655788+0000","last_became_active":"2023-12-08T08:38:19.634682+0000","last_became_peered":"2023-12-08T08:38:19.634682+0000","last_unstale":"2023-12-08T08:38:23.655788+0000","last_undegraded":"2023-12-08T08:38:23.655788+0000","last_fullsized":"2023-12-08T08:38:23.655788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661184+0000","last_change":"2023-12-08T08:38:18.631253+0000","last_active":"2023-12-08T08:38:23.661184+0000","last_peered":"2023-12-08T08:38:23.661184+0000","last_clean":"2023-12-08T08:38:23.661184+0000","last_became_active":"2023-12-08T08:38:18.630979+0000","last_became_peered":"2023-12-08T08:38:18.630979+0000","last_unstale":"2023-12-08T08:38:23.661184+0000","last_undegraded":"2023-12-08T08:38:23.661184+0000","last_fullsized":"2023-12-08T08:38:23.661184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:00:00.141773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851733+0000","last_change":"2023-12-08T08:38:08.276815+0000","last_active":"2023-12-08T08:38:23.851733+0000","last_peered":"2023-12-08T08:38:23.851733+0000","last_clean":"2023-12-08T08:38:23.851733+0000","last_became_active":"2023-12-08T08:38:08.275828+0000","last_became_peered":"2023-12-08T08:38:08.275828+0000","last_unstale":"2023-12-08T08:38:23.851733+0000","last_undegraded":"2023-12-08T08:38:23.851733+0000","last_fullsized":"2023-12-08T08:38:23.851733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_clean_scrub_stamp":"2023-12-08T08:38:07.104197+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:35:34.851149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.328098+0000","last_change":"2023-12-08T08:38:23.855031+0000","last_active":"2023-12-08T08:38:26.328098+0000","last_peered":"2023-12-08T08:38:26.328098+0000","last_clean":"2023-12-08T08:38:26.328098+0000","last_became_active":"2023-12-08T08:38:23.854583+0000","last_became_peered":"2023-12-08T08:38:23.854583+0000","last_unstale":"2023-12-08T08:38:26.328098+0000","last_undegraded":"2023-12-08T08:38:26.328098+0000","last_fullsized":"2023-12-08T08:38:26.328098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:01.683162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852252+0000","last_change":"2023-12-08T08:38:19.633600+0000","last_active":"2023-12-08T08:38:23.852252+0000","last_peered":"2023-12-08T08:38:23.852252+0000","last_clean":"2023-12-08T08:38:23.852252+0000","last_became_active":"2023-12-08T08:38:19.633358+0000","last_became_peered":"2023-12-08T08:38:19.633358+0000","last_unstale":"2023-12-08T08:38:23.852252+0000","last_undegraded":"2023-12-08T08:38:23.852252+0000","last_fullsized":"2023-12-08T08:38:23.852252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:13.356849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662546+0000","last_change":"2023-12-08T08:38:18.631378+0000","last_active":"2023-12-08T08:38:23.662546+0000","last_peered":"2023-12-08T08:38:23.662546+0000","last_clean":"2023-12-08T08:38:23.662546+0000","last_became_active":"2023-12-08T08:38:18.631072+0000","last_became_peered":"2023-12-08T08:38:18.631072+0000","last_unstale":"2023-12-08T08:38:23.662546+0000","last_undegraded":"2023-12-08T08:38:23.662546+0000","last_fullsized":"2023-12-08T08:38:23.662546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:27:03.426918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.327456+0000","last_change":"2023-12-08T08:38:23.853285+0000","last_active":"2023-12-08T08:38:26.327456+0000","last_peered":"2023-12-08T08:38:26.327456+0000","last_clean":"2023-12-08T08:38:26.327456+0000","last_became_active":"2023-12-08T08:38:23.852823+0000","last_became_peered":"2023-12-08T08:38:23.852823+0000","last_unstale":"2023-12-08T08:38:26.327456+0000","last_undegraded":"2023-12-08T08:38:26.327456+0000","last_fullsized":"2023-12-08T08:38:26.327456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:59.405196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661432+0000","last_change":"2023-12-08T08:38:19.632038+0000","last_active":"2023-12-08T08:38:23.661432+0000","last_peered":"2023-12-08T08:38:23.661432+0000","last_clean":"2023-12-08T08:38:23.661432+0000","last_became_active":"2023-12-08T08:38:19.630797+0000","last_became_peered":"2023-12-08T08:38:19.630797+0000","last_unstale":"2023-12-08T08:38:23.661432+0000","last_undegraded":"2023-12-08T08:38:23.661432+0000","last_fullsized":"2023-12-08T08:38:23.661432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:46:57.645302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662789+0000","last_change":"2023-12-08T08:38:18.633629+0000","last_active":"2023-12-08T08:38:23.662789+0000","last_peered":"2023-12-08T08:38:23.662789+0000","last_clean":"2023-12-08T08:38:23.662789+0000","last_became_active":"2023-12-08T08:38:18.633224+0000","last_became_peered":"2023-12-08T08:38:18.633224+0000","last_unstale":"2023-12-08T08:38:23.662789+0000","last_undegraded":"2023-12-08T08:38:23.662789+0000","last_fullsized":"2023-12-08T08:38:23.662789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:13:39.901023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.327158+0000","last_change":"2023-12-08T08:38:23.680301+0000","last_active":"2023-12-08T08:38:26.327158+0000","last_peered":"2023-12-08T08:38:26.327158+0000","last_clean":"2023-12-08T08:38:26.327158+0000","last_became_active":"2023-12-08T08:38:23.679652+0000","last_became_peered":"2023-12-08T08:38:23.679652+0000","last_unstale":"2023-12-08T08:38:26.327158+0000","last_undegraded":"2023-12-08T08:38:26.327158+0000","last_fullsized":"2023-12-08T08:38:26.327158+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:18:13.891999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.853813+0000","last_change":"2023-12-08T08:38:19.622603+0000","last_active":"2023-12-08T08:38:23.853813+0000","last_peered":"2023-12-08T08:38:23.853813+0000","last_clean":"2023-12-08T08:38:23.853813+0000","last_became_active":"2023-12-08T08:38:19.622325+0000","last_became_peered":"2023-12-08T08:38:19.622325+0000","last_unstale":"2023-12-08T08:38:23.853813+0000","last_undegraded":"2023-12-08T08:38:23.853813+0000","last_fullsized":"2023-12-08T08:38:23.853813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:01:22.871596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659628+0000","last_change":"2023-12-08T08:38:18.635791+0000","last_active":"2023-12-08T08:38:23.659628+0000","last_peered":"2023-12-08T08:38:23.659628+0000","last_clean":"2023-12-08T08:38:23.659628+0000","last_became_active":"2023-12-08T08:38:18.632773+0000","last_became_peered":"2023-12-08T08:38:18.632773+0000","last_unstale":"2023-12-08T08:38:23.659628+0000","last_undegraded":"2023-12-08T08:38:23.659628+0000","last_fullsized":"2023-12-08T08:38:23.659628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:46:19.770776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.626169+0000","last_change":"2023-12-08T08:38:23.853534+0000","last_active":"2023-12-08T08:38:24.626169+0000","last_peered":"2023-12-08T08:38:24.626169+0000","last_clean":"2023-12-08T08:38:24.626169+0000","last_became_active":"2023-12-08T08:38:23.852476+0000","last_became_peered":"2023-12-08T08:38:23.852476+0000","last_unstale":"2023-12-08T08:38:24.626169+0000","last_undegraded":"2023-12-08T08:38:24.626169+0000","last_fullsized":"2023-12-08T08:38:24.626169+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:01:26.909456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852501+0000","last_change":"2023-12-08T08:38:19.621763+0000","last_active":"2023-12-08T08:38:23.852501+0000","last_peered":"2023-12-08T08:38:23.852501+0000","last_clean":"2023-12-08T08:38:23.852501+0000","last_became_active":"2023-12-08T08:38:19.621504+0000","last_became_peered":"2023-12-08T08:38:19.621504+0000","last_unstale":"2023-12-08T08:38:23.852501+0000","last_undegraded":"2023-12-08T08:38:23.852501+0000","last_fullsized":"2023-12-08T08:38:23.852501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:34:25.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658538+0000","last_change":"2023-12-08T08:38:18.632647+0000","last_active":"2023-12-08T08:38:23.658538+0000","last_peered":"2023-12-08T08:38:23.658538+0000","last_clean":"2023-12-08T08:38:23.658538+0000","last_became_active":"2023-12-08T08:38:18.632398+0000","last_became_peered":"2023-12-08T08:38:18.632398+0000","last_unstale":"2023-12-08T08:38:23.658538+0000","last_undegraded":"2023-12-08T08:38:23.658538+0000","last_fullsized":"2023-12-08T08:38:23.658538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:42:30.138259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608618+0000","last_change":"2023-12-08T08:38:23.674111+0000","last_active":"2023-12-08T08:38:24.608618+0000","last_peered":"2023-12-08T08:38:24.608618+0000","last_clean":"2023-12-08T08:38:24.608618+0000","last_became_active":"2023-12-08T08:38:23.673287+0000","last_became_peered":"2023-12-08T08:38:23.673287+0000","last_unstale":"2023-12-08T08:38:24.608618+0000","last_undegraded":"2023-12-08T08:38:24.608618+0000","last_fullsized":"2023-12-08T08:38:24.608618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:22:01.560111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851975+0000","last_change":"2023-12-08T08:38:19.620974+0000","last_active":"2023-12-08T08:38:23.851975+0000","last_peered":"2023-12-08T08:38:23.851975+0000","last_clean":"2023-12-08T08:38:23.851975+0000","last_became_active":"2023-12-08T08:38:19.619478+0000","last_became_peered":"2023-12-08T08:38:19.619478+0000","last_unstale":"2023-12-08T08:38:23.851975+0000","last_undegraded":"2023-12-08T08:38:23.851975+0000","last_fullsized":"2023-12-08T08:38:23.851975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:55.367335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659793+0000","last_change":"2023-12-08T08:38:18.629485+0000","last_active":"2023-12-08T08:38:23.659793+0000","last_peered":"2023-12-08T08:38:23.659793+0000","last_clean":"2023-12-08T08:38:23.659793+0000","last_became_active":"2023-12-08T08:38:18.629182+0000","last_became_peered":"2023-12-08T08:38:18.629182+0000","last_unstale":"2023-12-08T08:38:23.659793+0000","last_undegraded":"2023-12-08T08:38:23.659793+0000","last_fullsized":"2023-12-08T08:38:23.659793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:47:43.142370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.326907+0000","last_change":"2023-12-08T08:38:23.673762+0000","last_active":"2023-12-08T08:38:26.326907+0000","last_peered":"2023-12-08T08:38:26.326907+0000","last_clean":"2023-12-08T08:38:26.326907+0000","last_became_active":"2023-12-08T08:38:23.673236+0000","last_became_peered":"2023-12-08T08:38:23.673236+0000","last_unstale":"2023-12-08T08:38:26.326907+0000","last_undegraded":"2023-12-08T08:38:26.326907+0000","last_fullsized":"2023-12-08T08:38:26.326907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:34:29.925547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664201+0000","last_change":"2023-12-08T08:38:19.624828+0000","last_active":"2023-12-08T08:38:23.664201+0000","last_peered":"2023-12-08T08:38:23.664201+0000","last_clean":"2023-12-08T08:38:23.664201+0000","last_became_active":"2023-12-08T08:38:19.624249+0000","last_became_peered":"2023-12-08T08:38:19.624249+0000","last_unstale":"2023-12-08T08:38:23.664201+0000","last_undegraded":"2023-12-08T08:38:23.664201+0000","last_fullsized":"2023-12-08T08:38:23.664201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.663711+0000","last_change":"2023-12-08T08:38:18.624832+0000","last_active":"2023-12-08T08:38:23.663711+0000","last_peered":"2023-12-08T08:38:23.663711+0000","last_clean":"2023-12-08T08:38:23.663711+0000","last_became_active":"2023-12-08T08:38:18.624580+0000","last_became_peered":"2023-12-08T08:38:18.624580+0000","last_unstale":"2023-12-08T08:38:23.663711+0000","last_undegraded":"2023-12-08T08:38:23.663711+0000","last_fullsized":"2023-12-08T08:38:23.663711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:13:39.901023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576884+0000","last_change":"2023-12-08T08:38:23.856627+0000","last_active":"2023-12-08T08:38:25.576884+0000","last_peered":"2023-12-08T08:38:25.576884+0000","last_clean":"2023-12-08T08:38:25.576884+0000","last_became_active":"2023-12-08T08:38:23.855845+0000","last_became_peered":"2023-12-08T08:38:23.855845+0000","last_unstale":"2023-12-08T08:38:25.576884+0000","last_undegraded":"2023-12-08T08:38:25.576884+0000","last_fullsized":"2023-12-08T08:38:25.576884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:20:54.195474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852053+0000","last_change":"2023-12-08T08:38:19.621686+0000","last_active":"2023-12-08T08:38:23.852053+0000","last_peered":"2023-12-08T08:38:23.852053+0000","last_clean":"2023-12-08T08:38:23.852053+0000","last_became_active":"2023-12-08T08:38:19.621366+0000","last_became_peered":"2023-12-08T08:38:19.621366+0000","last_unstale":"2023-12-08T08:38:23.852053+0000","last_undegraded":"2023-12-08T08:38:23.852053+0000","last_fullsized":"2023-12-08T08:38:23.852053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:18.334775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661960+0000","last_change":"2023-12-08T08:38:18.637940+0000","last_active":"2023-12-08T08:38:23.661960+0000","last_peered":"2023-12-08T08:38:23.661960+0000","last_clean":"2023-12-08T08:38:23.661960+0000","last_became_active":"2023-12-08T08:38:18.637552+0000","last_became_peered":"2023-12-08T08:38:18.637552+0000","last_unstale":"2023-12-08T08:38:23.661960+0000","last_undegraded":"2023-12-08T08:38:23.661960+0000","last_fullsized":"2023-12-08T08:38:23.661960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:14.342540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576721+0000","last_change":"2023-12-08T08:38:23.852937+0000","last_active":"2023-12-08T08:38:25.576721+0000","last_peered":"2023-12-08T08:38:25.576721+0000","last_clean":"2023-12-08T08:38:25.576721+0000","last_became_active":"2023-12-08T08:38:23.852228+0000","last_became_peered":"2023-12-08T08:38:23.852228+0000","last_unstale":"2023-12-08T08:38:25.576721+0000","last_undegraded":"2023-12-08T08:38:25.576721+0000","last_fullsized":"2023-12-08T08:38:25.576721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:46:14.364194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664273+0000","last_change":"2023-12-08T08:38:19.618681+0000","last_active":"2023-12-08T08:38:23.664273+0000","last_peered":"2023-12-08T08:38:23.664273+0000","last_clean":"2023-12-08T08:38:23.664273+0000","last_became_active":"2023-12-08T08:38:19.618102+0000","last_became_peered":"2023-12-08T08:38:19.618102+0000","last_unstale":"2023-12-08T08:38:23.664273+0000","last_undegraded":"2023-12-08T08:38:23.664273+0000","last_fullsized":"2023-12-08T08:38:23.664273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:00:01.146409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661116+0000","last_change":"2023-12-08T08:38:18.636521+0000","last_active":"2023-12-08T08:38:23.661116+0000","last_peered":"2023-12-08T08:38:23.661116+0000","last_clean":"2023-12-08T08:38:23.661116+0000","last_became_active":"2023-12-08T08:38:18.636242+0000","last_became_peered":"2023-12-08T08:38:18.636242+0000","last_unstale":"2023-12-08T08:38:23.661116+0000","last_undegraded":"2023-12-08T08:38:23.661116+0000","last_fullsized":"2023-12-08T08:38:23.661116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.626765+0000","last_change":"2023-12-08T08:38:23.855694+0000","last_active":"2023-12-08T08:38:24.626765+0000","last_peered":"2023-12-08T08:38:24.626765+0000","last_clean":"2023-12-08T08:38:24.626765+0000","last_became_active":"2023-12-08T08:38:23.855342+0000","last_became_peered":"2023-12-08T08:38:23.855342+0000","last_unstale":"2023-12-08T08:38:24.626765+0000","last_undegraded":"2023-12-08T08:38:24.626765+0000","last_fullsized":"2023-12-08T08:38:24.626765+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T20:21:44.003574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852120+0000","last_change":"2023-12-08T08:38:19.627826+0000","last_active":"2023-12-08T08:38:23.852120+0000","last_peered":"2023-12-08T08:38:23.852120+0000","last_clean":"2023-12-08T08:38:23.852120+0000","last_became_active":"2023-12-08T08:38:19.627154+0000","last_became_peered":"2023-12-08T08:38:19.627154+0000","last_unstale":"2023-12-08T08:38:23.852120+0000","last_undegraded":"2023-12-08T08:38:23.852120+0000","last_fullsized":"2023-12-08T08:38:23.852120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:11.797914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:29.035583+0000","last_change":"2023-12-08T08:38:18.630599+0000","last_active":"2023-12-08T08:38:29.035583+0000","last_peered":"2023-12-08T08:38:29.035583+0000","last_clean":"2023-12-08T08:38:29.035583+0000","last_became_active":"2023-12-08T08:38:18.630349+0000","last_became_peered":"2023-12-08T08:38:18.630349+0000","last_unstale":"2023-12-08T08:38:29.035583+0000","last_undegraded":"2023-12-08T08:38:29.035583+0000","last_fullsized":"2023-12-08T08:38:29.035583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:36:34.339195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3839,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.827898+0000","last_change":"2023-12-08T08:38:23.676381+0000","last_active":"2023-12-08T08:38:25.827898+0000","last_peered":"2023-12-08T08:38:25.827898+0000","last_clean":"2023-12-08T08:38:25.827898+0000","last_became_active":"2023-12-08T08:38:23.675561+0000","last_became_peered":"2023-12-08T08:38:23.675561+0000","last_unstale":"2023-12-08T08:38:25.827898+0000","last_undegraded":"2023-12-08T08:38:25.827898+0000","last_fullsized":"2023-12-08T08:38:25.827898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:22.372635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659110+0000","last_change":"2023-12-08T08:38:19.615536+0000","last_active":"2023-12-08T08:38:23.659110+0000","last_peered":"2023-12-08T08:38:23.659110+0000","last_clean":"2023-12-08T08:38:23.659110+0000","last_became_active":"2023-12-08T08:38:19.615243+0000","last_became_peered":"2023-12-08T08:38:19.615243+0000","last_unstale":"2023-12-08T08:38:23.659110+0000","last_undegraded":"2023-12-08T08:38:23.659110+0000","last_fullsized":"2023-12-08T08:38:23.659110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:13:40.905659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662690+0000","last_change":"2023-12-08T08:38:18.632950+0000","last_active":"2023-12-08T08:38:23.662690+0000","last_peered":"2023-12-08T08:38:23.662690+0000","last_clean":"2023-12-08T08:38:23.662690+0000","last_became_active":"2023-12-08T08:38:18.632652+0000","last_became_peered":"2023-12-08T08:38:18.632652+0000","last_unstale":"2023-12-08T08:38:23.662690+0000","last_undegraded":"2023-12-08T08:38:23.662690+0000","last_fullsized":"2023-12-08T08:38:23.662690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:36:34.339195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609288+0000","last_change":"2023-12-08T08:38:23.674379+0000","last_active":"2023-12-08T08:38:24.609288+0000","last_peered":"2023-12-08T08:38:24.609288+0000","last_clean":"2023-12-08T08:38:24.609288+0000","last_became_active":"2023-12-08T08:38:23.673581+0000","last_became_peered":"2023-12-08T08:38:23.673581+0000","last_unstale":"2023-12-08T08:38:24.609288+0000","last_undegraded":"2023-12-08T08:38:24.609288+0000","last_fullsized":"2023-12-08T08:38:24.609288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:01.683162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659853+0000","last_change":"2023-12-08T08:38:19.633344+0000","last_active":"2023-12-08T08:38:23.659853+0000","last_peered":"2023-12-08T08:38:23.659853+0000","last_clean":"2023-12-08T08:38:23.659853+0000","last_became_active":"2023-12-08T08:38:19.633035+0000","last_became_peered":"2023-12-08T08:38:19.633035+0000","last_unstale":"2023-12-08T08:38:23.659853+0000","last_undegraded":"2023-12-08T08:38:23.659853+0000","last_fullsized":"2023-12-08T08:38:23.659853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661808+0000","last_change":"2023-12-08T08:38:18.631126+0000","last_active":"2023-12-08T08:38:23.661808+0000","last_peered":"2023-12-08T08:38:23.661808+0000","last_clean":"2023-12-08T08:38:23.661808+0000","last_became_active":"2023-12-08T08:38:18.630841+0000","last_became_peered":"2023-12-08T08:38:18.630841+0000","last_unstale":"2023-12-08T08:38:23.661808+0000","last_undegraded":"2023-12-08T08:38:23.661808+0000","last_fullsized":"2023-12-08T08:38:23.661808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:07.497079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576255+0000","last_change":"2023-12-08T08:38:23.678178+0000","last_active":"2023-12-08T08:38:25.576255+0000","last_peered":"2023-12-08T08:38:25.576255+0000","last_clean":"2023-12-08T08:38:25.576255+0000","last_became_active":"2023-12-08T08:38:23.677800+0000","last_became_peered":"2023-12-08T08:38:23.677800+0000","last_unstale":"2023-12-08T08:38:25.576255+0000","last_undegraded":"2023-12-08T08:38:25.576255+0000","last_fullsized":"2023-12-08T08:38:25.576255+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:22:37.463722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661701+0000","last_change":"2023-12-08T08:38:19.631623+0000","last_active":"2023-12-08T08:38:23.661701+0000","last_peered":"2023-12-08T08:38:23.661701+0000","last_clean":"2023-12-08T08:38:23.661701+0000","last_became_active":"2023-12-08T08:38:19.630737+0000","last_became_peered":"2023-12-08T08:38:19.630737+0000","last_unstale":"2023-12-08T08:38:23.661701+0000","last_undegraded":"2023-12-08T08:38:23.661701+0000","last_fullsized":"2023-12-08T08:38:23.661701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:18.334775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665261+0000","last_change":"2023-12-08T08:38:18.630876+0000","last_active":"2023-12-08T08:38:23.665261+0000","last_peered":"2023-12-08T08:38:23.665261+0000","last_clean":"2023-12-08T08:38:23.665261+0000","last_became_active":"2023-12-08T08:38:18.630581+0000","last_became_peered":"2023-12-08T08:38:18.630581+0000","last_unstale":"2023-12-08T08:38:23.665261+0000","last_undegraded":"2023-12-08T08:38:23.665261+0000","last_fullsized":"2023-12-08T08:38:23.665261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576234+0000","last_change":"2023-12-08T08:38:23.855920+0000","last_active":"2023-12-08T08:38:25.576234+0000","last_peered":"2023-12-08T08:38:25.576234+0000","last_clean":"2023-12-08T08:38:25.576234+0000","last_became_active":"2023-12-08T08:38:23.855491+0000","last_became_peered":"2023-12-08T08:38:23.855491+0000","last_unstale":"2023-12-08T08:38:25.576234+0000","last_undegraded":"2023-12-08T08:38:25.576234+0000","last_fullsized":"2023-12-08T08:38:25.576234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:10:52.070614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.326821+0000","last_change":"2023-12-08T08:38:23.855660+0000","last_active":"2023-12-08T08:38:26.326821+0000","last_peered":"2023-12-08T08:38:26.326821+0000","last_clean":"2023-12-08T08:38:26.326821+0000","last_became_active":"2023-12-08T08:38:23.855190+0000","last_became_peered":"2023-12-08T08:38:23.855190+0000","last_unstale":"2023-12-08T08:38:26.326821+0000","last_undegraded":"2023-12-08T08:38:26.326821+0000","last_fullsized":"2023-12-08T08:38:26.326821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:48.357070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664507+0000","last_change":"2023-12-08T08:38:19.620116+0000","last_active":"2023-12-08T08:38:23.664507+0000","last_peered":"2023-12-08T08:38:23.664507+0000","last_clean":"2023-12-08T08:38:23.664507+0000","last_became_active":"2023-12-08T08:38:19.619412+0000","last_became_peered":"2023-12-08T08:38:19.619412+0000","last_unstale":"2023-12-08T08:38:23.664507+0000","last_undegraded":"2023-12-08T08:38:23.664507+0000","last_fullsized":"2023-12-08T08:38:23.664507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661919+0000","last_change":"2023-12-08T08:38:18.630228+0000","last_active":"2023-12-08T08:38:23.661919+0000","last_peered":"2023-12-08T08:38:23.661919+0000","last_clean":"2023-12-08T08:38:23.661919+0000","last_became_active":"2023-12-08T08:38:18.629445+0000","last_became_peered":"2023-12-08T08:38:18.629445+0000","last_unstale":"2023-12-08T08:38:23.661919+0000","last_undegraded":"2023-12-08T08:38:23.661919+0000","last_fullsized":"2023-12-08T08:38:23.661919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:57.526979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.328319+0000","last_change":"2023-12-08T08:38:23.678105+0000","last_active":"2023-12-08T08:38:26.328319+0000","last_peered":"2023-12-08T08:38:26.328319+0000","last_clean":"2023-12-08T08:38:26.328319+0000","last_became_active":"2023-12-08T08:38:23.677716+0000","last_became_peered":"2023-12-08T08:38:23.677716+0000","last_unstale":"2023-12-08T08:38:26.328319+0000","last_undegraded":"2023-12-08T08:38:26.328319+0000","last_fullsized":"2023-12-08T08:38:26.328319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:15.835775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655896+0000","last_change":"2023-12-08T08:38:19.628959+0000","last_active":"2023-12-08T08:38:23.655896+0000","last_peered":"2023-12-08T08:38:23.655896+0000","last_clean":"2023-12-08T08:38:23.655896+0000","last_became_active":"2023-12-08T08:38:19.628662+0000","last_became_peered":"2023-12-08T08:38:19.628662+0000","last_unstale":"2023-12-08T08:38:23.655896+0000","last_undegraded":"2023-12-08T08:38:23.655896+0000","last_fullsized":"2023-12-08T08:38:23.655896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658005+0000","last_change":"2023-12-08T08:38:18.637772+0000","last_active":"2023-12-08T08:38:23.658005+0000","last_peered":"2023-12-08T08:38:23.658005+0000","last_clean":"2023-12-08T08:38:23.658005+0000","last_became_active":"2023-12-08T08:38:18.637318+0000","last_became_peered":"2023-12-08T08:38:18.637318+0000","last_unstale":"2023-12-08T08:38:23.658005+0000","last_undegraded":"2023-12-08T08:38:23.658005+0000","last_fullsized":"2023-12-08T08:38:23.658005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:11.067589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664203+0000","last_change":"2023-12-08T08:38:18.630848+0000","last_active":"2023-12-08T08:38:23.664203+0000","last_peered":"2023-12-08T08:38:23.664203+0000","last_clean":"2023-12-08T08:38:23.664203+0000","last_became_active":"2023-12-08T08:38:18.629736+0000","last_became_peered":"2023-12-08T08:38:18.629736+0000","last_unstale":"2023-12-08T08:38:23.664203+0000","last_undegraded":"2023-12-08T08:38:23.664203+0000","last_fullsized":"2023-12-08T08:38:23.664203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:10.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661026+0000","last_change":"2023-12-08T08:38:19.612508+0000","last_active":"2023-12-08T08:38:23.661026+0000","last_peered":"2023-12-08T08:38:23.661026+0000","last_clean":"2023-12-08T08:38:23.661026+0000","last_became_active":"2023-12-08T08:38:19.612212+0000","last_became_peered":"2023-12-08T08:38:19.612212+0000","last_unstale":"2023-12-08T08:38:23.661026+0000","last_undegraded":"2023-12-08T08:38:23.661026+0000","last_fullsized":"2023-12-08T08:38:23.661026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608202+0000","last_change":"2023-12-08T08:38:23.678309+0000","last_active":"2023-12-08T08:38:24.608202+0000","last_peered":"2023-12-08T08:38:24.608202+0000","last_clean":"2023-12-08T08:38:24.608202+0000","last_became_active":"2023-12-08T08:38:23.677882+0000","last_became_peered":"2023-12-08T08:38:23.677882+0000","last_unstale":"2023-12-08T08:38:24.608202+0000","last_undegraded":"2023-12-08T08:38:24.608202+0000","last_fullsized":"2023-12-08T08:38:24.608202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:15.118488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655793+0000","last_change":"2023-12-08T08:38:18.638936+0000","last_active":"2023-12-08T08:38:23.655793+0000","last_peered":"2023-12-08T08:38:23.655793+0000","last_clean":"2023-12-08T08:38:23.655793+0000","last_became_active":"2023-12-08T08:38:18.638058+0000","last_became_peered":"2023-12-08T08:38:18.638058+0000","last_unstale":"2023-12-08T08:38:23.655793+0000","last_undegraded":"2023-12-08T08:38:23.655793+0000","last_fullsized":"2023-12-08T08:38:23.655793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:10.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658137+0000","last_change":"2023-12-08T08:38:19.619468+0000","last_active":"2023-12-08T08:38:23.658137+0000","last_peered":"2023-12-08T08:38:23.658137+0000","last_clean":"2023-12-08T08:38:23.658137+0000","last_became_active":"2023-12-08T08:38:19.619177+0000","last_became_peered":"2023-12-08T08:38:19.619177+0000","last_unstale":"2023-12-08T08:38:23.658137+0000","last_undegraded":"2023-12-08T08:38:23.658137+0000","last_fullsized":"2023-12-08T08:38:23.658137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:46:20.775412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576589+0000","last_change":"2023-12-08T08:38:23.679648+0000","last_active":"2023-12-08T08:38:25.576589+0000","last_peered":"2023-12-08T08:38:25.576589+0000","last_clean":"2023-12-08T08:38:25.576589+0000","last_became_active":"2023-12-08T08:38:23.679229+0000","last_became_peered":"2023-12-08T08:38:23.679229+0000","last_unstale":"2023-12-08T08:38:25.576589+0000","last_undegraded":"2023-12-08T08:38:25.576589+0000","last_fullsized":"2023-12-08T08:38:25.576589+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:53:05.969247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.663580+0000","last_change":"2023-12-08T08:38:19.624993+0000","last_active":"2023-12-08T08:38:23.663580+0000","last_peered":"2023-12-08T08:38:23.663580+0000","last_clean":"2023-12-08T08:38:23.663580+0000","last_became_active":"2023-12-08T08:38:19.624569+0000","last_became_peered":"2023-12-08T08:38:19.624569+0000","last_unstale":"2023-12-08T08:38:23.663580+0000","last_undegraded":"2023-12-08T08:38:23.663580+0000","last_fullsized":"2023-12-08T08:38:23.663580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662648+0000","last_change":"2023-12-08T08:38:19.631559+0000","last_active":"2023-12-08T08:38:23.662648+0000","last_peered":"2023-12-08T08:38:23.662648+0000","last_clean":"2023-12-08T08:38:23.662648+0000","last_became_active":"2023-12-08T08:38:19.631225+0000","last_became_peered":"2023-12-08T08:38:19.631225+0000","last_unstale":"2023-12-08T08:38:23.662648+0000","last_undegraded":"2023-12-08T08:38:23.662648+0000","last_fullsized":"2023-12-08T08:38:23.662648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:12.072225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661866+0000","last_change":"2023-12-08T08:38:19.631476+0000","last_active":"2023-12-08T08:38:23.661866+0000","last_peered":"2023-12-08T08:38:23.661866+0000","last_clean":"2023-12-08T08:38:23.661866+0000","last_became_active":"2023-12-08T08:38:19.630628+0000","last_became_peered":"2023-12-08T08:38:19.630628+0000","last_unstale":"2023-12-08T08:38:23.661866+0000","last_undegraded":"2023-12-08T08:38:23.661866+0000","last_fullsized":"2023-12-08T08:38:23.661866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:55.367335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.660943+0000","last_change":"2023-12-08T08:38:19.623252+0000","last_active":"2023-12-08T08:38:23.660943+0000","last_peered":"2023-12-08T08:38:23.660943+0000","last_clean":"2023-12-08T08:38:23.660943+0000","last_became_active":"2023-12-08T08:38:19.622970+0000","last_became_peered":"2023-12-08T08:38:19.622970+0000","last_unstale":"2023-12-08T08:38:23.660943+0000","last_undegraded":"2023-12-08T08:38:23.660943+0000","last_fullsized":"2023-12-08T08:38:23.660943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.657026+0000","last_change":"2023-12-08T08:38:19.621907+0000","last_active":"2023-12-08T08:38:23.657026+0000","last_peered":"2023-12-08T08:38:23.657026+0000","last_clean":"2023-12-08T08:38:23.657026+0000","last_became_active":"2023-12-08T08:38:19.621597+0000","last_became_peered":"2023-12-08T08:38:19.621597+0000","last_unstale":"2023-12-08T08:38:23.657026+0000","last_undegraded":"2023-12-08T08:38:23.657026+0000","last_fullsized":"2023-12-08T08:38:23.657026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:55.367335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662589+0000","last_change":"2023-12-08T08:38:19.621460+0000","last_active":"2023-12-08T08:38:23.662589+0000","last_peered":"2023-12-08T08:38:23.662589+0000","last_clean":"2023-12-08T08:38:23.662589+0000","last_became_active":"2023-12-08T08:38:19.621090+0000","last_became_peered":"2023-12-08T08:38:19.621090+0000","last_unstale":"2023-12-08T08:38:23.662589+0000","last_undegraded":"2023-12-08T08:38:23.662589+0000","last_fullsized":"2023-12-08T08:38:23.662589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665456+0000","last_change":"2023-12-08T08:38:19.620846+0000","last_active":"2023-12-08T08:38:23.665456+0000","last_peered":"2023-12-08T08:38:23.665456+0000","last_clean":"2023-12-08T08:38:23.665456+0000","last_became_active":"2023-12-08T08:38:19.620575+0000","last_became_peered":"2023-12-08T08:38:19.620575+0000","last_unstale":"2023-12-08T08:38:23.665456+0000","last_undegraded":"2023-12-08T08:38:23.665456+0000","last_fullsized":"2023-12-08T08:38:23.665456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656076+0000","last_change":"2023-12-08T08:38:19.623987+0000","last_active":"2023-12-08T08:38:23.656076+0000","last_peered":"2023-12-08T08:38:23.656076+0000","last_clean":"2023-12-08T08:38:23.656076+0000","last_became_active":"2023-12-08T08:38:19.623637+0000","last_became_peered":"2023-12-08T08:38:19.623637+0000","last_unstale":"2023-12-08T08:38:23.656076+0000","last_undegraded":"2023-12-08T08:38:23.656076+0000","last_fullsized":"2023-12-08T08:38:23.656076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:43.336635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661985+0000","last_change":"2023-12-08T08:38:19.627606+0000","last_active":"2023-12-08T08:38:23.661985+0000","last_peered":"2023-12-08T08:38:23.661985+0000","last_clean":"2023-12-08T08:38:23.661985+0000","last_became_active":"2023-12-08T08:38:19.627345+0000","last_became_peered":"2023-12-08T08:38:19.627345+0000","last_unstale":"2023-12-08T08:38:23.661985+0000","last_undegraded":"2023-12-08T08:38:23.661985+0000","last_fullsized":"2023-12-08T08:38:23.661985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:43.336635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.657865+0000","last_change":"2023-12-08T08:38:19.623195+0000","last_active":"2023-12-08T08:38:23.657865+0000","last_peered":"2023-12-08T08:38:23.657865+0000","last_clean":"2023-12-08T08:38:23.657865+0000","last_became_active":"2023-12-08T08:38:19.622896+0000","last_became_peered":"2023-12-08T08:38:19.622896+0000","last_unstale":"2023-12-08T08:38:23.657865+0000","last_undegraded":"2023-12-08T08:38:23.657865+0000","last_fullsized":"2023-12-08T08:38:23.657865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:59:11.073543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661771+0000","last_change":"2023-12-08T08:38:19.632017+0000","last_active":"2023-12-08T08:38:23.661771+0000","last_peered":"2023-12-08T08:38:23.661771+0000","last_clean":"2023-12-08T08:38:23.661771+0000","last_became_active":"2023-12-08T08:38:19.631771+0000","last_became_peered":"2023-12-08T08:38:19.631771+0000","last_unstale":"2023-12-08T08:38:23.661771+0000","last_undegraded":"2023-12-08T08:38:23.661771+0000","last_fullsized":"2023-12-08T08:38:23.661771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:59:11.073543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.660849+0000","last_change":"2023-12-08T08:38:19.618872+0000","last_active":"2023-12-08T08:38:23.660849+0000","last_peered":"2023-12-08T08:38:23.660849+0000","last_clean":"2023-12-08T08:38:23.660849+0000","last_became_active":"2023-12-08T08:38:19.617964+0000","last_became_peered":"2023-12-08T08:38:19.617964+0000","last_unstale":"2023-12-08T08:38:23.660849+0000","last_undegraded":"2023-12-08T08:38:23.660849+0000","last_fullsized":"2023-12-08T08:38:23.660849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665351+0000","last_change":"2023-12-08T08:38:19.622440+0000","last_active":"2023-12-08T08:38:23.665351+0000","last_peered":"2023-12-08T08:38:23.665351+0000","last_clean":"2023-12-08T08:38:23.665351+0000","last_became_active":"2023-12-08T08:38:19.621997+0000","last_became_peered":"2023-12-08T08:38:19.621997+0000","last_unstale":"2023-12-08T08:38:23.665351+0000","last_undegraded":"2023-12-08T08:38:23.665351+0000","last_fullsized":"2023-12-08T08:38:23.665351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852435+0000","last_change":"2023-12-08T08:38:19.621395+0000","last_active":"2023-12-08T08:38:23.852435+0000","last_peered":"2023-12-08T08:38:23.852435+0000","last_clean":"2023-12-08T08:38:23.852435+0000","last_became_active":"2023-12-08T08:38:19.621148+0000","last_became_peered":"2023-12-08T08:38:19.621148+0000","last_unstale":"2023-12-08T08:38:23.852435+0000","last_undegraded":"2023-12-08T08:38:23.852435+0000","last_fullsized":"2023-12-08T08:38:23.852435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:44.319210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659499+0000","last_change":"2023-12-08T08:38:19.626108+0000","last_active":"2023-12-08T08:38:23.659499+0000","last_peered":"2023-12-08T08:38:23.659499+0000","last_clean":"2023-12-08T08:38:23.659499+0000","last_became_active":"2023-12-08T08:38:19.625773+0000","last_became_peered":"2023-12-08T08:38:19.625773+0000","last_unstale":"2023-12-08T08:38:23.659499+0000","last_undegraded":"2023-12-08T08:38:23.659499+0000","last_fullsized":"2023-12-08T08:38:23.659499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:05:08.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662077+0000","last_change":"2023-12-08T08:38:19.631410+0000","last_active":"2023-12-08T08:38:23.662077+0000","last_peered":"2023-12-08T08:38:23.662077+0000","last_clean":"2023-12-08T08:38:23.662077+0000","last_became_active":"2023-12-08T08:38:19.630653+0000","last_became_peered":"2023-12-08T08:38:19.630653+0000","last_unstale":"2023-12-08T08:38:23.662077+0000","last_undegraded":"2023-12-08T08:38:23.662077+0000","last_fullsized":"2023-12-08T08:38:23.662077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:44.319210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661818+0000","last_change":"2023-12-08T08:38:19.630190+0000","last_active":"2023-12-08T08:38:23.661818+0000","last_peered":"2023-12-08T08:38:23.661818+0000","last_clean":"2023-12-08T08:38:23.661818+0000","last_became_active":"2023-12-08T08:38:19.629923+0000","last_became_peered":"2023-12-08T08:38:19.629923+0000","last_unstale":"2023-12-08T08:38:23.661818+0000","last_undegraded":"2023-12-08T08:38:23.661818+0000","last_fullsized":"2023-12-08T08:38:23.661818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664602+0000","last_change":"2023-12-08T08:38:19.620155+0000","last_active":"2023-12-08T08:38:23.664602+0000","last_peered":"2023-12-08T08:38:23.664602+0000","last_clean":"2023-12-08T08:38:23.664602+0000","last_became_active":"2023-12-08T08:38:19.619814+0000","last_became_peered":"2023-12-08T08:38:19.619814+0000","last_unstale":"2023-12-08T08:38:23.664602+0000","last_undegraded":"2023-12-08T08:38:23.664602+0000","last_fullsized":"2023-12-08T08:38:23.664602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664183+0000","last_change":"2023-12-08T08:38:19.616693+0000","last_active":"2023-12-08T08:38:23.664183+0000","last_peered":"2023-12-08T08:38:23.664183+0000","last_clean":"2023-12-08T08:38:23.664183+0000","last_became_active":"2023-12-08T08:38:19.616369+0000","last_became_peered":"2023-12-08T08:38:19.616369+0000","last_unstale":"2023-12-08T08:38:23.664183+0000","last_undegraded":"2023-12-08T08:38:23.664183+0000","last_fullsized":"2023-12-08T08:38:23.664183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:38.322348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655532+0000","last_change":"2023-12-08T08:38:19.625929+0000","last_active":"2023-12-08T08:38:23.655532+0000","last_peered":"2023-12-08T08:38:23.655532+0000","last_clean":"2023-12-08T08:38:23.655532+0000","last_became_active":"2023-12-08T08:38:19.624245+0000","last_became_peered":"2023-12-08T08:38:19.624245+0000","last_unstale":"2023-12-08T08:38:23.655532+0000","last_undegraded":"2023-12-08T08:38:23.655532+0000","last_fullsized":"2023-12-08T08:38:23.655532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659198+0000","last_change":"2023-12-08T08:38:19.625517+0000","last_active":"2023-12-08T08:38:23.659198+0000","last_peered":"2023-12-08T08:38:23.659198+0000","last_clean":"2023-12-08T08:38:23.659198+0000","last_became_active":"2023-12-08T08:38:19.625079+0000","last_became_peered":"2023-12-08T08:38:19.625079+0000","last_unstale":"2023-12-08T08:38:23.659198+0000","last_undegraded":"2023-12-08T08:38:23.659198+0000","last_fullsized":"2023-12-08T08:38:23.659198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:12.072225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665048+0000","last_change":"2023-12-08T08:38:19.632480+0000","last_active":"2023-12-08T08:38:23.665048+0000","last_peered":"2023-12-08T08:38:23.665048+0000","last_clean":"2023-12-08T08:38:23.665048+0000","last_became_active":"2023-12-08T08:38:19.632215+0000","last_became_peered":"2023-12-08T08:38:19.632215+0000","last_unstale":"2023-12-08T08:38:23.665048+0000","last_undegraded":"2023-12-08T08:38:23.665048+0000","last_fullsized":"2023-12-08T08:38:23.665048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665415+0000","last_change":"2023-12-08T08:38:19.635234+0000","last_active":"2023-12-08T08:38:23.665415+0000","last_peered":"2023-12-08T08:38:23.665415+0000","last_clean":"2023-12-08T08:38:23.665415+0000","last_became_active":"2023-12-08T08:38:19.634638+0000","last_became_peered":"2023-12-08T08:38:19.634638+0000","last_unstale":"2023-12-08T08:38:23.665415+0000","last_undegraded":"2023-12-08T08:38:23.665415+0000","last_fullsized":"2023-12-08T08:38:23.665415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.853718+0000","last_change":"2023-12-08T08:38:19.633557+0000","last_active":"2023-12-08T08:38:23.853718+0000","last_peered":"2023-12-08T08:38:23.853718+0000","last_clean":"2023-12-08T08:38:23.853718+0000","last_became_active":"2023-12-08T08:38:19.633168+0000","last_became_peered":"2023-12-08T08:38:19.633168+0000","last_unstale":"2023-12-08T08:38:23.853718+0000","last_undegraded":"2023-12-08T08:38:23.853718+0000","last_fullsized":"2023-12-08T08:38:23.853718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:18:09.854138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655258+0000","last_change":"2023-12-08T08:38:19.635104+0000","last_active":"2023-12-08T08:38:23.655258+0000","last_peered":"2023-12-08T08:38:23.655258+0000","last_clean":"2023-12-08T08:38:23.655258+0000","last_became_active":"2023-12-08T08:38:19.634711+0000","last_became_peered":"2023-12-08T08:38:19.634711+0000","last_unstale":"2023-12-08T08:38:23.655258+0000","last_undegraded":"2023-12-08T08:38:23.655258+0000","last_fullsized":"2023-12-08T08:38:23.655258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851929+0000","last_change":"2023-12-08T08:38:19.618128+0000","last_active":"2023-12-08T08:38:23.851929+0000","last_peered":"2023-12-08T08:38:23.851929+0000","last_clean":"2023-12-08T08:38:23.851929+0000","last_became_active":"2023-12-08T08:38:19.617870+0000","last_became_peered":"2023-12-08T08:38:19.617870+0000","last_unstale":"2023-12-08T08:38:23.851929+0000","last_undegraded":"2023-12-08T08:38:23.851929+0000","last_fullsized":"2023-12-08T08:38:23.851929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:32.116348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661665+0000","last_change":"2023-12-08T08:38:19.631953+0000","last_active":"2023-12-08T08:38:23.661665+0000","last_peered":"2023-12-08T08:38:23.661665+0000","last_clean":"2023-12-08T08:38:23.661665+0000","last_became_active":"2023-12-08T08:38:19.631638+0000","last_became_peered":"2023-12-08T08:38:19.631638+0000","last_unstale":"2023-12-08T08:38:23.661665+0000","last_undegraded":"2023-12-08T08:38:23.661665+0000","last_fullsized":"2023-12-08T08:38:23.661665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851995+0000","last_change":"2023-12-08T08:38:19.621369+0000","last_active":"2023-12-08T08:38:23.851995+0000","last_peered":"2023-12-08T08:38:23.851995+0000","last_clean":"2023-12-08T08:38:23.851995+0000","last_became_active":"2023-12-08T08:38:19.621103+0000","last_became_peered":"2023-12-08T08:38:19.621103+0000","last_unstale":"2023-12-08T08:38:23.851995+0000","last_undegraded":"2023-12-08T08:38:23.851995+0000","last_fullsized":"2023-12-08T08:38:23.851995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:46:57.645302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661240+0000","last_change":"2023-12-08T08:38:19.630541+0000","last_active":"2023-12-08T08:38:23.661240+0000","last_peered":"2023-12-08T08:38:23.661240+0000","last_clean":"2023-12-08T08:38:23.661240+0000","last_became_active":"2023-12-08T08:38:19.630175+0000","last_became_peered":"2023-12-08T08:38:19.630175+0000","last_unstale":"2023-12-08T08:38:23.661240+0000","last_undegraded":"2023-12-08T08:38:23.661240+0000","last_fullsized":"2023-12-08T08:38:23.661240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:49:29.898351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664269+0000","last_change":"2023-12-08T08:38:19.626884+0000","last_active":"2023-12-08T08:38:23.664269+0000","last_peered":"2023-12-08T08:38:23.664269+0000","last_clean":"2023-12-08T08:38:23.664269+0000","last_became_active":"2023-12-08T08:38:19.626525+0000","last_became_peered":"2023-12-08T08:38:19.626525+0000","last_unstale":"2023-12-08T08:38:23.664269+0000","last_undegraded":"2023-12-08T08:38:23.664269+0000","last_fullsized":"2023-12-08T08:38:23.664269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664251+0000","last_change":"2023-12-08T08:38:19.621125+0000","last_active":"2023-12-08T08:38:23.664251+0000","last_peered":"2023-12-08T08:38:23.664251+0000","last_clean":"2023-12-08T08:38:23.664251+0000","last_became_active":"2023-12-08T08:38:19.620831+0000","last_became_peered":"2023-12-08T08:38:19.620831+0000","last_unstale":"2023-12-08T08:38:23.664251+0000","last_undegraded":"2023-12-08T08:38:23.664251+0000","last_fullsized":"2023-12-08T08:38:23.664251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852760+0000","last_change":"2023-12-08T08:38:19.624484+0000","last_active":"2023-12-08T08:38:23.852760+0000","last_peered":"2023-12-08T08:38:23.852760+0000","last_clean":"2023-12-08T08:38:23.852760+0000","last_became_active":"2023-12-08T08:38:19.623921+0000","last_became_peered":"2023-12-08T08:38:19.623921+0000","last_unstale":"2023-12-08T08:38:23.852760+0000","last_undegraded":"2023-12-08T08:38:23.852760+0000","last_fullsized":"2023-12-08T08:38:23.852760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:32:54.146197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4637,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":99918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":99952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":690798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.625}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-08T08:38:42.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-08T08:38:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.858+0000 7f022fd89640 1 Processor -- start 2023-12-08T08:38:42.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.858+0000 7f022fd89640 1 -- start start 2023-12-08T08:38:42.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.859+0000 7f022fd89640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0228158ee0 0x7f0228070e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.859+0000 7f022fd89640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02280cbd10 0x7f022806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.859+0000 7f022fd89640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02280720d0 con 0x7f022815b750 2023-12-08T08:38:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.859+0000 7f022fd89640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02280cc890 con 0x7f0228158ee0 2023-12-08T08:38:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.859+0000 7f022fd89640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02280714c0 con 0x7f02280cbd10 2023-12-08T08:38:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.859+0000 7f022e326640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02280cbd10 0x7f022806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.859+0000 7f022db25640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f0228158ee0 0x7f0228070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.859+0000 7f022d324640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f022815b750 0x7f02280cb600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:33928/0 (socket says 172.21.15.16:33928) 2023-12-08T08:38:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.859+0000 7f022d324640 1 -- 172.21.15.16:0/4127238251 learned_addr learned my addr 172.21.15.16:0/4127238251 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.860+0000 7f022cb23640 1 -- 172.21.15.16:0/4127238251 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030762917 0 0) 0x7f02280720d0 con 0x7f022815b750 2023-12-08T08:38:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.860+0000 7f022cb23640 1 -- 172.21.15.16:0/4127238251 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0214003520 con 0x7f022815b750 2023-12-08T08:38:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.860+0000 7f022db25640 1 -- 172.21.15.16:0/4127238251 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02280cbd10 msgr2=0x7f022806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.860+0000 7f022db25640 1 --2- 172.21.15.16:0/4127238251 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02280cbd10 0x7f022806c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.860+0000 7f022db25640 1 -- 172.21.15.16:0/4127238251 >> v1:172.21.15.112:6789/0 conn(0x7f022815b750 legacy=0x7f02280cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.860+0000 7f022db25640 1 -- 172.21.15.16:0/4127238251 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02280717a0 con 0x7f0228158ee0 2023-12-08T08:38:42.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.860+0000 7f022e326640 1 --2- 172.21.15.16:0/4127238251 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02280cbd10 0x7f022806c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:42.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.861+0000 7f022db25640 1 --2- 172.21.15.16:0/4127238251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0228158ee0 0x7f0228070e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f02200097a0 tx=0x7f022002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d38db15328b6247b server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:42.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.861+0000 7f022cb23640 1 -- 172.21.15.16:0/4127238251 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022002e070 con 0x7f0228158ee0 2023-12-08T08:38:42.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.861+0000 7f022cb23640 1 -- 172.21.15.16:0/4127238251 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02200315a0 con 0x7f0228158ee0 2023-12-08T08:38:42.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.861+0000 7f022cb23640 1 -- 172.21.15.16:0/4127238251 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0220031930 con 0x7f0228158ee0 2023-12-08T08:38:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.861+0000 7f022fd89640 1 -- 172.21.15.16:0/4127238251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0228158ee0 msgr2=0x7f0228070e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.861+0000 7f022fd89640 1 --2- 172.21.15.16:0/4127238251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0228158ee0 0x7f0228070e30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f02200097a0 tx=0x7f022002b840 comp rx=0 tx=0).stop 2023-12-08T08:38:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.861+0000 7f022fd89640 1 -- 172.21.15.16:0/4127238251 shutdown_connections 2023-12-08T08:38:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.861+0000 7f022fd89640 1 --2- 172.21.15.16:0/4127238251 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02280cbd10 0x7f022806c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.861+0000 7f022fd89640 1 --2- 172.21.15.16:0/4127238251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0228158ee0 0x7f0228070e30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.861+0000 7f022fd89640 1 -- 172.21.15.16:0/4127238251 >> 172.21.15.16:0/4127238251 conn(0x7f02280b09f0 msgr2=0x7f02280b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.861+0000 7f022fd89640 1 -- 172.21.15.16:0/4127238251 shutdown_connections 2023-12-08T08:38:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.861+0000 7f022fd89640 1 -- 172.21.15.16:0/4127238251 wait complete. 2023-12-08T08:38:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.863+0000 7f022fd89640 1 Processor -- start 2023-12-08T08:38:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.863+0000 7f022fd89640 1 -- start start 2023-12-08T08:38:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.863+0000 7f022fd89640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02280cbd10 0x7f02281429a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.863+0000 7f022fd89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0228158ee0 0x7f0228159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.863+0000 7f022fd89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022815b750 0x7f022815a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.863+0000 7f022fd89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0228142ee0 con 0x7f022815b750 2023-12-08T08:38:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.863+0000 7f022fd89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02280cdaf0 con 0x7f0228158ee0 2023-12-08T08:38:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.863+0000 7f022fd89640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02280cddd0 con 0x7f02280cbd10 2023-12-08T08:38:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f022e326640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022815b750 0x7f022815a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f022e326640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022815b750 0x7f022815a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52342/0 (socket says 172.21.15.16:52342) 2023-12-08T08:38:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f022e326640 1 -- 172.21.15.16:0/2958540643 learned_addr learned my addr 172.21.15.16:0/2958540643 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f022d324640 1 --2- 172.21.15.16:0/2958540643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0228158ee0 0x7f0228159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f022db25640 1 --2- 172.21.15.16:0/2958540643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02280cbd10 0x7f02281429a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f022e326640 1 -- 172.21.15.16:0/2958540643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02280cbd10 msgr2=0x7f02281429a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f022e326640 1 --2- 172.21.15.16:0/2958540643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02280cbd10 0x7f02281429a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f022e326640 1 -- 172.21.15.16:0/2958540643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0228158ee0 msgr2=0x7f0228159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f022e326640 1 --2- 172.21.15.16:0/2958540643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0228158ee0 0x7f0228159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f022e326640 1 -- 172.21.15.16:0/2958540643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0224009a70 con 0x7f022815b750 2023-12-08T08:38:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f022e326640 1 --2- 172.21.15.16:0/2958540643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022815b750 0x7f022815a050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f022400efc0 tx=0x7f0224009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f021e7fc640 1 -- 172.21.15.16:0/2958540643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224009d70 con 0x7f022815b750 2023-12-08T08:38:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f021e7fc640 1 -- 172.21.15.16:0/2958540643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022400f040 con 0x7f022815b750 2023-12-08T08:38:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f021e7fc640 1 -- 172.21.15.16:0/2958540643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224003990 con 0x7f022815b750 2023-12-08T08:38:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f022fd89640 1 -- 172.21.15.16:0/2958540643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0220009400 con 0x7f022815b750 2023-12-08T08:38:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.864+0000 7f022fd89640 1 -- 172.21.15.16:0/2958540643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02280ce390 con 0x7f022815b750 2023-12-08T08:38:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.865+0000 7f022fd89640 1 -- 172.21.15.16:0/2958540643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0228067430 con 0x7f022815b750 2023-12-08T08:38:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.866+0000 7f021e7fc640 1 -- 172.21.15.16:0/2958540643 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f02240144a0 con 0x7f022815b750 2023-12-08T08:38:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.866+0000 7f021e7fc640 1 --2- 172.21.15.16:0/2958540643 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f020c06ae40 0x7f020c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.866+0000 7f021e7fc640 1 -- 172.21.15.16:0/2958540643 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f022408a4e0 con 0x7f022815b750 2023-12-08T08:38:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.866+0000 7f022db25640 1 --2- 172.21.15.16:0/2958540643 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f020c06ae40 0x7f020c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.867+0000 7f022db25640 1 --2- 172.21.15.16:0/2958540643 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f020c06ae40 0x7f020c06d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0220036000 tx=0x7f022003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:42.870+0000 7f021e7fc640 1 -- 172.21.15.16:0/2958540643 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0224055330 con 0x7f022815b750 2023-12-08T08:38:43.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.042+0000 7f022fd89640 1 -- 172.21.15.16:0/2958540643 --> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0228071d60 con 0x7f020c06ae40 2023-12-08T08:38:43.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.051+0000 7f021e7fc640 1 -- 172.21.15.16:0/2958540643 <== mgr.4111 v2:172.21.15.112:6832/804067392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345283 (secure 0 0 0) 0x7f02200319d0 con 0x7f020c06ae40 2023-12-08T08:38:43.058 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:43.060 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-12-08T08:38:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.055+0000 7f022fd89640 1 -- 172.21.15.16:0/2958540643 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f020c06ae40 msgr2=0x7f020c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.055+0000 7f022fd89640 1 --2- 172.21.15.16:0/2958540643 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f020c06ae40 0x7f020c06d300 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0220036000 tx=0x7f022003d040 comp rx=0 tx=0).stop 2023-12-08T08:38:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.055+0000 7f022fd89640 1 -- 172.21.15.16:0/2958540643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022815b750 msgr2=0x7f022815a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.055+0000 7f022fd89640 1 --2- 172.21.15.16:0/2958540643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022815b750 0x7f022815a050 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f022400efc0 tx=0x7f0224009f90 comp rx=0 tx=0).stop 2023-12-08T08:38:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.055+0000 7f022fd89640 1 -- 172.21.15.16:0/2958540643 shutdown_connections 2023-12-08T08:38:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.055+0000 7f022fd89640 1 --2- 172.21.15.16:0/2958540643 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f020c06ae40 0x7f020c06d300 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.055+0000 7f022fd89640 1 --2- 172.21.15.16:0/2958540643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f022815b750 0x7f022815a050 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.055+0000 7f022fd89640 1 --2- 172.21.15.16:0/2958540643 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0228158ee0 0x7f0228159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.055+0000 7f022fd89640 1 --2- 172.21.15.16:0/2958540643 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02280cbd10 0x7f02281429a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.055+0000 7f022fd89640 1 -- 172.21.15.16:0/2958540643 >> 172.21.15.16:0/2958540643 conn(0x7f02280b09f0 msgr2=0x7f022815aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:43.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.055+0000 7f022fd89640 1 -- 172.21.15.16:0/2958540643 shutdown_connections 2023-12-08T08:38:43.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.055+0000 7f022fd89640 1 -- 172.21.15.16:0/2958540643 wait complete. 2023-12-08T08:38:43.068 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2023-12-08T08:38:42.979864+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231864,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742856,"statfs":{"total":773094113280,"available":772856684544,"internally_reserved":0,"allocated":3284992,"data_stored":1988346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003364"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659961+0000","last_change":"2023-12-08T08:38:18.634619+0000","last_active":"2023-12-08T08:38:23.659961+0000","last_peered":"2023-12-08T08:38:23.659961+0000","last_clean":"2023-12-08T08:38:23.659961+0000","last_became_active":"2023-12-08T08:38:18.634328+0000","last_became_peered":"2023-12-08T08:38:18.634328+0000","last_unstale":"2023-12-08T08:38:23.659961+0000","last_undegraded":"2023-12-08T08:38:23.659961+0000","last_fullsized":"2023-12-08T08:38:23.659961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:11.067589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656122+0000","last_change":"2023-12-08T08:38:19.627512+0000","last_active":"2023-12-08T08:38:23.656122+0000","last_peered":"2023-12-08T08:38:23.656122+0000","last_clean":"2023-12-08T08:38:23.656122+0000","last_became_active":"2023-12-08T08:38:19.625456+0000","last_became_peered":"2023-12-08T08:38:19.625456+0000","last_unstale":"2023-12-08T08:38:23.656122+0000","last_undegraded":"2023-12-08T08:38:23.656122+0000","last_fullsized":"2023-12-08T08:38:23.656122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576137+0000","last_change":"2023-12-08T08:38:23.856216+0000","last_active":"2023-12-08T08:38:25.576137+0000","last_peered":"2023-12-08T08:38:25.576137+0000","last_clean":"2023-12-08T08:38:25.576137+0000","last_became_active":"2023-12-08T08:38:23.855623+0000","last_became_peered":"2023-12-08T08:38:23.855623+0000","last_unstale":"2023-12-08T08:38:25.576137+0000","last_undegraded":"2023-12-08T08:38:25.576137+0000","last_fullsized":"2023-12-08T08:38:25.576137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:26:16.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656314+0000","last_change":"2023-12-08T08:38:18.631598+0000","last_active":"2023-12-08T08:38:23.656314+0000","last_peered":"2023-12-08T08:38:23.656314+0000","last_clean":"2023-12-08T08:38:23.656314+0000","last_became_active":"2023-12-08T08:38:18.631295+0000","last_became_peered":"2023-12-08T08:38:18.631295+0000","last_unstale":"2023-12-08T08:38:23.656314+0000","last_undegraded":"2023-12-08T08:38:23.656314+0000","last_fullsized":"2023-12-08T08:38:23.656314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659744+0000","last_change":"2023-12-08T08:38:19.633463+0000","last_active":"2023-12-08T08:38:23.659744+0000","last_peered":"2023-12-08T08:38:23.659744+0000","last_clean":"2023-12-08T08:38:23.659744+0000","last_became_active":"2023-12-08T08:38:19.633094+0000","last_became_peered":"2023-12-08T08:38:19.633094+0000","last_unstale":"2023-12-08T08:38:23.659744+0000","last_undegraded":"2023-12-08T08:38:23.659744+0000","last_fullsized":"2023-12-08T08:38:23.659744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608661+0000","last_change":"2023-12-08T08:38:23.674366+0000","last_active":"2023-12-08T08:38:24.608661+0000","last_peered":"2023-12-08T08:38:24.608661+0000","last_clean":"2023-12-08T08:38:24.608661+0000","last_became_active":"2023-12-08T08:38:23.673810+0000","last_became_peered":"2023-12-08T08:38:23.673810+0000","last_unstale":"2023-12-08T08:38:24.608661+0000","last_undegraded":"2023-12-08T08:38:24.608661+0000","last_fullsized":"2023-12-08T08:38:24.608661+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:47.374495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852405+0000","last_change":"2023-12-08T08:38:18.633940+0000","last_active":"2023-12-08T08:38:23.852405+0000","last_peered":"2023-12-08T08:38:23.852405+0000","last_clean":"2023-12-08T08:38:23.852405+0000","last_became_active":"2023-12-08T08:38:18.631964+0000","last_became_peered":"2023-12-08T08:38:18.631964+0000","last_unstale":"2023-12-08T08:38:23.852405+0000","last_undegraded":"2023-12-08T08:38:23.852405+0000","last_fullsized":"2023-12-08T08:38:23.852405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:37.317712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661342+0000","last_change":"2023-12-08T08:38:19.627445+0000","last_active":"2023-12-08T08:38:23.661342+0000","last_peered":"2023-12-08T08:38:23.661342+0000","last_clean":"2023-12-08T08:38:23.661342+0000","last_became_active":"2023-12-08T08:38:19.627174+0000","last_became_peered":"2023-12-08T08:38:19.627174+0000","last_unstale":"2023-12-08T08:38:23.661342+0000","last_undegraded":"2023-12-08T08:38:23.661342+0000","last_fullsized":"2023-12-08T08:38:23.661342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:32.116348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609171+0000","last_change":"2023-12-08T08:38:23.674338+0000","last_active":"2023-12-08T08:38:24.609171+0000","last_peered":"2023-12-08T08:38:24.609171+0000","last_clean":"2023-12-08T08:38:24.609171+0000","last_became_active":"2023-12-08T08:38:23.673614+0000","last_became_peered":"2023-12-08T08:38:23.673614+0000","last_unstale":"2023-12-08T08:38:24.609171+0000","last_undegraded":"2023-12-08T08:38:24.609171+0000","last_fullsized":"2023-12-08T08:38:24.609171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:59.405196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852122+0000","last_change":"2023-12-08T08:38:18.633561+0000","last_active":"2023-12-08T08:38:23.852122+0000","last_peered":"2023-12-08T08:38:23.852122+0000","last_clean":"2023-12-08T08:38:23.852122+0000","last_became_active":"2023-12-08T08:38:18.632646+0000","last_became_peered":"2023-12-08T08:38:18.632646+0000","last_unstale":"2023-12-08T08:38:23.852122+0000","last_undegraded":"2023-12-08T08:38:23.852122+0000","last_fullsized":"2023-12-08T08:38:23.852122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:36:34.339195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662141+0000","last_change":"2023-12-08T08:38:19.632216+0000","last_active":"2023-12-08T08:38:23.662141+0000","last_peered":"2023-12-08T08:38:23.662141+0000","last_clean":"2023-12-08T08:38:23.662141+0000","last_became_active":"2023-12-08T08:38:19.631599+0000","last_became_peered":"2023-12-08T08:38:19.631599+0000","last_unstale":"2023-12-08T08:38:23.662141+0000","last_undegraded":"2023-12-08T08:38:23.662141+0000","last_fullsized":"2023-12-08T08:38:23.662141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:11.797914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608922+0000","last_change":"2023-12-08T08:38:23.669631+0000","last_active":"2023-12-08T08:38:24.608922+0000","last_peered":"2023-12-08T08:38:24.608922+0000","last_clean":"2023-12-08T08:38:24.608922+0000","last_became_active":"2023-12-08T08:38:23.669222+0000","last_became_peered":"2023-12-08T08:38:23.669222+0000","last_unstale":"2023-12-08T08:38:24.608922+0000","last_undegraded":"2023-12-08T08:38:24.608922+0000","last_fullsized":"2023-12-08T08:38:24.608922+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:49:33.936211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664041+0000","last_change":"2023-12-08T08:38:19.620724+0000","last_active":"2023-12-08T08:38:23.664041+0000","last_peered":"2023-12-08T08:38:23.664041+0000","last_clean":"2023-12-08T08:38:23.664041+0000","last_became_active":"2023-12-08T08:38:19.620366+0000","last_became_peered":"2023-12-08T08:38:19.620366+0000","last_unstale":"2023-12-08T08:38:23.664041+0000","last_undegraded":"2023-12-08T08:38:23.664041+0000","last_fullsized":"2023-12-08T08:38:23.664041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:00:01.146409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852136+0000","last_change":"2023-12-08T08:38:18.632086+0000","last_active":"2023-12-08T08:38:23.852136+0000","last_peered":"2023-12-08T08:38:23.852136+0000","last_clean":"2023-12-08T08:38:23.852136+0000","last_became_active":"2023-12-08T08:38:18.631756+0000","last_became_peered":"2023-12-08T08:38:18.631756+0000","last_unstale":"2023-12-08T08:38:23.852136+0000","last_undegraded":"2023-12-08T08:38:23.852136+0000","last_fullsized":"2023-12-08T08:38:23.852136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:14.342540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609352+0000","last_change":"2023-12-08T08:38:23.673898+0000","last_active":"2023-12-08T08:38:24.609352+0000","last_peered":"2023-12-08T08:38:24.609352+0000","last_clean":"2023-12-08T08:38:24.609352+0000","last_became_active":"2023-12-08T08:38:23.673389+0000","last_became_peered":"2023-12-08T08:38:23.673389+0000","last_unstale":"2023-12-08T08:38:24.609352+0000","last_undegraded":"2023-12-08T08:38:24.609352+0000","last_fullsized":"2023-12-08T08:38:24.609352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:15.835775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655652+0000","last_change":"2023-12-08T08:38:19.627148+0000","last_active":"2023-12-08T08:38:23.655652+0000","last_peered":"2023-12-08T08:38:23.655652+0000","last_clean":"2023-12-08T08:38:23.655652+0000","last_became_active":"2023-12-08T08:38:19.626846+0000","last_became_peered":"2023-12-08T08:38:19.626846+0000","last_unstale":"2023-12-08T08:38:23.655652+0000","last_undegraded":"2023-12-08T08:38:23.655652+0000","last_fullsized":"2023-12-08T08:38:23.655652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.666654+0000","last_change":"2023-12-08T08:38:18.632191+0000","last_active":"2023-12-08T08:38:23.666654+0000","last_peered":"2023-12-08T08:38:23.666654+0000","last_clean":"2023-12-08T08:38:23.666654+0000","last_became_active":"2023-12-08T08:38:18.631927+0000","last_became_peered":"2023-12-08T08:38:18.631927+0000","last_unstale":"2023-12-08T08:38:23.666654+0000","last_undegraded":"2023-12-08T08:38:23.666654+0000","last_fullsized":"2023-12-08T08:38:23.666654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:10.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608968+0000","last_change":"2023-12-08T08:38:23.678875+0000","last_active":"2023-12-08T08:38:24.608968+0000","last_peered":"2023-12-08T08:38:24.608968+0000","last_clean":"2023-12-08T08:38:24.608968+0000","last_became_active":"2023-12-08T08:38:23.678332+0000","last_became_peered":"2023-12-08T08:38:23.678332+0000","last_unstale":"2023-12-08T08:38:24.608968+0000","last_undegraded":"2023-12-08T08:38:24.608968+0000","last_fullsized":"2023-12-08T08:38:24.608968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:36.154208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656075+0000","last_change":"2023-12-08T08:38:19.625657+0000","last_active":"2023-12-08T08:38:23.656075+0000","last_peered":"2023-12-08T08:38:23.656075+0000","last_clean":"2023-12-08T08:38:23.656075+0000","last_became_active":"2023-12-08T08:38:19.624790+0000","last_became_peered":"2023-12-08T08:38:19.624790+0000","last_unstale":"2023-12-08T08:38:23.656075+0000","last_undegraded":"2023-12-08T08:38:23.656075+0000","last_fullsized":"2023-12-08T08:38:23.656075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:32.116348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664683+0000","last_change":"2023-12-08T08:38:18.637627+0000","last_active":"2023-12-08T08:38:23.664683+0000","last_peered":"2023-12-08T08:38:23.664683+0000","last_clean":"2023-12-08T08:38:23.664683+0000","last_became_active":"2023-12-08T08:38:18.637335+0000","last_became_peered":"2023-12-08T08:38:18.637335+0000","last_unstale":"2023-12-08T08:38:23.664683+0000","last_undegraded":"2023-12-08T08:38:23.664683+0000","last_fullsized":"2023-12-08T08:38:23.664683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.828080+0000","last_change":"2023-12-08T08:38:23.852913+0000","last_active":"2023-12-08T08:38:25.828080+0000","last_peered":"2023-12-08T08:38:25.828080+0000","last_clean":"2023-12-08T08:38:25.828080+0000","last_became_active":"2023-12-08T08:38:23.852508+0000","last_became_peered":"2023-12-08T08:38:23.852508+0000","last_unstale":"2023-12-08T08:38:25.828080+0000","last_undegraded":"2023-12-08T08:38:25.828080+0000","last_fullsized":"2023-12-08T08:38:25.828080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:48.357070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658598+0000","last_change":"2023-12-08T08:38:19.623518+0000","last_active":"2023-12-08T08:38:23.658598+0000","last_peered":"2023-12-08T08:38:23.658598+0000","last_clean":"2023-12-08T08:38:23.658598+0000","last_became_active":"2023-12-08T08:38:19.623240+0000","last_became_peered":"2023-12-08T08:38:19.623240+0000","last_unstale":"2023-12-08T08:38:23.658598+0000","last_undegraded":"2023-12-08T08:38:23.658598+0000","last_fullsized":"2023-12-08T08:38:23.658598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:43.773215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851377+0000","last_change":"2023-12-08T08:38:18.633585+0000","last_active":"2023-12-08T08:38:23.851377+0000","last_peered":"2023-12-08T08:38:23.851377+0000","last_clean":"2023-12-08T08:38:23.851377+0000","last_became_active":"2023-12-08T08:38:18.633112+0000","last_became_peered":"2023-12-08T08:38:18.633112+0000","last_unstale":"2023-12-08T08:38:23.851377+0000","last_undegraded":"2023-12-08T08:38:23.851377+0000","last_fullsized":"2023-12-08T08:38:23.851377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:42.768579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609486+0000","last_change":"2023-12-08T08:38:23.678810+0000","last_active":"2023-12-08T08:38:24.609486+0000","last_peered":"2023-12-08T08:38:24.609486+0000","last_clean":"2023-12-08T08:38:24.609486+0000","last_became_active":"2023-12-08T08:38:23.678331+0000","last_became_peered":"2023-12-08T08:38:23.678331+0000","last_unstale":"2023-12-08T08:38:24.609486+0000","last_undegraded":"2023-12-08T08:38:24.609486+0000","last_fullsized":"2023-12-08T08:38:24.609486+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:17.394710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.666222+0000","last_change":"2023-12-08T08:38:19.619485+0000","last_active":"2023-12-08T08:38:23.666222+0000","last_peered":"2023-12-08T08:38:23.666222+0000","last_clean":"2023-12-08T08:38:23.666222+0000","last_became_active":"2023-12-08T08:38:19.618880+0000","last_became_peered":"2023-12-08T08:38:19.618880+0000","last_unstale":"2023-12-08T08:38:23.666222+0000","last_undegraded":"2023-12-08T08:38:23.666222+0000","last_fullsized":"2023-12-08T08:38:23.666222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:43.336635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665379+0000","last_change":"2023-12-08T08:38:18.633093+0000","last_active":"2023-12-08T08:38:23.665379+0000","last_peered":"2023-12-08T08:38:23.665379+0000","last_clean":"2023-12-08T08:38:23.665379+0000","last_became_active":"2023-12-08T08:38:18.632832+0000","last_became_peered":"2023-12-08T08:38:18.632832+0000","last_unstale":"2023-12-08T08:38:23.665379+0000","last_undegraded":"2023-12-08T08:38:23.665379+0000","last_fullsized":"2023-12-08T08:38:23.665379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:37.317712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608658+0000","last_change":"2023-12-08T08:38:23.853769+0000","last_active":"2023-12-08T08:38:24.608658+0000","last_peered":"2023-12-08T08:38:24.608658+0000","last_clean":"2023-12-08T08:38:24.608658+0000","last_became_active":"2023-12-08T08:38:23.853211+0000","last_became_peered":"2023-12-08T08:38:23.853211+0000","last_unstale":"2023-12-08T08:38:24.608658+0000","last_undegraded":"2023-12-08T08:38:24.608658+0000","last_fullsized":"2023-12-08T08:38:24.608658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:59:15.111403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656171+0000","last_change":"2023-12-08T08:38:19.626321+0000","last_active":"2023-12-08T08:38:23.656171+0000","last_peered":"2023-12-08T08:38:23.656171+0000","last_clean":"2023-12-08T08:38:23.656171+0000","last_became_active":"2023-12-08T08:38:19.624741+0000","last_became_peered":"2023-12-08T08:38:19.624741+0000","last_unstale":"2023-12-08T08:38:23.656171+0000","last_undegraded":"2023-12-08T08:38:23.656171+0000","last_fullsized":"2023-12-08T08:38:23.656171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:49:29.898351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664728+0000","last_change":"2023-12-08T08:38:18.636600+0000","last_active":"2023-12-08T08:38:23.664728+0000","last_peered":"2023-12-08T08:38:23.664728+0000","last_clean":"2023-12-08T08:38:23.664728+0000","last_became_active":"2023-12-08T08:38:18.633400+0000","last_became_peered":"2023-12-08T08:38:18.633400+0000","last_unstale":"2023-12-08T08:38:23.664728+0000","last_undegraded":"2023-12-08T08:38:23.664728+0000","last_fullsized":"2023-12-08T08:38:23.664728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:37.317712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576086+0000","last_change":"2023-12-08T08:38:23.856023+0000","last_active":"2023-12-08T08:38:25.576086+0000","last_peered":"2023-12-08T08:38:25.576086+0000","last_clean":"2023-12-08T08:38:25.576086+0000","last_became_active":"2023-12-08T08:38:23.855599+0000","last_became_peered":"2023-12-08T08:38:23.855599+0000","last_unstale":"2023-12-08T08:38:25.576086+0000","last_undegraded":"2023-12-08T08:38:25.576086+0000","last_fullsized":"2023-12-08T08:38:25.576086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T20:21:44.003574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662450+0000","last_change":"2023-12-08T08:38:19.622140+0000","last_active":"2023-12-08T08:38:23.662450+0000","last_peered":"2023-12-08T08:38:23.662450+0000","last_clean":"2023-12-08T08:38:23.662450+0000","last_became_active":"2023-12-08T08:38:19.621784+0000","last_became_peered":"2023-12-08T08:38:19.621784+0000","last_unstale":"2023-12-08T08:38:23.662450+0000","last_undegraded":"2023-12-08T08:38:23.662450+0000","last_fullsized":"2023-12-08T08:38:23.662450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852845+0000","last_change":"2023-12-08T08:38:18.634123+0000","last_active":"2023-12-08T08:38:23.852845+0000","last_peered":"2023-12-08T08:38:23.852845+0000","last_clean":"2023-12-08T08:38:23.852845+0000","last_became_active":"2023-12-08T08:38:18.632874+0000","last_became_peered":"2023-12-08T08:38:18.632874+0000","last_unstale":"2023-12-08T08:38:23.852845+0000","last_undegraded":"2023-12-08T08:38:23.852845+0000","last_fullsized":"2023-12-08T08:38:23.852845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:07.497079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.326786+0000","last_change":"2023-12-08T08:38:23.852386+0000","last_active":"2023-12-08T08:38:26.326786+0000","last_peered":"2023-12-08T08:38:26.326786+0000","last_clean":"2023-12-08T08:38:26.326786+0000","last_became_active":"2023-12-08T08:38:23.851973+0000","last_became_peered":"2023-12-08T08:38:23.851973+0000","last_unstale":"2023-12-08T08:38:26.326786+0000","last_undegraded":"2023-12-08T08:38:26.326786+0000","last_fullsized":"2023-12-08T08:38:26.326786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:22.372635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662330+0000","last_change":"2023-12-08T08:38:19.631928+0000","last_active":"2023-12-08T08:38:23.662330+0000","last_peered":"2023-12-08T08:38:23.662330+0000","last_clean":"2023-12-08T08:38:23.662330+0000","last_became_active":"2023-12-08T08:38:19.631447+0000","last_became_peered":"2023-12-08T08:38:19.631447+0000","last_unstale":"2023-12-08T08:38:23.662330+0000","last_undegraded":"2023-12-08T08:38:23.662330+0000","last_fullsized":"2023-12-08T08:38:23.662330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:13.356849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662548+0000","last_change":"2023-12-08T08:38:18.634530+0000","last_active":"2023-12-08T08:38:23.662548+0000","last_peered":"2023-12-08T08:38:23.662548+0000","last_clean":"2023-12-08T08:38:23.662548+0000","last_became_active":"2023-12-08T08:38:18.634239+0000","last_became_peered":"2023-12-08T08:38:18.634239+0000","last_unstale":"2023-12-08T08:38:23.662548+0000","last_undegraded":"2023-12-08T08:38:23.662548+0000","last_fullsized":"2023-12-08T08:38:23.662548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:42.768579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.827628+0000","last_change":"2023-12-08T08:38:23.676640+0000","last_active":"2023-12-08T08:38:25.827628+0000","last_peered":"2023-12-08T08:38:25.827628+0000","last_clean":"2023-12-08T08:38:25.827628+0000","last_became_active":"2023-12-08T08:38:23.675999+0000","last_became_peered":"2023-12-08T08:38:23.675999+0000","last_unstale":"2023-12-08T08:38:25.827628+0000","last_undegraded":"2023-12-08T08:38:25.827628+0000","last_fullsized":"2023-12-08T08:38:25.827628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:15.835775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658939+0000","last_change":"2023-12-08T08:38:19.615261+0000","last_active":"2023-12-08T08:38:23.658939+0000","last_peered":"2023-12-08T08:38:23.658939+0000","last_clean":"2023-12-08T08:38:23.658939+0000","last_became_active":"2023-12-08T08:38:19.614990+0000","last_became_peered":"2023-12-08T08:38:19.614990+0000","last_unstale":"2023-12-08T08:38:23.658939+0000","last_undegraded":"2023-12-08T08:38:23.658939+0000","last_fullsized":"2023-12-08T08:38:23.658939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:05:08.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662040+0000","last_change":"2023-12-08T08:38:18.630062+0000","last_active":"2023-12-08T08:38:23.662040+0000","last_peered":"2023-12-08T08:38:23.662040+0000","last_clean":"2023-12-08T08:38:23.662040+0000","last_became_active":"2023-12-08T08:38:18.629798+0000","last_became_peered":"2023-12-08T08:38:18.629798+0000","last_unstale":"2023-12-08T08:38:23.662040+0000","last_undegraded":"2023-12-08T08:38:23.662040+0000","last_fullsized":"2023-12-08T08:38:23.662040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:13.807850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.827791+0000","last_change":"2023-12-08T08:38:23.674624+0000","last_active":"2023-12-08T08:38:25.827791+0000","last_peered":"2023-12-08T08:38:25.827791+0000","last_clean":"2023-12-08T08:38:25.827791+0000","last_became_active":"2023-12-08T08:38:23.674121+0000","last_became_peered":"2023-12-08T08:38:23.674121+0000","last_unstale":"2023-12-08T08:38:25.827791+0000","last_undegraded":"2023-12-08T08:38:25.827791+0000","last_fullsized":"2023-12-08T08:38:25.827791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:17.394710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665266+0000","last_change":"2023-12-08T08:38:19.621679+0000","last_active":"2023-12-08T08:38:23.665266+0000","last_peered":"2023-12-08T08:38:23.665266+0000","last_clean":"2023-12-08T08:38:23.665266+0000","last_became_active":"2023-12-08T08:38:19.621388+0000","last_became_peered":"2023-12-08T08:38:19.621388+0000","last_unstale":"2023-12-08T08:38:23.665266+0000","last_undegraded":"2023-12-08T08:38:23.665266+0000","last_fullsized":"2023-12-08T08:38:23.665266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662810+0000","last_change":"2023-12-08T08:38:18.632407+0000","last_active":"2023-12-08T08:38:23.662810+0000","last_peered":"2023-12-08T08:38:23.662810+0000","last_clean":"2023-12-08T08:38:23.662810+0000","last_became_active":"2023-12-08T08:38:18.632115+0000","last_became_peered":"2023-12-08T08:38:18.632115+0000","last_unstale":"2023-12-08T08:38:23.662810+0000","last_undegraded":"2023-12-08T08:38:23.662810+0000","last_fullsized":"2023-12-08T08:38:23.662810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:05:14.591351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.828126+0000","last_change":"2023-12-08T08:38:23.675339+0000","last_active":"2023-12-08T08:38:25.828126+0000","last_peered":"2023-12-08T08:38:25.828126+0000","last_clean":"2023-12-08T08:38:25.828126+0000","last_became_active":"2023-12-08T08:38:23.674249+0000","last_became_peered":"2023-12-08T08:38:23.674249+0000","last_unstale":"2023-12-08T08:38:25.828126+0000","last_undegraded":"2023-12-08T08:38:25.828126+0000","last_fullsized":"2023-12-08T08:38:25.828126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:34:29.925547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.660253+0000","last_change":"2023-12-08T08:38:19.611646+0000","last_active":"2023-12-08T08:38:23.660253+0000","last_peered":"2023-12-08T08:38:23.660253+0000","last_clean":"2023-12-08T08:38:23.660253+0000","last_became_active":"2023-12-08T08:38:19.611315+0000","last_became_peered":"2023-12-08T08:38:19.611315+0000","last_unstale":"2023-12-08T08:38:23.660253+0000","last_undegraded":"2023-12-08T08:38:23.660253+0000","last_fullsized":"2023-12-08T08:38:23.660253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:11.612354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851317+0000","last_change":"2023-12-08T08:38:18.634473+0000","last_active":"2023-12-08T08:38:23.851317+0000","last_peered":"2023-12-08T08:38:23.851317+0000","last_clean":"2023-12-08T08:38:23.851317+0000","last_became_active":"2023-12-08T08:38:18.633755+0000","last_became_peered":"2023-12-08T08:38:18.633755+0000","last_unstale":"2023-12-08T08:38:23.851317+0000","last_undegraded":"2023-12-08T08:38:23.851317+0000","last_fullsized":"2023-12-08T08:38:23.851317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:11.067589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.327085+0000","last_change":"2023-12-08T08:38:23.852846+0000","last_active":"2023-12-08T08:38:26.327085+0000","last_peered":"2023-12-08T08:38:26.327085+0000","last_clean":"2023-12-08T08:38:26.327085+0000","last_became_active":"2023-12-08T08:38:23.852393+0000","last_became_peered":"2023-12-08T08:38:23.852393+0000","last_unstale":"2023-12-08T08:38:26.327085+0000","last_undegraded":"2023-12-08T08:38:26.327085+0000","last_fullsized":"2023-12-08T08:38:26.327085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:32:58.184058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655788+0000","last_change":"2023-12-08T08:38:19.634986+0000","last_active":"2023-12-08T08:38:23.655788+0000","last_peered":"2023-12-08T08:38:23.655788+0000","last_clean":"2023-12-08T08:38:23.655788+0000","last_became_active":"2023-12-08T08:38:19.634682+0000","last_became_peered":"2023-12-08T08:38:19.634682+0000","last_unstale":"2023-12-08T08:38:23.655788+0000","last_undegraded":"2023-12-08T08:38:23.655788+0000","last_fullsized":"2023-12-08T08:38:23.655788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661184+0000","last_change":"2023-12-08T08:38:18.631253+0000","last_active":"2023-12-08T08:38:23.661184+0000","last_peered":"2023-12-08T08:38:23.661184+0000","last_clean":"2023-12-08T08:38:23.661184+0000","last_became_active":"2023-12-08T08:38:18.630979+0000","last_became_peered":"2023-12-08T08:38:18.630979+0000","last_unstale":"2023-12-08T08:38:23.661184+0000","last_undegraded":"2023-12-08T08:38:23.661184+0000","last_fullsized":"2023-12-08T08:38:23.661184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:00:00.141773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851733+0000","last_change":"2023-12-08T08:38:08.276815+0000","last_active":"2023-12-08T08:38:23.851733+0000","last_peered":"2023-12-08T08:38:23.851733+0000","last_clean":"2023-12-08T08:38:23.851733+0000","last_became_active":"2023-12-08T08:38:08.275828+0000","last_became_peered":"2023-12-08T08:38:08.275828+0000","last_unstale":"2023-12-08T08:38:23.851733+0000","last_undegraded":"2023-12-08T08:38:23.851733+0000","last_fullsized":"2023-12-08T08:38:23.851733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_clean_scrub_stamp":"2023-12-08T08:38:07.104197+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:35:34.851149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.328098+0000","last_change":"2023-12-08T08:38:23.855031+0000","last_active":"2023-12-08T08:38:26.328098+0000","last_peered":"2023-12-08T08:38:26.328098+0000","last_clean":"2023-12-08T08:38:26.328098+0000","last_became_active":"2023-12-08T08:38:23.854583+0000","last_became_peered":"2023-12-08T08:38:23.854583+0000","last_unstale":"2023-12-08T08:38:26.328098+0000","last_undegraded":"2023-12-08T08:38:26.328098+0000","last_fullsized":"2023-12-08T08:38:26.328098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:01.683162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852252+0000","last_change":"2023-12-08T08:38:19.633600+0000","last_active":"2023-12-08T08:38:23.852252+0000","last_peered":"2023-12-08T08:38:23.852252+0000","last_clean":"2023-12-08T08:38:23.852252+0000","last_became_active":"2023-12-08T08:38:19.633358+0000","last_became_peered":"2023-12-08T08:38:19.633358+0000","last_unstale":"2023-12-08T08:38:23.852252+0000","last_undegraded":"2023-12-08T08:38:23.852252+0000","last_fullsized":"2023-12-08T08:38:23.852252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:13.356849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662546+0000","last_change":"2023-12-08T08:38:18.631378+0000","last_active":"2023-12-08T08:38:23.662546+0000","last_peered":"2023-12-08T08:38:23.662546+0000","last_clean":"2023-12-08T08:38:23.662546+0000","last_became_active":"2023-12-08T08:38:18.631072+0000","last_became_peered":"2023-12-08T08:38:18.631072+0000","last_unstale":"2023-12-08T08:38:23.662546+0000","last_undegraded":"2023-12-08T08:38:23.662546+0000","last_fullsized":"2023-12-08T08:38:23.662546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:27:03.426918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.327456+0000","last_change":"2023-12-08T08:38:23.853285+0000","last_active":"2023-12-08T08:38:26.327456+0000","last_peered":"2023-12-08T08:38:26.327456+0000","last_clean":"2023-12-08T08:38:26.327456+0000","last_became_active":"2023-12-08T08:38:23.852823+0000","last_became_peered":"2023-12-08T08:38:23.852823+0000","last_unstale":"2023-12-08T08:38:26.327456+0000","last_undegraded":"2023-12-08T08:38:26.327456+0000","last_fullsized":"2023-12-08T08:38:26.327456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:59.405196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661432+0000","last_change":"2023-12-08T08:38:19.632038+0000","last_active":"2023-12-08T08:38:23.661432+0000","last_peered":"2023-12-08T08:38:23.661432+0000","last_clean":"2023-12-08T08:38:23.661432+0000","last_became_active":"2023-12-08T08:38:19.630797+0000","last_became_peered":"2023-12-08T08:38:19.630797+0000","last_unstale":"2023-12-08T08:38:23.661432+0000","last_undegraded":"2023-12-08T08:38:23.661432+0000","last_fullsized":"2023-12-08T08:38:23.661432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:46:57.645302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662789+0000","last_change":"2023-12-08T08:38:18.633629+0000","last_active":"2023-12-08T08:38:23.662789+0000","last_peered":"2023-12-08T08:38:23.662789+0000","last_clean":"2023-12-08T08:38:23.662789+0000","last_became_active":"2023-12-08T08:38:18.633224+0000","last_became_peered":"2023-12-08T08:38:18.633224+0000","last_unstale":"2023-12-08T08:38:23.662789+0000","last_undegraded":"2023-12-08T08:38:23.662789+0000","last_fullsized":"2023-12-08T08:38:23.662789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:13:39.901023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.327158+0000","last_change":"2023-12-08T08:38:23.680301+0000","last_active":"2023-12-08T08:38:26.327158+0000","last_peered":"2023-12-08T08:38:26.327158+0000","last_clean":"2023-12-08T08:38:26.327158+0000","last_became_active":"2023-12-08T08:38:23.679652+0000","last_became_peered":"2023-12-08T08:38:23.679652+0000","last_unstale":"2023-12-08T08:38:26.327158+0000","last_undegraded":"2023-12-08T08:38:26.327158+0000","last_fullsized":"2023-12-08T08:38:26.327158+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:18:13.891999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.853813+0000","last_change":"2023-12-08T08:38:19.622603+0000","last_active":"2023-12-08T08:38:23.853813+0000","last_peered":"2023-12-08T08:38:23.853813+0000","last_clean":"2023-12-08T08:38:23.853813+0000","last_became_active":"2023-12-08T08:38:19.622325+0000","last_became_peered":"2023-12-08T08:38:19.622325+0000","last_unstale":"2023-12-08T08:38:23.853813+0000","last_undegraded":"2023-12-08T08:38:23.853813+0000","last_fullsized":"2023-12-08T08:38:23.853813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:01:22.871596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659628+0000","last_change":"2023-12-08T08:38:18.635791+0000","last_active":"2023-12-08T08:38:23.659628+0000","last_peered":"2023-12-08T08:38:23.659628+0000","last_clean":"2023-12-08T08:38:23.659628+0000","last_became_active":"2023-12-08T08:38:18.632773+0000","last_became_peered":"2023-12-08T08:38:18.632773+0000","last_unstale":"2023-12-08T08:38:23.659628+0000","last_undegraded":"2023-12-08T08:38:23.659628+0000","last_fullsized":"2023-12-08T08:38:23.659628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:46:19.770776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.626169+0000","last_change":"2023-12-08T08:38:23.853534+0000","last_active":"2023-12-08T08:38:24.626169+0000","last_peered":"2023-12-08T08:38:24.626169+0000","last_clean":"2023-12-08T08:38:24.626169+0000","last_became_active":"2023-12-08T08:38:23.852476+0000","last_became_peered":"2023-12-08T08:38:23.852476+0000","last_unstale":"2023-12-08T08:38:24.626169+0000","last_undegraded":"2023-12-08T08:38:24.626169+0000","last_fullsized":"2023-12-08T08:38:24.626169+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:01:26.909456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852501+0000","last_change":"2023-12-08T08:38:19.621763+0000","last_active":"2023-12-08T08:38:23.852501+0000","last_peered":"2023-12-08T08:38:23.852501+0000","last_clean":"2023-12-08T08:38:23.852501+0000","last_became_active":"2023-12-08T08:38:19.621504+0000","last_became_peered":"2023-12-08T08:38:19.621504+0000","last_unstale":"2023-12-08T08:38:23.852501+0000","last_undegraded":"2023-12-08T08:38:23.852501+0000","last_fullsized":"2023-12-08T08:38:23.852501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:34:25.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658538+0000","last_change":"2023-12-08T08:38:18.632647+0000","last_active":"2023-12-08T08:38:23.658538+0000","last_peered":"2023-12-08T08:38:23.658538+0000","last_clean":"2023-12-08T08:38:23.658538+0000","last_became_active":"2023-12-08T08:38:18.632398+0000","last_became_peered":"2023-12-08T08:38:18.632398+0000","last_unstale":"2023-12-08T08:38:23.658538+0000","last_undegraded":"2023-12-08T08:38:23.658538+0000","last_fullsized":"2023-12-08T08:38:23.658538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:42:30.138259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608618+0000","last_change":"2023-12-08T08:38:23.674111+0000","last_active":"2023-12-08T08:38:24.608618+0000","last_peered":"2023-12-08T08:38:24.608618+0000","last_clean":"2023-12-08T08:38:24.608618+0000","last_became_active":"2023-12-08T08:38:23.673287+0000","last_became_peered":"2023-12-08T08:38:23.673287+0000","last_unstale":"2023-12-08T08:38:24.608618+0000","last_undegraded":"2023-12-08T08:38:24.608618+0000","last_fullsized":"2023-12-08T08:38:24.608618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:22:01.560111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851975+0000","last_change":"2023-12-08T08:38:19.620974+0000","last_active":"2023-12-08T08:38:23.851975+0000","last_peered":"2023-12-08T08:38:23.851975+0000","last_clean":"2023-12-08T08:38:23.851975+0000","last_became_active":"2023-12-08T08:38:19.619478+0000","last_became_peered":"2023-12-08T08:38:19.619478+0000","last_unstale":"2023-12-08T08:38:23.851975+0000","last_undegraded":"2023-12-08T08:38:23.851975+0000","last_fullsized":"2023-12-08T08:38:23.851975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:55.367335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659793+0000","last_change":"2023-12-08T08:38:18.629485+0000","last_active":"2023-12-08T08:38:23.659793+0000","last_peered":"2023-12-08T08:38:23.659793+0000","last_clean":"2023-12-08T08:38:23.659793+0000","last_became_active":"2023-12-08T08:38:18.629182+0000","last_became_peered":"2023-12-08T08:38:18.629182+0000","last_unstale":"2023-12-08T08:38:23.659793+0000","last_undegraded":"2023-12-08T08:38:23.659793+0000","last_fullsized":"2023-12-08T08:38:23.659793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:47:43.142370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.326907+0000","last_change":"2023-12-08T08:38:23.673762+0000","last_active":"2023-12-08T08:38:26.326907+0000","last_peered":"2023-12-08T08:38:26.326907+0000","last_clean":"2023-12-08T08:38:26.326907+0000","last_became_active":"2023-12-08T08:38:23.673236+0000","last_became_peered":"2023-12-08T08:38:23.673236+0000","last_unstale":"2023-12-08T08:38:26.326907+0000","last_undegraded":"2023-12-08T08:38:26.326907+0000","last_fullsized":"2023-12-08T08:38:26.326907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:34:29.925547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664201+0000","last_change":"2023-12-08T08:38:19.624828+0000","last_active":"2023-12-08T08:38:23.664201+0000","last_peered":"2023-12-08T08:38:23.664201+0000","last_clean":"2023-12-08T08:38:23.664201+0000","last_became_active":"2023-12-08T08:38:19.624249+0000","last_became_peered":"2023-12-08T08:38:19.624249+0000","last_unstale":"2023-12-08T08:38:23.664201+0000","last_undegraded":"2023-12-08T08:38:23.664201+0000","last_fullsized":"2023-12-08T08:38:23.664201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.663711+0000","last_change":"2023-12-08T08:38:18.624832+0000","last_active":"2023-12-08T08:38:23.663711+0000","last_peered":"2023-12-08T08:38:23.663711+0000","last_clean":"2023-12-08T08:38:23.663711+0000","last_became_active":"2023-12-08T08:38:18.624580+0000","last_became_peered":"2023-12-08T08:38:18.624580+0000","last_unstale":"2023-12-08T08:38:23.663711+0000","last_undegraded":"2023-12-08T08:38:23.663711+0000","last_fullsized":"2023-12-08T08:38:23.663711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:13:39.901023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576884+0000","last_change":"2023-12-08T08:38:23.856627+0000","last_active":"2023-12-08T08:38:25.576884+0000","last_peered":"2023-12-08T08:38:25.576884+0000","last_clean":"2023-12-08T08:38:25.576884+0000","last_became_active":"2023-12-08T08:38:23.855845+0000","last_became_peered":"2023-12-08T08:38:23.855845+0000","last_unstale":"2023-12-08T08:38:25.576884+0000","last_undegraded":"2023-12-08T08:38:25.576884+0000","last_fullsized":"2023-12-08T08:38:25.576884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:20:54.195474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852053+0000","last_change":"2023-12-08T08:38:19.621686+0000","last_active":"2023-12-08T08:38:23.852053+0000","last_peered":"2023-12-08T08:38:23.852053+0000","last_clean":"2023-12-08T08:38:23.852053+0000","last_became_active":"2023-12-08T08:38:19.621366+0000","last_became_peered":"2023-12-08T08:38:19.621366+0000","last_unstale":"2023-12-08T08:38:23.852053+0000","last_undegraded":"2023-12-08T08:38:23.852053+0000","last_fullsized":"2023-12-08T08:38:23.852053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:18.334775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661960+0000","last_change":"2023-12-08T08:38:18.637940+0000","last_active":"2023-12-08T08:38:23.661960+0000","last_peered":"2023-12-08T08:38:23.661960+0000","last_clean":"2023-12-08T08:38:23.661960+0000","last_became_active":"2023-12-08T08:38:18.637552+0000","last_became_peered":"2023-12-08T08:38:18.637552+0000","last_unstale":"2023-12-08T08:38:23.661960+0000","last_undegraded":"2023-12-08T08:38:23.661960+0000","last_fullsized":"2023-12-08T08:38:23.661960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:14.342540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576721+0000","last_change":"2023-12-08T08:38:23.852937+0000","last_active":"2023-12-08T08:38:25.576721+0000","last_peered":"2023-12-08T08:38:25.576721+0000","last_clean":"2023-12-08T08:38:25.576721+0000","last_became_active":"2023-12-08T08:38:23.852228+0000","last_became_peered":"2023-12-08T08:38:23.852228+0000","last_unstale":"2023-12-08T08:38:25.576721+0000","last_undegraded":"2023-12-08T08:38:25.576721+0000","last_fullsized":"2023-12-08T08:38:25.576721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:46:14.364194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664273+0000","last_change":"2023-12-08T08:38:19.618681+0000","last_active":"2023-12-08T08:38:23.664273+0000","last_peered":"2023-12-08T08:38:23.664273+0000","last_clean":"2023-12-08T08:38:23.664273+0000","last_became_active":"2023-12-08T08:38:19.618102+0000","last_became_peered":"2023-12-08T08:38:19.618102+0000","last_unstale":"2023-12-08T08:38:23.664273+0000","last_undegraded":"2023-12-08T08:38:23.664273+0000","last_fullsized":"2023-12-08T08:38:23.664273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:00:01.146409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661116+0000","last_change":"2023-12-08T08:38:18.636521+0000","last_active":"2023-12-08T08:38:23.661116+0000","last_peered":"2023-12-08T08:38:23.661116+0000","last_clean":"2023-12-08T08:38:23.661116+0000","last_became_active":"2023-12-08T08:38:18.636242+0000","last_became_peered":"2023-12-08T08:38:18.636242+0000","last_unstale":"2023-12-08T08:38:23.661116+0000","last_undegraded":"2023-12-08T08:38:23.661116+0000","last_fullsized":"2023-12-08T08:38:23.661116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.626765+0000","last_change":"2023-12-08T08:38:23.855694+0000","last_active":"2023-12-08T08:38:24.626765+0000","last_peered":"2023-12-08T08:38:24.626765+0000","last_clean":"2023-12-08T08:38:24.626765+0000","last_became_active":"2023-12-08T08:38:23.855342+0000","last_became_peered":"2023-12-08T08:38:23.855342+0000","last_unstale":"2023-12-08T08:38:24.626765+0000","last_undegraded":"2023-12-08T08:38:24.626765+0000","last_fullsized":"2023-12-08T08:38:24.626765+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T20:21:44.003574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852120+0000","last_change":"2023-12-08T08:38:19.627826+0000","last_active":"2023-12-08T08:38:23.852120+0000","last_peered":"2023-12-08T08:38:23.852120+0000","last_clean":"2023-12-08T08:38:23.852120+0000","last_became_active":"2023-12-08T08:38:19.627154+0000","last_became_peered":"2023-12-08T08:38:19.627154+0000","last_unstale":"2023-12-08T08:38:23.852120+0000","last_undegraded":"2023-12-08T08:38:23.852120+0000","last_fullsized":"2023-12-08T08:38:23.852120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:11.797914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:29.035583+0000","last_change":"2023-12-08T08:38:18.630599+0000","last_active":"2023-12-08T08:38:29.035583+0000","last_peered":"2023-12-08T08:38:29.035583+0000","last_clean":"2023-12-08T08:38:29.035583+0000","last_became_active":"2023-12-08T08:38:18.630349+0000","last_became_peered":"2023-12-08T08:38:18.630349+0000","last_unstale":"2023-12-08T08:38:29.035583+0000","last_undegraded":"2023-12-08T08:38:29.035583+0000","last_fullsized":"2023-12-08T08:38:29.035583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:36:34.339195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3839,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.827898+0000","last_change":"2023-12-08T08:38:23.676381+0000","last_active":"2023-12-08T08:38:25.827898+0000","last_peered":"2023-12-08T08:38:25.827898+0000","last_clean":"2023-12-08T08:38:25.827898+0000","last_became_active":"2023-12-08T08:38:23.675561+0000","last_became_peered":"2023-12-08T08:38:23.675561+0000","last_unstale":"2023-12-08T08:38:25.827898+0000","last_undegraded":"2023-12-08T08:38:25.827898+0000","last_fullsized":"2023-12-08T08:38:25.827898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:22.372635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659110+0000","last_change":"2023-12-08T08:38:19.615536+0000","last_active":"2023-12-08T08:38:23.659110+0000","last_peered":"2023-12-08T08:38:23.659110+0000","last_clean":"2023-12-08T08:38:23.659110+0000","last_became_active":"2023-12-08T08:38:19.615243+0000","last_became_peered":"2023-12-08T08:38:19.615243+0000","last_unstale":"2023-12-08T08:38:23.659110+0000","last_undegraded":"2023-12-08T08:38:23.659110+0000","last_fullsized":"2023-12-08T08:38:23.659110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:13:40.905659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662690+0000","last_change":"2023-12-08T08:38:18.632950+0000","last_active":"2023-12-08T08:38:23.662690+0000","last_peered":"2023-12-08T08:38:23.662690+0000","last_clean":"2023-12-08T08:38:23.662690+0000","last_became_active":"2023-12-08T08:38:18.632652+0000","last_became_peered":"2023-12-08T08:38:18.632652+0000","last_unstale":"2023-12-08T08:38:23.662690+0000","last_undegraded":"2023-12-08T08:38:23.662690+0000","last_fullsized":"2023-12-08T08:38:23.662690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:36:34.339195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609288+0000","last_change":"2023-12-08T08:38:23.674379+0000","last_active":"2023-12-08T08:38:24.609288+0000","last_peered":"2023-12-08T08:38:24.609288+0000","last_clean":"2023-12-08T08:38:24.609288+0000","last_became_active":"2023-12-08T08:38:23.673581+0000","last_became_peered":"2023-12-08T08:38:23.673581+0000","last_unstale":"2023-12-08T08:38:24.609288+0000","last_undegraded":"2023-12-08T08:38:24.609288+0000","last_fullsized":"2023-12-08T08:38:24.609288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:01.683162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659853+0000","last_change":"2023-12-08T08:38:19.633344+0000","last_active":"2023-12-08T08:38:23.659853+0000","last_peered":"2023-12-08T08:38:23.659853+0000","last_clean":"2023-12-08T08:38:23.659853+0000","last_became_active":"2023-12-08T08:38:19.633035+0000","last_became_peered":"2023-12-08T08:38:19.633035+0000","last_unstale":"2023-12-08T08:38:23.659853+0000","last_undegraded":"2023-12-08T08:38:23.659853+0000","last_fullsized":"2023-12-08T08:38:23.659853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661808+0000","last_change":"2023-12-08T08:38:18.631126+0000","last_active":"2023-12-08T08:38:23.661808+0000","last_peered":"2023-12-08T08:38:23.661808+0000","last_clean":"2023-12-08T08:38:23.661808+0000","last_became_active":"2023-12-08T08:38:18.630841+0000","last_became_peered":"2023-12-08T08:38:18.630841+0000","last_unstale":"2023-12-08T08:38:23.661808+0000","last_undegraded":"2023-12-08T08:38:23.661808+0000","last_fullsized":"2023-12-08T08:38:23.661808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:07.497079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576255+0000","last_change":"2023-12-08T08:38:23.678178+0000","last_active":"2023-12-08T08:38:25.576255+0000","last_peered":"2023-12-08T08:38:25.576255+0000","last_clean":"2023-12-08T08:38:25.576255+0000","last_became_active":"2023-12-08T08:38:23.677800+0000","last_became_peered":"2023-12-08T08:38:23.677800+0000","last_unstale":"2023-12-08T08:38:25.576255+0000","last_undegraded":"2023-12-08T08:38:25.576255+0000","last_fullsized":"2023-12-08T08:38:25.576255+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:22:37.463722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661701+0000","last_change":"2023-12-08T08:38:19.631623+0000","last_active":"2023-12-08T08:38:23.661701+0000","last_peered":"2023-12-08T08:38:23.661701+0000","last_clean":"2023-12-08T08:38:23.661701+0000","last_became_active":"2023-12-08T08:38:19.630737+0000","last_became_peered":"2023-12-08T08:38:19.630737+0000","last_unstale":"2023-12-08T08:38:23.661701+0000","last_undegraded":"2023-12-08T08:38:23.661701+0000","last_fullsized":"2023-12-08T08:38:23.661701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:18.334775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665261+0000","last_change":"2023-12-08T08:38:18.630876+0000","last_active":"2023-12-08T08:38:23.665261+0000","last_peered":"2023-12-08T08:38:23.665261+0000","last_clean":"2023-12-08T08:38:23.665261+0000","last_became_active":"2023-12-08T08:38:18.630581+0000","last_became_peered":"2023-12-08T08:38:18.630581+0000","last_unstale":"2023-12-08T08:38:23.665261+0000","last_undegraded":"2023-12-08T08:38:23.665261+0000","last_fullsized":"2023-12-08T08:38:23.665261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576234+0000","last_change":"2023-12-08T08:38:23.855920+0000","last_active":"2023-12-08T08:38:25.576234+0000","last_peered":"2023-12-08T08:38:25.576234+0000","last_clean":"2023-12-08T08:38:25.576234+0000","last_became_active":"2023-12-08T08:38:23.855491+0000","last_became_peered":"2023-12-08T08:38:23.855491+0000","last_unstale":"2023-12-08T08:38:25.576234+0000","last_undegraded":"2023-12-08T08:38:25.576234+0000","last_fullsized":"2023-12-08T08:38:25.576234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:10:52.070614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.326821+0000","last_change":"2023-12-08T08:38:23.855660+0000","last_active":"2023-12-08T08:38:26.326821+0000","last_peered":"2023-12-08T08:38:26.326821+0000","last_clean":"2023-12-08T08:38:26.326821+0000","last_became_active":"2023-12-08T08:38:23.855190+0000","last_became_peered":"2023-12-08T08:38:23.855190+0000","last_unstale":"2023-12-08T08:38:26.326821+0000","last_undegraded":"2023-12-08T08:38:26.326821+0000","last_fullsized":"2023-12-08T08:38:26.326821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:48.357070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664507+0000","last_change":"2023-12-08T08:38:19.620116+0000","last_active":"2023-12-08T08:38:23.664507+0000","last_peered":"2023-12-08T08:38:23.664507+0000","last_clean":"2023-12-08T08:38:23.664507+0000","last_became_active":"2023-12-08T08:38:19.619412+0000","last_became_peered":"2023-12-08T08:38:19.619412+0000","last_unstale":"2023-12-08T08:38:23.664507+0000","last_undegraded":"2023-12-08T08:38:23.664507+0000","last_fullsized":"2023-12-08T08:38:23.664507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661919+0000","last_change":"2023-12-08T08:38:18.630228+0000","last_active":"2023-12-08T08:38:23.661919+0000","last_peered":"2023-12-08T08:38:23.661919+0000","last_clean":"2023-12-08T08:38:23.661919+0000","last_became_active":"2023-12-08T08:38:18.629445+0000","last_became_peered":"2023-12-08T08:38:18.629445+0000","last_unstale":"2023-12-08T08:38:23.661919+0000","last_undegraded":"2023-12-08T08:38:23.661919+0000","last_fullsized":"2023-12-08T08:38:23.661919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:57.526979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.328319+0000","last_change":"2023-12-08T08:38:23.678105+0000","last_active":"2023-12-08T08:38:26.328319+0000","last_peered":"2023-12-08T08:38:26.328319+0000","last_clean":"2023-12-08T08:38:26.328319+0000","last_became_active":"2023-12-08T08:38:23.677716+0000","last_became_peered":"2023-12-08T08:38:23.677716+0000","last_unstale":"2023-12-08T08:38:26.328319+0000","last_undegraded":"2023-12-08T08:38:26.328319+0000","last_fullsized":"2023-12-08T08:38:26.328319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:15.835775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655896+0000","last_change":"2023-12-08T08:38:19.628959+0000","last_active":"2023-12-08T08:38:23.655896+0000","last_peered":"2023-12-08T08:38:23.655896+0000","last_clean":"2023-12-08T08:38:23.655896+0000","last_became_active":"2023-12-08T08:38:19.628662+0000","last_became_peered":"2023-12-08T08:38:19.628662+0000","last_unstale":"2023-12-08T08:38:23.655896+0000","last_undegraded":"2023-12-08T08:38:23.655896+0000","last_fullsized":"2023-12-08T08:38:23.655896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658005+0000","last_change":"2023-12-08T08:38:18.637772+0000","last_active":"2023-12-08T08:38:23.658005+0000","last_peered":"2023-12-08T08:38:23.658005+0000","last_clean":"2023-12-08T08:38:23.658005+0000","last_became_active":"2023-12-08T08:38:18.637318+0000","last_became_peered":"2023-12-08T08:38:18.637318+0000","last_unstale":"2023-12-08T08:38:23.658005+0000","last_undegraded":"2023-12-08T08:38:23.658005+0000","last_fullsized":"2023-12-08T08:38:23.658005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:11.067589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664203+0000","last_change":"2023-12-08T08:38:18.630848+0000","last_active":"2023-12-08T08:38:23.664203+0000","last_peered":"2023-12-08T08:38:23.664203+0000","last_clean":"2023-12-08T08:38:23.664203+0000","last_became_active":"2023-12-08T08:38:18.629736+0000","last_became_peered":"2023-12-08T08:38:18.629736+0000","last_unstale":"2023-12-08T08:38:23.664203+0000","last_undegraded":"2023-12-08T08:38:23.664203+0000","last_fullsized":"2023-12-08T08:38:23.664203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:10.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661026+0000","last_change":"2023-12-08T08:38:19.612508+0000","last_active":"2023-12-08T08:38:23.661026+0000","last_peered":"2023-12-08T08:38:23.661026+0000","last_clean":"2023-12-08T08:38:23.661026+0000","last_became_active":"2023-12-08T08:38:19.612212+0000","last_became_peered":"2023-12-08T08:38:19.612212+0000","last_unstale":"2023-12-08T08:38:23.661026+0000","last_undegraded":"2023-12-08T08:38:23.661026+0000","last_fullsized":"2023-12-08T08:38:23.661026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608202+0000","last_change":"2023-12-08T08:38:23.678309+0000","last_active":"2023-12-08T08:38:24.608202+0000","last_peered":"2023-12-08T08:38:24.608202+0000","last_clean":"2023-12-08T08:38:24.608202+0000","last_became_active":"2023-12-08T08:38:23.677882+0000","last_became_peered":"2023-12-08T08:38:23.677882+0000","last_unstale":"2023-12-08T08:38:24.608202+0000","last_undegraded":"2023-12-08T08:38:24.608202+0000","last_fullsized":"2023-12-08T08:38:24.608202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:15.118488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655793+0000","last_change":"2023-12-08T08:38:18.638936+0000","last_active":"2023-12-08T08:38:23.655793+0000","last_peered":"2023-12-08T08:38:23.655793+0000","last_clean":"2023-12-08T08:38:23.655793+0000","last_became_active":"2023-12-08T08:38:18.638058+0000","last_became_peered":"2023-12-08T08:38:18.638058+0000","last_unstale":"2023-12-08T08:38:23.655793+0000","last_undegraded":"2023-12-08T08:38:23.655793+0000","last_fullsized":"2023-12-08T08:38:23.655793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:10.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658137+0000","last_change":"2023-12-08T08:38:19.619468+0000","last_active":"2023-12-08T08:38:23.658137+0000","last_peered":"2023-12-08T08:38:23.658137+0000","last_clean":"2023-12-08T08:38:23.658137+0000","last_became_active":"2023-12-08T08:38:19.619177+0000","last_became_peered":"2023-12-08T08:38:19.619177+0000","last_unstale":"2023-12-08T08:38:23.658137+0000","last_undegraded":"2023-12-08T08:38:23.658137+0000","last_fullsized":"2023-12-08T08:38:23.658137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:46:20.775412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576589+0000","last_change":"2023-12-08T08:38:23.679648+0000","last_active":"2023-12-08T08:38:25.576589+0000","last_peered":"2023-12-08T08:38:25.576589+0000","last_clean":"2023-12-08T08:38:25.576589+0000","last_became_active":"2023-12-08T08:38:23.679229+0000","last_became_peered":"2023-12-08T08:38:23.679229+0000","last_unstale":"2023-12-08T08:38:25.576589+0000","last_undegraded":"2023-12-08T08:38:25.576589+0000","last_fullsized":"2023-12-08T08:38:25.576589+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:53:05.969247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.663580+0000","last_change":"2023-12-08T08:38:19.624993+0000","last_active":"2023-12-08T08:38:23.663580+0000","last_peered":"2023-12-08T08:38:23.663580+0000","last_clean":"2023-12-08T08:38:23.663580+0000","last_became_active":"2023-12-08T08:38:19.624569+0000","last_became_peered":"2023-12-08T08:38:19.624569+0000","last_unstale":"2023-12-08T08:38:23.663580+0000","last_undegraded":"2023-12-08T08:38:23.663580+0000","last_fullsized":"2023-12-08T08:38:23.663580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662648+0000","last_change":"2023-12-08T08:38:19.631559+0000","last_active":"2023-12-08T08:38:23.662648+0000","last_peered":"2023-12-08T08:38:23.662648+0000","last_clean":"2023-12-08T08:38:23.662648+0000","last_became_active":"2023-12-08T08:38:19.631225+0000","last_became_peered":"2023-12-08T08:38:19.631225+0000","last_unstale":"2023-12-08T08:38:23.662648+0000","last_undegraded":"2023-12-08T08:38:23.662648+0000","last_fullsized":"2023-12-08T08:38:23.662648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:12.072225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661866+0000","last_change":"2023-12-08T08:38:19.631476+0000","last_active":"2023-12-08T08:38:23.661866+0000","last_peered":"2023-12-08T08:38:23.661866+0000","last_clean":"2023-12-08T08:38:23.661866+0000","last_became_active":"2023-12-08T08:38:19.630628+0000","last_became_peered":"2023-12-08T08:38:19.630628+0000","last_unstale":"2023-12-08T08:38:23.661866+0000","last_undegraded":"2023-12-08T08:38:23.661866+0000","last_fullsized":"2023-12-08T08:38:23.661866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:55.367335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.660943+0000","last_change":"2023-12-08T08:38:19.623252+0000","last_active":"2023-12-08T08:38:23.660943+0000","last_peered":"2023-12-08T08:38:23.660943+0000","last_clean":"2023-12-08T08:38:23.660943+0000","last_became_active":"2023-12-08T08:38:19.622970+0000","last_became_peered":"2023-12-08T08:38:19.622970+0000","last_unstale":"2023-12-08T08:38:23.660943+0000","last_undegraded":"2023-12-08T08:38:23.660943+0000","last_fullsized":"2023-12-08T08:38:23.660943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.657026+0000","last_change":"2023-12-08T08:38:19.621907+0000","last_active":"2023-12-08T08:38:23.657026+0000","last_peered":"2023-12-08T08:38:23.657026+0000","last_clean":"2023-12-08T08:38:23.657026+0000","last_became_active":"2023-12-08T08:38:19.621597+0000","last_became_peered":"2023-12-08T08:38:19.621597+0000","last_unstale":"2023-12-08T08:38:23.657026+0000","last_undegraded":"2023-12-08T08:38:23.657026+0000","last_fullsized":"2023-12-08T08:38:23.657026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:55.367335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662589+0000","last_change":"2023-12-08T08:38:19.621460+0000","last_active":"2023-12-08T08:38:23.662589+0000","last_peered":"2023-12-08T08:38:23.662589+0000","last_clean":"2023-12-08T08:38:23.662589+0000","last_became_active":"2023-12-08T08:38:19.621090+0000","last_became_peered":"2023-12-08T08:38:19.621090+0000","last_unstale":"2023-12-08T08:38:23.662589+0000","last_undegraded":"2023-12-08T08:38:23.662589+0000","last_fullsized":"2023-12-08T08:38:23.662589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665456+0000","last_change":"2023-12-08T08:38:19.620846+0000","last_active":"2023-12-08T08:38:23.665456+0000","last_peered":"2023-12-08T08:38:23.665456+0000","last_clean":"2023-12-08T08:38:23.665456+0000","last_became_active":"2023-12-08T08:38:19.620575+0000","last_became_peered":"2023-12-08T08:38:19.620575+0000","last_unstale":"2023-12-08T08:38:23.665456+0000","last_undegraded":"2023-12-08T08:38:23.665456+0000","last_fullsized":"2023-12-08T08:38:23.665456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656076+0000","last_change":"2023-12-08T08:38:19.623987+0000","last_active":"2023-12-08T08:38:23.656076+0000","last_peered":"2023-12-08T08:38:23.656076+0000","last_clean":"2023-12-08T08:38:23.656076+0000","last_became_active":"2023-12-08T08:38:19.623637+0000","last_became_peered":"2023-12-08T08:38:19.623637+0000","last_unstale":"2023-12-08T08:38:23.656076+0000","last_undegraded":"2023-12-08T08:38:23.656076+0000","last_fullsized":"2023-12-08T08:38:23.656076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:43.336635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661985+0000","last_change":"2023-12-08T08:38:19.627606+0000","last_active":"2023-12-08T08:38:23.661985+0000","last_peered":"2023-12-08T08:38:23.661985+0000","last_clean":"2023-12-08T08:38:23.661985+0000","last_became_active":"2023-12-08T08:38:19.627345+0000","last_became_peered":"2023-12-08T08:38:19.627345+0000","last_unstale":"2023-12-08T08:38:23.661985+0000","last_undegraded":"2023-12-08T08:38:23.661985+0000","last_fullsized":"2023-12-08T08:38:23.661985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:43.336635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.657865+0000","last_change":"2023-12-08T08:38:19.623195+0000","last_active":"2023-12-08T08:38:23.657865+0000","last_peered":"2023-12-08T08:38:23.657865+0000","last_clean":"2023-12-08T08:38:23.657865+0000","last_became_active":"2023-12-08T08:38:19.622896+0000","last_became_peered":"2023-12-08T08:38:19.622896+0000","last_unstale":"2023-12-08T08:38:23.657865+0000","last_undegraded":"2023-12-08T08:38:23.657865+0000","last_fullsized":"2023-12-08T08:38:23.657865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:59:11.073543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661771+0000","last_change":"2023-12-08T08:38:19.632017+0000","last_active":"2023-12-08T08:38:23.661771+0000","last_peered":"2023-12-08T08:38:23.661771+0000","last_clean":"2023-12-08T08:38:23.661771+0000","last_became_active":"2023-12-08T08:38:19.631771+0000","last_became_peered":"2023-12-08T08:38:19.631771+0000","last_unstale":"2023-12-08T08:38:23.661771+0000","last_undegraded":"2023-12-08T08:38:23.661771+0000","last_fullsized":"2023-12-08T08:38:23.661771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:59:11.073543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.660849+0000","last_change":"2023-12-08T08:38:19.618872+0000","last_active":"2023-12-08T08:38:23.660849+0000","last_peered":"2023-12-08T08:38:23.660849+0000","last_clean":"2023-12-08T08:38:23.660849+0000","last_became_active":"2023-12-08T08:38:19.617964+0000","last_became_peered":"2023-12-08T08:38:19.617964+0000","last_unstale":"2023-12-08T08:38:23.660849+0000","last_undegraded":"2023-12-08T08:38:23.660849+0000","last_fullsized":"2023-12-08T08:38:23.660849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665351+0000","last_change":"2023-12-08T08:38:19.622440+0000","last_active":"2023-12-08T08:38:23.665351+0000","last_peered":"2023-12-08T08:38:23.665351+0000","last_clean":"2023-12-08T08:38:23.665351+0000","last_became_active":"2023-12-08T08:38:19.621997+0000","last_became_peered":"2023-12-08T08:38:19.621997+0000","last_unstale":"2023-12-08T08:38:23.665351+0000","last_undegraded":"2023-12-08T08:38:23.665351+0000","last_fullsized":"2023-12-08T08:38:23.665351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852435+0000","last_change":"2023-12-08T08:38:19.621395+0000","last_active":"2023-12-08T08:38:23.852435+0000","last_peered":"2023-12-08T08:38:23.852435+0000","last_clean":"2023-12-08T08:38:23.852435+0000","last_became_active":"2023-12-08T08:38:19.621148+0000","last_became_peered":"2023-12-08T08:38:19.621148+0000","last_unstale":"2023-12-08T08:38:23.852435+0000","last_undegraded":"2023-12-08T08:38:23.852435+0000","last_fullsized":"2023-12-08T08:38:23.852435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:44.319210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659499+0000","last_change":"2023-12-08T08:38:19.626108+0000","last_active":"2023-12-08T08:38:23.659499+0000","last_peered":"2023-12-08T08:38:23.659499+0000","last_clean":"2023-12-08T08:38:23.659499+0000","last_became_active":"2023-12-08T08:38:19.625773+0000","last_became_peered":"2023-12-08T08:38:19.625773+0000","last_unstale":"2023-12-08T08:38:23.659499+0000","last_undegraded":"2023-12-08T08:38:23.659499+0000","last_fullsized":"2023-12-08T08:38:23.659499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:05:08.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662077+0000","last_change":"2023-12-08T08:38:19.631410+0000","last_active":"2023-12-08T08:38:23.662077+0000","last_peered":"2023-12-08T08:38:23.662077+0000","last_clean":"2023-12-08T08:38:23.662077+0000","last_became_active":"2023-12-08T08:38:19.630653+0000","last_became_peered":"2023-12-08T08:38:19.630653+0000","last_unstale":"2023-12-08T08:38:23.662077+0000","last_undegraded":"2023-12-08T08:38:23.662077+0000","last_fullsized":"2023-12-08T08:38:23.662077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:44.319210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661818+0000","last_change":"2023-12-08T08:38:19.630190+0000","last_active":"2023-12-08T08:38:23.661818+0000","last_peered":"2023-12-08T08:38:23.661818+0000","last_clean":"2023-12-08T08:38:23.661818+0000","last_became_active":"2023-12-08T08:38:19.629923+0000","last_became_peered":"2023-12-08T08:38:19.629923+0000","last_unstale":"2023-12-08T08:38:23.661818+0000","last_undegraded":"2023-12-08T08:38:23.661818+0000","last_fullsized":"2023-12-08T08:38:23.661818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664602+0000","last_change":"2023-12-08T08:38:19.620155+0000","last_active":"2023-12-08T08:38:23.664602+0000","last_peered":"2023-12-08T08:38:23.664602+0000","last_clean":"2023-12-08T08:38:23.664602+0000","last_became_active":"2023-12-08T08:38:19.619814+0000","last_became_peered":"2023-12-08T08:38:19.619814+0000","last_unstale":"2023-12-08T08:38:23.664602+0000","last_undegraded":"2023-12-08T08:38:23.664602+0000","last_fullsized":"2023-12-08T08:38:23.664602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664183+0000","last_change":"2023-12-08T08:38:19.616693+0000","last_active":"2023-12-08T08:38:23.664183+0000","last_peered":"2023-12-08T08:38:23.664183+0000","last_clean":"2023-12-08T08:38:23.664183+0000","last_became_active":"2023-12-08T08:38:19.616369+0000","last_became_peered":"2023-12-08T08:38:19.616369+0000","last_unstale":"2023-12-08T08:38:23.664183+0000","last_undegraded":"2023-12-08T08:38:23.664183+0000","last_fullsized":"2023-12-08T08:38:23.664183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:38.322348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655532+0000","last_change":"2023-12-08T08:38:19.625929+0000","last_active":"2023-12-08T08:38:23.655532+0000","last_peered":"2023-12-08T08:38:23.655532+0000","last_clean":"2023-12-08T08:38:23.655532+0000","last_became_active":"2023-12-08T08:38:19.624245+0000","last_became_peered":"2023-12-08T08:38:19.624245+0000","last_unstale":"2023-12-08T08:38:23.655532+0000","last_undegraded":"2023-12-08T08:38:23.655532+0000","last_fullsized":"2023-12-08T08:38:23.655532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659198+0000","last_change":"2023-12-08T08:38:19.625517+0000","last_active":"2023-12-08T08:38:23.659198+0000","last_peered":"2023-12-08T08:38:23.659198+0000","last_clean":"2023-12-08T08:38:23.659198+0000","last_became_active":"2023-12-08T08:38:19.625079+0000","last_became_peered":"2023-12-08T08:38:19.625079+0000","last_unstale":"2023-12-08T08:38:23.659198+0000","last_undegraded":"2023-12-08T08:38:23.659198+0000","last_fullsized":"2023-12-08T08:38:23.659198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:12.072225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665048+0000","last_change":"2023-12-08T08:38:19.632480+0000","last_active":"2023-12-08T08:38:23.665048+0000","last_peered":"2023-12-08T08:38:23.665048+0000","last_clean":"2023-12-08T08:38:23.665048+0000","last_became_active":"2023-12-08T08:38:19.632215+0000","last_became_peered":"2023-12-08T08:38:19.632215+0000","last_unstale":"2023-12-08T08:38:23.665048+0000","last_undegraded":"2023-12-08T08:38:23.665048+0000","last_fullsized":"2023-12-08T08:38:23.665048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665415+0000","last_change":"2023-12-08T08:38:19.635234+0000","last_active":"2023-12-08T08:38:23.665415+0000","last_peered":"2023-12-08T08:38:23.665415+0000","last_clean":"2023-12-08T08:38:23.665415+0000","last_became_active":"2023-12-08T08:38:19.634638+0000","last_became_peered":"2023-12-08T08:38:19.634638+0000","last_unstale":"2023-12-08T08:38:23.665415+0000","last_undegraded":"2023-12-08T08:38:23.665415+0000","last_fullsized":"2023-12-08T08:38:23.665415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.853718+0000","last_change":"2023-12-08T08:38:19.633557+0000","last_active":"2023-12-08T08:38:23.853718+0000","last_peered":"2023-12-08T08:38:23.853718+0000","last_clean":"2023-12-08T08:38:23.853718+0000","last_became_active":"2023-12-08T08:38:19.633168+0000","last_became_peered":"2023-12-08T08:38:19.633168+0000","last_unstale":"2023-12-08T08:38:23.853718+0000","last_undegraded":"2023-12-08T08:38:23.853718+0000","last_fullsized":"2023-12-08T08:38:23.853718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:18:09.854138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655258+0000","last_change":"2023-12-08T08:38:19.635104+0000","last_active":"2023-12-08T08:38:23.655258+0000","last_peered":"2023-12-08T08:38:23.655258+0000","last_clean":"2023-12-08T08:38:23.655258+0000","last_became_active":"2023-12-08T08:38:19.634711+0000","last_became_peered":"2023-12-08T08:38:19.634711+0000","last_unstale":"2023-12-08T08:38:23.655258+0000","last_undegraded":"2023-12-08T08:38:23.655258+0000","last_fullsized":"2023-12-08T08:38:23.655258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851929+0000","last_change":"2023-12-08T08:38:19.618128+0000","last_active":"2023-12-08T08:38:23.851929+0000","last_peered":"2023-12-08T08:38:23.851929+0000","last_clean":"2023-12-08T08:38:23.851929+0000","last_became_active":"2023-12-08T08:38:19.617870+0000","last_became_peered":"2023-12-08T08:38:19.617870+0000","last_unstale":"2023-12-08T08:38:23.851929+0000","last_undegraded":"2023-12-08T08:38:23.851929+0000","last_fullsized":"2023-12-08T08:38:23.851929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:32.116348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661665+0000","last_change":"2023-12-08T08:38:19.631953+0000","last_active":"2023-12-08T08:38:23.661665+0000","last_peered":"2023-12-08T08:38:23.661665+0000","last_clean":"2023-12-08T08:38:23.661665+0000","last_became_active":"2023-12-08T08:38:19.631638+0000","last_became_peered":"2023-12-08T08:38:19.631638+0000","last_unstale":"2023-12-08T08:38:23.661665+0000","last_undegraded":"2023-12-08T08:38:23.661665+0000","last_fullsized":"2023-12-08T08:38:23.661665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851995+0000","last_change":"2023-12-08T08:38:19.621369+0000","last_active":"2023-12-08T08:38:23.851995+0000","last_peered":"2023-12-08T08:38:23.851995+0000","last_clean":"2023-12-08T08:38:23.851995+0000","last_became_active":"2023-12-08T08:38:19.621103+0000","last_became_peered":"2023-12-08T08:38:19.621103+0000","last_unstale":"2023-12-08T08:38:23.851995+0000","last_undegraded":"2023-12-08T08:38:23.851995+0000","last_fullsized":"2023-12-08T08:38:23.851995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:46:57.645302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661240+0000","last_change":"2023-12-08T08:38:19.630541+0000","last_active":"2023-12-08T08:38:23.661240+0000","last_peered":"2023-12-08T08:38:23.661240+0000","last_clean":"2023-12-08T08:38:23.661240+0000","last_became_active":"2023-12-08T08:38:19.630175+0000","last_became_peered":"2023-12-08T08:38:19.630175+0000","last_unstale":"2023-12-08T08:38:23.661240+0000","last_undegraded":"2023-12-08T08:38:23.661240+0000","last_fullsized":"2023-12-08T08:38:23.661240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:49:29.898351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664269+0000","last_change":"2023-12-08T08:38:19.626884+0000","last_active":"2023-12-08T08:38:23.664269+0000","last_peered":"2023-12-08T08:38:23.664269+0000","last_clean":"2023-12-08T08:38:23.664269+0000","last_became_active":"2023-12-08T08:38:19.626525+0000","last_became_peered":"2023-12-08T08:38:19.626525+0000","last_unstale":"2023-12-08T08:38:23.664269+0000","last_undegraded":"2023-12-08T08:38:23.664269+0000","last_fullsized":"2023-12-08T08:38:23.664269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664251+0000","last_change":"2023-12-08T08:38:19.621125+0000","last_active":"2023-12-08T08:38:23.664251+0000","last_peered":"2023-12-08T08:38:23.664251+0000","last_clean":"2023-12-08T08:38:23.664251+0000","last_became_active":"2023-12-08T08:38:19.620831+0000","last_became_peered":"2023-12-08T08:38:19.620831+0000","last_unstale":"2023-12-08T08:38:23.664251+0000","last_undegraded":"2023-12-08T08:38:23.664251+0000","last_fullsized":"2023-12-08T08:38:23.664251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852760+0000","last_change":"2023-12-08T08:38:19.624484+0000","last_active":"2023-12-08T08:38:23.852760+0000","last_peered":"2023-12-08T08:38:23.852760+0000","last_clean":"2023-12-08T08:38:23.852760+0000","last_became_active":"2023-12-08T08:38:19.623921+0000","last_became_peered":"2023-12-08T08:38:19.623921+0000","last_unstale":"2023-12-08T08:38:23.852760+0000","last_undegraded":"2023-12-08T08:38:23.852760+0000","last_fullsized":"2023-12-08T08:38:23.852760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:32:54.146197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4637,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":99918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":99952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":690798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-08T08:38:43.073 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-08T08:38:43.073 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-08T08:38:43.074 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-12-08T08:38:43.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-08T08:38:43.074 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-12-08T08:38:43.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.212+0000 7f76518ec640 1 Processor -- start 2023-12-08T08:38:43.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.212+0000 7f76518ec640 1 -- start start 2023-12-08T08:38:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.213+0000 7f76518ec640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f764c1608a0 0x7f764c0d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.213+0000 7f76518ec640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f764c0b0cf0 0x7f764c0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.213+0000 7f76518ec640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764c0d20b0 con 0x7f764c0b1d70 2023-12-08T08:38:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.213+0000 7f76518ec640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764c0d0620 con 0x7f764c0b0cf0 2023-12-08T08:38:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.213+0000 7f76518ec640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0d1b80 con 0x7f764c1608a0 2023-12-08T08:38:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.213+0000 7f764b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f764c0b0cf0 0x7f764c0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.213+0000 7f764b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f764c0b0cf0 0x7f764c0cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52352/0 (socket says 172.21.15.16:52352) 2023-12-08T08:38:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.213+0000 7f764b7fe640 1 -- 172.21.15.16:0/3557934875 learned_addr learned my addr 172.21.15.16:0/3557934875 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:43.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.214+0000 7f764affd640 1 --2- 172.21.15.16:0/3557934875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f764c1608a0 0x7f764c0d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.214+0000 7f764b7fe640 1 -- 172.21.15.16:0/3557934875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f764c1608a0 msgr2=0x7f764c0d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.214+0000 7f764b7fe640 1 --2- 172.21.15.16:0/3557934875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f764c1608a0 0x7f764c0d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.214+0000 7f764b7fe640 1 -- 172.21.15.16:0/3557934875 >> v1:172.21.15.112:6789/0 conn(0x7f764c0b1d70 legacy=0x7f764c0d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.214+0000 7f764b7fe640 1 -- 172.21.15.16:0/3557934875 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c0d1dd0 con 0x7f764c0b0cf0 2023-12-08T08:38:43.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.214+0000 7f764b7fe640 1 --2- 172.21.15.16:0/3557934875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764c0b0cf0 0x7f764c0cdc40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7644009e50 tx=0x7f764402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e4202d2e9321 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:43.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.215+0000 7f764a7fc640 1 -- 172.21.15.16:0/3557934875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764402ca70 con 0x7f764c0b0cf0 2023-12-08T08:38:43.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.215+0000 7f764a7fc640 1 -- 172.21.15.16:0/3557934875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764402cbd0 con 0x7f764c0b0cf0 2023-12-08T08:38:43.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.215+0000 7f764a7fc640 1 -- 172.21.15.16:0/3557934875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76440343f0 con 0x7f764c0b0cf0 2023-12-08T08:38:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.215+0000 7f76518ec640 1 -- 172.21.15.16:0/3557934875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764c0b0cf0 msgr2=0x7f764c0cdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.215+0000 7f76518ec640 1 --2- 172.21.15.16:0/3557934875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764c0b0cf0 0x7f764c0cdc40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7644009e50 tx=0x7f764402bed0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.215+0000 7f76518ec640 1 -- 172.21.15.16:0/3557934875 shutdown_connections 2023-12-08T08:38:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.215+0000 7f76518ec640 1 --2- 172.21.15.16:0/3557934875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764c0b0cf0 0x7f764c0cdc40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.215+0000 7f76518ec640 1 --2- 172.21.15.16:0/3557934875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f764c1608a0 0x7f764c0d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.215+0000 7f76518ec640 1 -- 172.21.15.16:0/3557934875 >> 172.21.15.16:0/3557934875 conn(0x7f764c0d3030 msgr2=0x7f764c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.215+0000 7f76518ec640 1 -- 172.21.15.16:0/3557934875 shutdown_connections 2023-12-08T08:38:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.215+0000 7f76518ec640 1 -- 172.21.15.16:0/3557934875 wait complete. 2023-12-08T08:38:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.216+0000 7f76518ec640 1 Processor -- start 2023-12-08T08:38:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.217+0000 7f76518ec640 1 -- start start 2023-12-08T08:38:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.217+0000 7f76518ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764c0b0cf0 0x7f764c0c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.217+0000 7f76518ec640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f764c0b1d70 0x7f764c1f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.217+0000 7f76518ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764c1608a0 0x7f764c15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.217+0000 7f76518ec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c1f29d0 con 0x7f764c1608a0 2023-12-08T08:38:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.217+0000 7f76518ec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c15cdf0 con 0x7f764c0b0cf0 2023-12-08T08:38:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.217+0000 7f76518ec640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0c7160 con 0x7f764c0b1d70 2023-12-08T08:38:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.217+0000 7f764b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764c1608a0 0x7f764c15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.217+0000 7f764b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764c1608a0 0x7f764c15c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52360/0 (socket says 172.21.15.16:52360) 2023-12-08T08:38:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.217+0000 7f764b7fe640 1 -- 172.21.15.16:0/658212988 learned_addr learned my addr 172.21.15.16:0/658212988 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.218+0000 7f764affd640 1 --2- 172.21.15.16:0/658212988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764c0b0cf0 0x7f764c0c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.218+0000 7f7643fff640 1 --2- 172.21.15.16:0/658212988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f764c0b1d70 0x7f764c1f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.218+0000 7f764b7fe640 1 -- 172.21.15.16:0/658212988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f764c0b1d70 msgr2=0x7f764c1f2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.218+0000 7f764b7fe640 1 --2- 172.21.15.16:0/658212988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f764c0b1d70 0x7f764c1f2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.218+0000 7f764b7fe640 1 -- 172.21.15.16:0/658212988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764c0b0cf0 msgr2=0x7f764c0c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.218+0000 7f764b7fe640 1 --2- 172.21.15.16:0/658212988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764c0b0cf0 0x7f764c0c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.218+0000 7f764b7fe640 1 -- 172.21.15.16:0/658212988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7644009b30 con 0x7f764c1608a0 2023-12-08T08:38:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.218+0000 7f764b7fe640 1 --2- 172.21.15.16:0/658212988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764c1608a0 0x7f764c15c8b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7644009e50 tx=0x7f7644004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.218+0000 7f7648ff9640 1 -- 172.21.15.16:0/658212988 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644009510 con 0x7f764c1608a0 2023-12-08T08:38:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.218+0000 7f7648ff9640 1 -- 172.21.15.16:0/658212988 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764403e450 con 0x7f764c1608a0 2023-12-08T08:38:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.218+0000 7f76518ec640 1 -- 172.21.15.16:0/658212988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c0c7440 con 0x7f764c1608a0 2023-12-08T08:38:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.218+0000 7f7648ff9640 1 -- 172.21.15.16:0/658212988 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764403e800 con 0x7f764c1608a0 2023-12-08T08:38:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.218+0000 7f76518ec640 1 -- 172.21.15.16:0/658212988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c0c7a00 con 0x7f764c1608a0 2023-12-08T08:38:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.219+0000 7f76518ec640 1 -- 172.21.15.16:0/658212988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c067430 con 0x7f764c1608a0 2023-12-08T08:38:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.219+0000 7f7648ff9640 1 -- 172.21.15.16:0/658212988 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f764403e960 con 0x7f764c1608a0 2023-12-08T08:38:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.220+0000 7f7648ff9640 1 --2- 172.21.15.16:0/658212988 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f762c06ae40 0x7f762c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.220+0000 7f7648ff9640 1 -- 172.21.15.16:0/658212988 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f76440ac640 con 0x7f764c1608a0 2023-12-08T08:38:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.220+0000 7f764affd640 1 --2- 172.21.15.16:0/658212988 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f762c06ae40 0x7f762c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.221+0000 7f764affd640 1 --2- 172.21.15.16:0/658212988 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f762c06ae40 0x7f762c06d300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f763c006fd0 tx=0x7f763c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:43.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.224+0000 7f7648ff9640 1 -- 172.21.15.16:0/658212988 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7644077490 con 0x7f764c1608a0 2023-12-08T08:38:43.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.447+0000 7f76518ec640 1 -- 172.21.15.16:0/658212988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f764c0d35b0 con 0x7f764c1608a0 2023-12-08T08:38:43.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.450+0000 7f7648ff9640 1 -- 172.21.15.16:0/658212988 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171036 (secure 0 0 0) 0x7f764407aab0 con 0x7f764c1608a0 2023-12-08T08:38:43.456 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.453+0000 7f76518ec640 1 -- 172.21.15.16:0/658212988 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f762c06ae40 msgr2=0x7f762c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.453+0000 7f76518ec640 1 --2- 172.21.15.16:0/658212988 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f762c06ae40 0x7f762c06d300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f763c006fd0 tx=0x7f763c008040 comp rx=0 tx=0).stop 2023-12-08T08:38:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.453+0000 7f76518ec640 1 -- 172.21.15.16:0/658212988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764c1608a0 msgr2=0x7f764c15c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.453+0000 7f76518ec640 1 --2- 172.21.15.16:0/658212988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764c1608a0 0x7f764c15c8b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7644009e50 tx=0x7f7644004a40 comp rx=0 tx=0).stop 2023-12-08T08:38:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.453+0000 7f76518ec640 1 -- 172.21.15.16:0/658212988 shutdown_connections 2023-12-08T08:38:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.453+0000 7f76518ec640 1 --2- 172.21.15.16:0/658212988 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f762c06ae40 0x7f762c06d300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.453+0000 7f76518ec640 1 --2- 172.21.15.16:0/658212988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f764c1608a0 0x7f764c15c8b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.453+0000 7f76518ec640 1 --2- 172.21.15.16:0/658212988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f764c0b1d70 0x7f764c1f2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.453+0000 7f76518ec640 1 --2- 172.21.15.16:0/658212988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f764c0b0cf0 0x7f764c0c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.453+0000 7f76518ec640 1 -- 172.21.15.16:0/658212988 >> 172.21.15.16:0/658212988 conn(0x7f764c0d3030 msgr2=0x7f764c0d22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:43.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.453+0000 7f76518ec640 1 -- 172.21.15.16:0/658212988 shutdown_connections 2023-12-08T08:38:43.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.453+0000 7f76518ec640 1 -- 172.21.15.16:0/658212988 wait complete. 2023-12-08T08:38:43.466 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6832","nonce":804067392},{"type":"v1","addr":"172.21.15.112:6833","nonce":804067392}]},"active_addr":"172.21.15.112:6833/804067392","active_change":"2023-12-08T08:37:58.962709+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4098,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":3000954695}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":2680901631}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":1383055365}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.112:0","nonce":1426255716}]}]} 2023-12-08T08:38:43.469 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-12-08T08:38:43.469 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-12-08T08:38:43.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:43.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.573+0000 7fdebaeb4640 1 Processor -- start 2023-12-08T08:38:43.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.573+0000 7fdebaeb4640 1 -- start start 2023-12-08T08:38:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdebaeb4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdeb4165ed0 0x7fdeb41682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdebaeb4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeb40ce480 con 0x7fdeb40b6ac0 2023-12-08T08:38:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdebaeb4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeb40cd200 con 0x7fdeb40cfb30 2023-12-08T08:38:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdebaeb4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4160cb0 con 0x7fdeb4165ed0 2023-12-08T08:38:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdeb8c50640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdeb40b6ac0 0x7fdeb40cd660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59696/0 (socket says 172.21.15.16:59696) 2023-12-08T08:38:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdeb8c50640 1 -- 172.21.15.16:0/1757582901 learned_addr learned my addr 172.21.15.16:0/1757582901 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdeb9451640 1 --2- 172.21.15.16:0/1757582901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdeb4165ed0 0x7fdeb41682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214977953 0 0) 0x7fdeb40ce480 con 0x7fdeb40b6ac0 2023-12-08T08:38:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde9c003520 con 0x7fdeb40b6ac0 2023-12-08T08:38:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767954309 0 0) 0x7fdeb40cd200 con 0x7fdeb40cfb30 2023-12-08T08:38:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdeb40ce480 con 0x7fdeb40cfb30 2023-12-08T08:38:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3652053833 0 0) 0x7fde9c003520 con 0x7fdeb40b6ac0 2023-12-08T08:38:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdeb4165ed0 msgr2=0x7fdeb41682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdeb37fe640 1 --2- 172.21.15.16:0/1757582901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdeb4165ed0 0x7fdeb41682c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 >> v1:172.21.15.112:6789/0 conn(0x7fdeb40cfb30 legacy=0x7fdeb40cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb40d02b0 con 0x7fdeb40b6ac0 2023-12-08T08:38:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.574+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7fdea4001670 con 0x7fdeb40b6ac0 2023-12-08T08:38:43.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 >> v1:172.21.15.16:6789/0 conn(0x7fdeb40b6ac0 legacy=0x7fdeb40cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb37fe640 1 --2- 172.21.15.16:0/1757582901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde9c003b30 0x7fde9c003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb37fe640 1 --2- 172.21.15.16:0/1757582901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdeb4165ed0 0x7fde9c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb37fe640 1 --2- 172.21.15.16:0/1757582901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c004d00 0x7fde9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4160cb0 con 0x7fde9c004d00 2023-12-08T08:38:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c00bb40 con 0x7fde9c003b30 2023-12-08T08:38:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c00bd00 con 0x7fdeb4165ed0 2023-12-08T08:38:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb9451640 1 --2- 172.21.15.16:0/1757582901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c004d00 0x7fde9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb8c50640 1 --2- 172.21.15.16:0/1757582901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde9c003b30 0x7fde9c003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb3fff640 1 --2- 172.21.15.16:0/1757582901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdeb4165ed0 0x7fde9c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb9451640 1 -- 172.21.15.16:0/1757582901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdeb4165ed0 msgr2=0x7fde9c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb9451640 1 --2- 172.21.15.16:0/1757582901 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdeb4165ed0 0x7fde9c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb9451640 1 -- 172.21.15.16:0/1757582901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde9c003b30 msgr2=0x7fde9c003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb9451640 1 --2- 172.21.15.16:0/1757582901 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fde9c003b30 0x7fde9c003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.575+0000 7fdeb9451640 1 -- 172.21.15.16:0/1757582901 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c003520 con 0x7fde9c004d00 2023-12-08T08:38:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.576+0000 7fdeb9451640 1 --2- 172.21.15.16:0/1757582901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c004d00 0x7fde9c005180 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009f90 tx=0x7fde9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8d05de7ec6fbfe server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.576+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac02cbd0 con 0x7fde9c004d00 2023-12-08T08:38:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.576+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac00f040 con 0x7fde9c004d00 2023-12-08T08:38:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.576+0000 7fdeb37fe640 1 -- 172.21.15.16:0/1757582901 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac034440 con 0x7fde9c004d00 2023-12-08T08:38:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.576+0000 7fdebaeb4640 1 -- 172.21.15.16:0/1757582901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c004d00 msgr2=0x7fde9c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.576+0000 7fdebaeb4640 1 --2- 172.21.15.16:0/1757582901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c004d00 0x7fde9c005180 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009f90 tx=0x7fde9c0057f0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.576+0000 7fdeb9451640 1 -- 172.21.15.16:0/1757582901 reap_dead start 2023-12-08T08:38:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.576+0000 7fdebaeb4640 1 -- 172.21.15.16:0/1757582901 shutdown_connections 2023-12-08T08:38:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.576+0000 7fdebaeb4640 1 -- 172.21.15.16:0/1757582901 >> 172.21.15.16:0/1757582901 conn(0x7fdeb40d0de0 msgr2=0x7fdeb40d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.576+0000 7fdebaeb4640 1 -- 172.21.15.16:0/1757582901 shutdown_connections 2023-12-08T08:38:43.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.576+0000 7fdebaeb4640 1 -- 172.21.15.16:0/1757582901 wait complete. 2023-12-08T08:38:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.578+0000 7fdebaeb4640 1 Processor -- start 2023-12-08T08:38:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.578+0000 7fdebaeb4640 1 -- start start 2023-12-08T08:38:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.578+0000 7fdebaeb4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c004d00 0x7fdeb415d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.578+0000 7fdebaeb4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdeb414cbc0 0x7fdeb41505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.578+0000 7fdebaeb4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdeb4150b10 0x7fdeb40b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.578+0000 7fdebaeb4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4159180 con 0x7fde9c004d00 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdebaeb4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4151840 con 0x7fdeb414cbc0 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdebaeb4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb4151280 con 0x7fdeb4150b10 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb8c50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c004d00 0x7fdeb415d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb8c50640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c004d00 0x7fdeb415d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52376/0 (socket says 172.21.15.16:52376) 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb8c50640 1 -- 172.21.15.16:0/3995406974 learned_addr learned my addr 172.21.15.16:0/3995406974 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb3fff640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdeb414cbc0 0x7fdeb41505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb9451640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdeb4150b10 0x7fdeb40b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb8c50640 1 -- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdeb4150b10 msgr2=0x7fdeb40b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb8c50640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdeb4150b10 0x7fdeb40b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb8c50640 1 -- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdeb414cbc0 msgr2=0x7fdeb41505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb8c50640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdeb414cbc0 0x7fdeb41505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb8c50640 1 -- 172.21.15.16:0/3995406974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb4151560 con 0x7fde9c004d00 2023-12-08T08:38:43.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb3fff640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdeb414cbc0 0x7fdeb41505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb8c50640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c004d00 0x7fdeb415d180 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdea4007340 tx=0x7fdea4007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb17fa640 1 -- 172.21.15.16:0/3995406974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400f860 con 0x7fde9c004d00 2023-12-08T08:38:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb17fa640 1 -- 172.21.15.16:0/3995406974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea400f9e0 con 0x7fde9c004d00 2023-12-08T08:38:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdebaeb4640 1 -- 172.21.15.16:0/3995406974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeb415c980 con 0x7fde9c004d00 2023-12-08T08:38:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdeb17fa640 1 -- 172.21.15.16:0/3995406974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400fd90 con 0x7fde9c004d00 2023-12-08T08:38:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.579+0000 7fdebaeb4640 1 -- 172.21.15.16:0/3995406974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb4150f90 con 0x7fde9c004d00 2023-12-08T08:38:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.580+0000 7fdebaeb4640 1 -- 172.21.15.16:0/3995406974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeb415cea0 con 0x7fde9c004d00 2023-12-08T08:38:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.581+0000 7fdeb17fa640 1 -- 172.21.15.16:0/3995406974 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdea401d460 con 0x7fde9c004d00 2023-12-08T08:38:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.581+0000 7fdeb17fa640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdea006ae40 0x7fdea006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.581+0000 7fdeb17fa640 1 -- 172.21.15.16:0/3995406974 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdea4089a90 con 0x7fde9c004d00 2023-12-08T08:38:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.581+0000 7fdeb3fff640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdea006ae40 0x7fdea006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.582+0000 7fdeb3fff640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdea006ae40 0x7fdea006d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdea80010e0 tx=0x7fdea8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:43.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.586+0000 7fdeb17fa640 1 -- 172.21.15.16:0/3995406974 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdeb415cea0 con 0x7fde9c004d00 2023-12-08T08:38:43.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.768+0000 7fdebaeb4640 1 -- 172.21.15.16:0/3995406974 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdeb415cea0 con 0x7fde9c004d00 2023-12-08T08:38:43.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.769+0000 7fdeb17fa640 1 -- 172.21.15.16:0/3995406974 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fdeb415cea0 con 0x7fde9c004d00 2023-12-08T08:38:43.772 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:43.772 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:24.030625+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:43.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.771+0000 7fdebaeb4640 1 -- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdea006ae40 msgr2=0x7fdea006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.771+0000 7fdebaeb4640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdea006ae40 0x7fdea006d300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdea80010e0 tx=0x7fdea8009040 comp rx=0 tx=0).stop 2023-12-08T08:38:43.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.771+0000 7fdebaeb4640 1 -- 172.21.15.16:0/3995406974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c004d00 msgr2=0x7fdeb415d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.771+0000 7fdebaeb4640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c004d00 0x7fdeb415d180 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdea4007340 tx=0x7fdea4007800 comp rx=0 tx=0).stop 2023-12-08T08:38:43.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.771+0000 7fdebaeb4640 1 -- 172.21.15.16:0/3995406974 shutdown_connections 2023-12-08T08:38:43.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.771+0000 7fdebaeb4640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdea006ae40 0x7fdea006d300 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.771+0000 7fdebaeb4640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdeb4150b10 0x7fdeb40b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.771+0000 7fdebaeb4640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdeb414cbc0 0x7fdeb41505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.771+0000 7fdebaeb4640 1 --2- 172.21.15.16:0/3995406974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c004d00 0x7fdeb415d180 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.771+0000 7fdebaeb4640 1 -- 172.21.15.16:0/3995406974 >> 172.21.15.16:0/3995406974 conn(0x7fdeb40d0de0 msgr2=0x7fdeb4160480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:43.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.771+0000 7fdebaeb4640 1 -- 172.21.15.16:0/3995406974 shutdown_connections 2023-12-08T08:38:43.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.771+0000 7fdebaeb4640 1 -- 172.21.15.16:0/3995406974 wait complete. 2023-12-08T08:38:43.785 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-12-08T08:38:43.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:43.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.889+0000 7fa7835dc640 1 Processor -- start 2023-12-08T08:38:43.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.889+0000 7fa7835dc640 1 -- start start 2023-12-08T08:38:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.890+0000 7fa7835dc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa77c15d390 0x7fa77c0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.890+0000 7fa7835dc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c0ced90 con 0x7fa77c06cf50 2023-12-08T08:38:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.890+0000 7fa7835dc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c0ae4f0 con 0x7fa77c0b1330 2023-12-08T08:38:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.890+0000 7fa7835dc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77c0aeab0 con 0x7fa77c15d390 2023-12-08T08:38:43.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.890+0000 7fa781378640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa77c15d390 0x7fa77c0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.890+0000 7fa780b77640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa77c06cf50 0x7fa77c0b0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59708/0 (socket says 172.21.15.16:59708) 2023-12-08T08:38:43.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.890+0000 7fa780b77640 1 -- 172.21.15.16:0/726420748 learned_addr learned my addr 172.21.15.16:0/726420748 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:43.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.890+0000 7fa781378640 1 -- 172.21.15.16:0/726420748 >> v1:172.21.15.112:6789/0 conn(0x7fa77c0b1330 legacy=0x7fa77c0c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:43.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.890+0000 7fa781378640 1 -- 172.21.15.16:0/726420748 >> v1:172.21.15.16:6789/0 conn(0x7fa77c06cf50 legacy=0x7fa77c0b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.890+0000 7fa773fff640 1 -- 172.21.15.16:0/726420748 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320706913 0 0) 0x7fa77c0ced90 con 0x7fa77c06cf50 2023-12-08T08:38:43.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.890+0000 7fa781378640 1 -- 172.21.15.16:0/726420748 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c0ae7d0 con 0x7fa77c15d390 2023-12-08T08:38:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.891+0000 7fa781378640 1 --2- 172.21.15.16:0/726420748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa77c15d390 0x7fa77c0ce740 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa7780097a0 tx=0x7fa77802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e208d74bdcdde81 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.891+0000 7fa773fff640 1 -- 172.21.15.16:0/726420748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778031830 con 0x7fa77c15d390 2023-12-08T08:38:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.891+0000 7fa773fff640 1 -- 172.21.15.16:0/726420748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa778031990 con 0x7fa77c15d390 2023-12-08T08:38:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.891+0000 7fa773fff640 1 -- 172.21.15.16:0/726420748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778031d20 con 0x7fa77c15d390 2023-12-08T08:38:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.892+0000 7fa7835dc640 1 -- 172.21.15.16:0/726420748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa77c15d390 msgr2=0x7fa77c0ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.892+0000 7fa7835dc640 1 --2- 172.21.15.16:0/726420748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa77c15d390 0x7fa77c0ce740 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa7780097a0 tx=0x7fa77802b8d0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.892+0000 7fa7835dc640 1 -- 172.21.15.16:0/726420748 shutdown_connections 2023-12-08T08:38:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.892+0000 7fa7835dc640 1 --2- 172.21.15.16:0/726420748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa77c15d390 0x7fa77c0ce740 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.892+0000 7fa7835dc640 1 -- 172.21.15.16:0/726420748 >> 172.21.15.16:0/726420748 conn(0x7fa77c0cf040 msgr2=0x7fa77c0cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.892+0000 7fa7835dc640 1 -- 172.21.15.16:0/726420748 shutdown_connections 2023-12-08T08:38:43.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.892+0000 7fa7835dc640 1 -- 172.21.15.16:0/726420748 wait complete. 2023-12-08T08:38:43.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.893+0000 7fa7835dc640 1 Processor -- start 2023-12-08T08:38:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.893+0000 7fa7835dc640 1 -- start start 2023-12-08T08:38:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa7835dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa77c06cf50 0x7fa77c154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa7835dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa77c0b1330 0x7fa77c1f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa7835dc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa77c15d390 0x7fa77c0c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa7835dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c1546d0 con 0x7fa77c06cf50 2023-12-08T08:38:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa7835dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c15cc60 con 0x7fa77c0b1330 2023-12-08T08:38:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa7835dc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c1f6fe0 con 0x7fa77c15d390 2023-12-08T08:38:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa781378640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa77c06cf50 0x7fa77c154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa781378640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa77c06cf50 0x7fa77c154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52404/0 (socket says 172.21.15.16:52404) 2023-12-08T08:38:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa781378640 1 -- 172.21.15.16:0/3406106027 learned_addr learned my addr 172.21.15.16:0/3406106027 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa781378640 1 -- 172.21.15.16:0/3406106027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa77c15d390 msgr2=0x7fa77c0c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa781378640 1 --2- 172.21.15.16:0/3406106027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa77c15d390 0x7fa77c0c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa781378640 1 -- 172.21.15.16:0/3406106027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa77c0b1330 msgr2=0x7fa77c1f6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:43.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa781378640 1 --2- 172.21.15.16:0/3406106027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa77c0b1330 0x7fa77c1f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:43.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa781378640 1 -- 172.21.15.16:0/3406106027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778009400 con 0x7fa77c06cf50 2023-12-08T08:38:43.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa781378640 1 --2- 172.21.15.16:0/3406106027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa77c06cf50 0x7fa77c154190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa778000c00 tx=0x7fa7780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:43.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa771ffb640 1 -- 172.21.15.16:0/3406106027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa77803aa60 con 0x7fa77c06cf50 2023-12-08T08:38:43.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa771ffb640 1 -- 172.21.15.16:0/3406106027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa77803abc0 con 0x7fa77c06cf50 2023-12-08T08:38:43.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa771ffb640 1 -- 172.21.15.16:0/3406106027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7780443f0 con 0x7fa77c06cf50 2023-12-08T08:38:43.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.894+0000 7fa7835dc640 1 -- 172.21.15.16:0/3406106027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77c071700 con 0x7fa77c06cf50 2023-12-08T08:38:43.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.895+0000 7fa7835dc640 1 -- 172.21.15.16:0/3406106027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77c0b2100 con 0x7fa77c06cf50 2023-12-08T08:38:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.896+0000 7fa7835dc640 1 -- 172.21.15.16:0/3406106027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77c0b1c80 con 0x7fa77c06cf50 2023-12-08T08:38:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.896+0000 7fa771ffb640 1 -- 172.21.15.16:0/3406106027 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa77804d450 con 0x7fa77c06cf50 2023-12-08T08:38:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.897+0000 7fa771ffb640 1 --2- 172.21.15.16:0/3406106027 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa76006ae40 0x7fa76006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.897+0000 7fa771ffb640 1 -- 172.21.15.16:0/3406106027 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa7780aba80 con 0x7fa77c06cf50 2023-12-08T08:38:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.897+0000 7fa780b77640 1 --2- 172.21.15.16:0/3406106027 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa76006ae40 0x7fa76006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:43.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.897+0000 7fa780b77640 1 --2- 172.21.15.16:0/3406106027 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa76006ae40 0x7fa76006d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa76c007fd0 tx=0x7fa76c007900 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:43.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:43.901+0000 7fa771ffb640 1 -- 172.21.15.16:0/3406106027 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa77c0b1c80 con 0x7fa77c06cf50 2023-12-08T08:38:44.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.076+0000 7fa7835dc640 1 -- 172.21.15.16:0/3406106027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa77c0b1c60 con 0x7fa77c06cf50 2023-12-08T08:38:44.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.077+0000 7fa771ffb640 1 -- 172.21.15.16:0/3406106027 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fa778035070 con 0x7fa77c06cf50 2023-12-08T08:38:44.081 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:44.081 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:24.030625+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.079+0000 7fa7835dc640 1 -- 172.21.15.16:0/3406106027 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa76006ae40 msgr2=0x7fa76006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.079+0000 7fa7835dc640 1 --2- 172.21.15.16:0/3406106027 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa76006ae40 0x7fa76006d300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa76c007fd0 tx=0x7fa76c007900 comp rx=0 tx=0).stop 2023-12-08T08:38:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.079+0000 7fa7835dc640 1 -- 172.21.15.16:0/3406106027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa77c06cf50 msgr2=0x7fa77c154190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.079+0000 7fa7835dc640 1 --2- 172.21.15.16:0/3406106027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa77c06cf50 0x7fa77c154190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa778000c00 tx=0x7fa7780046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.079+0000 7fa7835dc640 1 -- 172.21.15.16:0/3406106027 shutdown_connections 2023-12-08T08:38:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.079+0000 7fa7835dc640 1 --2- 172.21.15.16:0/3406106027 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa76006ae40 0x7fa76006d300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.079+0000 7fa7835dc640 1 --2- 172.21.15.16:0/3406106027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa77c15d390 0x7fa77c0c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.079+0000 7fa7835dc640 1 --2- 172.21.15.16:0/3406106027 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa77c0b1330 0x7fa77c1f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.079+0000 7fa7835dc640 1 --2- 172.21.15.16:0/3406106027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa77c06cf50 0x7fa77c154190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.079+0000 7fa7835dc640 1 -- 172.21.15.16:0/3406106027 >> 172.21.15.16:0/3406106027 conn(0x7fa77c0cf040 msgr2=0x7fa77c0acad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.080+0000 7fa7835dc640 1 -- 172.21.15.16:0/3406106027 shutdown_connections 2023-12-08T08:38:44.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.080+0000 7fa7835dc640 1 -- 172.21.15.16:0/3406106027 wait complete. 2023-12-08T08:38:44.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-12-08T08:38:44.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-12-08T08:38:44.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-12-08T08:38:44.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-12-08T08:38:44.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-12-08T08:38:44.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-12-08T08:38:44.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-12-08T08:38:44.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-12-08T08:38:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.276+0000 7f361730f640 1 Processor -- start 2023-12-08T08:38:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.276+0000 7f259d43d640 1 Processor -- start 2023-12-08T08:38:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.276+0000 7fac51561640 1 Processor -- start 2023-12-08T08:38:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.277+0000 7f45f24bb640 1 Processor -- start 2023-12-08T08:38:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.277+0000 7fac51561640 1 -- start start 2023-12-08T08:38:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.276+0000 7f259d43d640 1 -- start start 2023-12-08T08:38:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.277+0000 7f45f24bb640 1 -- start start 2023-12-08T08:38:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7f259d43d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2598159580 0x7f2598073740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7f259d43d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2598159960 con 0x7f25980b2c30 2023-12-08T08:38:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7f259d43d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259815d1d0 con 0x7f25980cb590 2023-12-08T08:38:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7f259d43d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259815de50 con 0x7f2598159580 2023-12-08T08:38:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.277+0000 7f361730f640 1 -- start start 2023-12-08T08:38:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7fac51561640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac4c06e2b0 0x7fac4c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7fac51561640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4c0cf460 con 0x7fac4c159f80 2023-12-08T08:38:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7fac51561640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac4c15a3c0 con 0x7fac4c15e850 2023-12-08T08:38:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7fac51561640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4c160e90 con 0x7fac4c06e2b0 2023-12-08T08:38:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7fac4a7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fac4c159f80 0x7fac4c06dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59718/0 (socket says 172.21.15.16:59718) 2023-12-08T08:38:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7fac4affd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fac4c15e850 0x7fac4c0cde90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:33984/0 (socket says 172.21.15.16:33984) 2023-12-08T08:38:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7fac4affd640 1 -- 172.21.15.16:0/2939995036 learned_addr learned my addr 172.21.15.16:0/2939995036 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7fac4b7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fac4c06e2b0 0x7fac4c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7fac49ffb640 1 -- 172.21.15.16:0/2939995036 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136149222 0 0) 0x7fac4c15a3c0 con 0x7fac4c15e850 2023-12-08T08:38:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7fac49ffb640 1 -- 172.21.15.16:0/2939995036 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac34003520 con 0x7fac4c15e850 2023-12-08T08:38:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f45f24bb640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f45e40e7850 0x7f45e4050410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f45f24bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45e40071b0 0x7f45e4050950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f45f24bb640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e40539a0 con 0x7f45e40e7470 2023-12-08T08:38:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f45f24bb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e4050e90 con 0x7f45e40071b0 2023-12-08T08:38:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f45f24bb640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e4051090 con 0x7f45e40e7850 2023-12-08T08:38:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f45f1cba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45e40071b0 0x7f45e4050950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f45f1cba640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45e40071b0 0x7f45e4050950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52422/0 (socket says 172.21.15.16:52422) 2023-12-08T08:38:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f45f1cba640 1 -- 172.21.15.16:0/3333890423 learned_addr learned my addr 172.21.15.16:0/3333890423 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f45f1cba640 1 -- 172.21.15.16:0/3333890423 >> v2:172.21.15.112:3300/0 conn(0x7f45e40e7850 msgr2=0x7f45e4050410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-08T08:38:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f45f1cba640 1 --2- 172.21.15.16:0/3333890423 >> v2:172.21.15.112:3300/0 conn(0x7f45e40e7850 0x7f45e4050410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f45f1cba640 1 -- 172.21.15.16:0/3333890423 >> v1:172.21.15.112:6789/0 conn(0x7f45e40e7470 legacy=0x7f45e4006aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f45f1cba640 1 -- 172.21.15.16:0/3333890423 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e4051370 con 0x7f45e40071b0 2023-12-08T08:38:44.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7f37517dc640 1 Processor -- start 2023-12-08T08:38:44.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7f37517dc640 1 -- start start 2023-12-08T08:38:44.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7f25977fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f25980b2c30 0x7f259806a560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59714/0 (socket says 172.21.15.16:59714) 2023-12-08T08:38:44.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7f25977fe640 1 -- 172.21.15.16:0/3677282068 learned_addr learned my addr 172.21.15.16:0/3677282068 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:44.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f2595ffb640 1 -- 172.21.15.16:0/3677282068 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205039859 0 0) 0x7f2598159960 con 0x7f25980b2c30 2023-12-08T08:38:44.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f2595ffb640 1 -- 172.21.15.16:0/3677282068 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2580003520 con 0x7f25980b2c30 2023-12-08T08:38:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7fac49ffb640 1 -- 172.21.15.16:0/2939995036 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081712619 0 0) 0x7fac4c0cf460 con 0x7fac4c159f80 2023-12-08T08:38:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7f361730f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36100c3d50 0x7f36100c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7f361730f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f361006b5e0 0x7f36100d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7f361730f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f361006ba40 0x7f36100d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.278+0000 7f361730f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36100cd990 con 0x7f361006ba40 2023-12-08T08:38:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f361730f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36100cdf50 con 0x7f361006b5e0 2023-12-08T08:38:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f361730f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36101626a0 con 0x7f36100c3d50 2023-12-08T08:38:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f36158ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f361006ba40 0x7f36100d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f36158ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f361006ba40 0x7f36100d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52408/0 (socket says 172.21.15.16:52408) 2023-12-08T08:38:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f36158ac640 1 -- 172.21.15.16:0/3722687510 learned_addr learned my addr 172.21.15.16:0/3722687510 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7fac49ffb640 1 -- 172.21.15.16:0/2939995036 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac4c15a3c0 con 0x7fac4c159f80 2023-12-08T08:38:44.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f36148aa640 1 --2- 172.21.15.16:0/3722687510 >> v2:172.21.15.112:3300/0 conn(0x7f361006b5e0 0x7f36100d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f36158ac640 1 -- 172.21.15.16:0/3722687510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36100c3d50 msgr2=0x7f36100c4130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-08T08:38:44.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f36158ac640 1 --2- 172.21.15.16:0/3722687510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36100c3d50 0x7f36100c4130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f2596ffd640 1 --2- 172.21.15.16:0/3677282068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2598159580 0x7f2598073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2939995036 >> v1:172.21.15.112:6789/0 conn(0x7fac4c15e850 legacy=0x7fac4c0cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2939995036 >> v1:172.21.15.16:6789/0 conn(0x7fac4c159f80 legacy=0x7fac4c06dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f2595ffb640 1 -- 172.21.15.16:0/3677282068 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2399774760 0 0) 0x7f2580003520 con 0x7f25980b2c30 2023-12-08T08:38:44.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.279+0000 7f19ceda9640 1 Processor -- start 2023-12-08T08:38:44.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7fac4b7fe640 1 -- 172.21.15.16:0/2939995036 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c0ce5a0 con 0x7fac4c06e2b0 2023-12-08T08:38:44.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f2595ffb640 1 -- 172.21.15.16:0/3677282068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2598159580 msgr2=0x7f2598073740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f2595ffb640 1 --2- 172.21.15.16:0/3677282068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2598159580 0x7f2598073740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f2595ffb640 1 -- 172.21.15.16:0/3677282068 >> v1:172.21.15.112:6789/0 conn(0x7f25980cb590 legacy=0x7f25980b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f2595ffb640 1 -- 172.21.15.16:0/3677282068 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259815d4b0 con 0x7f25980b2c30 2023-12-08T08:38:44.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f19ceda9640 1 -- start start 2023-12-08T08:38:44.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f2595ffb640 1 -- 172.21.15.16:0/3677282068 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f25900019f0 con 0x7f25980b2c30 2023-12-08T08:38:44.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f2595ffb640 1 -- 172.21.15.16:0/3677282068 >> v1:172.21.15.16:6789/0 conn(0x7f25980b2c30 legacy=0x7f259806a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f37517dc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f374c160a00 0x7f374c0d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f37517dc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f374c0cffe0 0x7f374c0b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f37517dc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f374c0b28e0 con 0x7f374c0b1ea0 2023-12-08T08:38:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f37517dc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f374c0b2610 con 0x7f374c160a00 2023-12-08T08:38:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f37517dc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374c0ce7c0 con 0x7f374c0cffe0 2023-12-08T08:38:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f374b7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f374c0cffe0 0x7f374c0b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f374b7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f374c0cffe0 0x7f374c0b2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58098/0 (socket says 172.21.15.16:58098) 2023-12-08T08:38:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f374b7fe640 1 -- 172.21.15.16:0/1754844007 learned_addr learned my addr 172.21.15.16:0/1754844007 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f36158ac640 1 -- 172.21.15.16:0/3722687510 >> v2:172.21.15.112:3300/0 conn(0x7f361006b5e0 msgr2=0x7f36100d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f36158ac640 1 --2- 172.21.15.16:0/3722687510 >> v2:172.21.15.112:3300/0 conn(0x7f361006b5e0 0x7f36100d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f36158ac640 1 -- 172.21.15.16:0/3722687510 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36100c4670 con 0x7f361006ba40 2023-12-08T08:38:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f36148aa640 1 --2- 172.21.15.16:0/3722687510 >> v2:172.21.15.112:3300/0 conn(0x7f361006b5e0 0x7f36100d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f2595ffb640 1 --2- 172.21.15.16:0/3677282068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25800039a0 0x7f2580003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f2595ffb640 1 --2- 172.21.15.16:0/3677282068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25800047c0 0x7f2580004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7fd63359e640 1 Processor -- start 2023-12-08T08:38:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7f2595ffb640 1 --2- 172.21.15.16:0/3677282068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2598159580 0x7f258000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f2595ffb640 1 -- 172.21.15.16:0/3677282068 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259815de50 con 0x7f25800047c0 2023-12-08T08:38:44.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7fb363a89640 1 Processor -- start 2023-12-08T08:38:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f374affd640 1 --2- 172.21.15.16:0/1754844007 >> v2:172.21.15.16:3300/0 conn(0x7f374c160a00 0x7f374c0d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f25977fe640 1 --2- 172.21.15.16:0/3677282068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2598159580 0x7f258000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f45f1cba640 1 --2- 172.21.15.16:0/3333890423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e40071b0 0x7f45e4050950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0684d0 tx=0x7f45ec0a5c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36b9d1dae84ec61 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f25967fc640 1 --2- 172.21.15.16:0/3677282068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25800047c0 0x7f2580004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f2596ffd640 1 --2- 172.21.15.16:0/3677282068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25800039a0 0x7f2580003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f2595ffb640 1 -- 172.21.15.16:0/3677282068 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25800055c0 con 0x7f25800039a0 2023-12-08T08:38:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f2595ffb640 1 -- 172.21.15.16:0/3677282068 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258000bec0 con 0x7f2598159580 2023-12-08T08:38:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7fac4b7fe640 1 --2- 172.21.15.16:0/2939995036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac4c06e2b0 0x7fac4c0caea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fac44009d90 tx=0x7fac4402bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=674bbff79f22d63e server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7fac49ffb640 1 -- 172.21.15.16:0/2939995036 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4402ccf0 con 0x7fac4c06e2b0 2023-12-08T08:38:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f45e3fff640 1 -- 172.21.15.16:0/3333890423 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0a6a70 con 0x7f45e40071b0 2023-12-08T08:38:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7fac49ffb640 1 -- 172.21.15.16:0/2939995036 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4402ce50 con 0x7fac4c06e2b0 2023-12-08T08:38:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7fac49ffb640 1 -- 172.21.15.16:0/2939995036 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac44034660 con 0x7fac4c06e2b0 2023-12-08T08:38:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f45e3fff640 1 -- 172.21.15.16:0/3333890423 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec0a6c10 con 0x7f45e40071b0 2023-12-08T08:38:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7fb363a89640 1 -- start start 2023-12-08T08:38:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f374affd640 1 -- 172.21.15.16:0/1754844007 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f374c0cffe0 msgr2=0x7f374c0b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f374affd640 1 --2- 172.21.15.16:0/1754844007 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f374c0cffe0 0x7f374c0b2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f374affd640 1 -- 172.21.15.16:0/1754844007 >> v1:172.21.15.16:6789/0 conn(0x7f374c0b1ea0 legacy=0x7f374c0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f374affd640 1 -- 172.21.15.16:0/1754844007 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c0ceaa0 con 0x7f374c160a00 2023-12-08T08:38:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f19ceda9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f19c00df1e0 0x7f19c0038040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f19ceda9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19c0053c30 0x7f19c004fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f19ceda9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c0054480 con 0x7f19c0033eb0 2023-12-08T08:38:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f19ceda9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c00541a0 con 0x7f19c00df1e0 2023-12-08T08:38:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.281+0000 7f19ceda9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c0005ef0 con 0x7f19c0053c30 2023-12-08T08:38:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f19ccb45640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f19c00df1e0 0x7f19c0038040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f19ccb45640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f19c00df1e0 0x7f19c0038040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:45936/0 (socket says 172.21.15.16:45936) 2023-12-08T08:38:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f19ccb45640 1 -- 172.21.15.16:0/2335551654 learned_addr learned my addr 172.21.15.16:0/2335551654 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f19c7fff640 1 --2- 172.21.15.16:0/2335551654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19c0053c30 0x7f19c004fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f19ccb45640 1 -- 172.21.15.16:0/2335551654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19c0053c30 msgr2=0x7f19c004fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f19ccb45640 1 --2- 172.21.15.16:0/2335551654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19c0053c30 0x7f19c004fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f36158ac640 1 --2- 172.21.15.16:0/3722687510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361006ba40 0x7f36100d1250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f36100d1e20 tx=0x7f360002b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf0e02c5857b4277 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f19ccb45640 1 -- 172.21.15.16:0/2335551654 >> v1:172.21.15.16:6789/0 conn(0x7f19c0033eb0 legacy=0x7f19c00069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.280+0000 7fd63359e640 1 -- start start 2023-12-08T08:38:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7fd63359e640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd634161250 0x7fd63406e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7fd63359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd634162640 0x7fd6340707f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7fb363a89640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb35c146460 0x7fb35c06c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f45f24bb640 1 -- 172.21.15.16:0/3333890423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e40071b0 msgr2=0x7f45e4050950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7fb363a89640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb35c146f50 0x7fb35c0cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7fb363a89640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35c1447b0 con 0x7fb35c06c5a0 2023-12-08T08:38:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7fb363a89640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35c06c9e0 con 0x7fb35c146f50 2023-12-08T08:38:44.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7fb363a89640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c161cb0 con 0x7fb35c146460 2023-12-08T08:38:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f45f24bb640 1 --2- 172.21.15.16:0/3333890423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e40071b0 0x7f45e4050950 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0684d0 tx=0x7f45ec0a5c00 comp rx=0 tx=0).stop 2023-12-08T08:38:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f374affd640 1 --2- 172.21.15.16:0/1754844007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f374c160a00 0x7f374c0d3a60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3744009d90 tx=0x7f374402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711b604cb432b7e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fac51561640 1 -- 172.21.15.16:0/2939995036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac4c06e2b0 msgr2=0x7fac4c0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7f45f24bb640 1 -- 172.21.15.16:0/3333890423 shutdown_connections 2023-12-08T08:38:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7f45f24bb640 1 --2- 172.21.15.16:0/3333890423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e40071b0 0x7f45e4050950 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7f45f24bb640 1 --2- 172.21.15.16:0/3333890423 >> v2:172.21.15.112:3300/0 conn(0x7f45e40e7850 0x7f45e4050410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7f45f24bb640 1 -- 172.21.15.16:0/3333890423 >> 172.21.15.16:0/3333890423 conn(0x7f45e4055ba0 msgr2=0x7f45e4055fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fb362026640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb35c146f50 0x7fb35c0cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fac51561640 1 --2- 172.21.15.16:0/2939995036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac4c06e2b0 0x7fac4c0caea0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fac44009d90 tx=0x7fac4402bf40 comp rx=0 tx=0).stop 2023-12-08T08:38:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fac51561640 1 -- 172.21.15.16:0/2939995036 shutdown_connections 2023-12-08T08:38:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fac51561640 1 --2- 172.21.15.16:0/2939995036 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac4c06e2b0 0x7fac4c0caea0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fac51561640 1 -- 172.21.15.16:0/2939995036 >> 172.21.15.16:0/2939995036 conn(0x7fac4c0b1860 msgr2=0x7fac4c0b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7f3749ffb640 1 -- 172.21.15.16:0/1754844007 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374402cbd0 con 0x7f374c160a00 2023-12-08T08:38:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fac51561640 1 -- 172.21.15.16:0/2939995036 shutdown_connections 2023-12-08T08:38:44.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fac51561640 1 -- 172.21.15.16:0/2939995036 wait complete. 2023-12-08T08:38:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fb362026640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb35c146f50 0x7fb35c0cda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52450/0 (socket says 172.21.15.16:52450) 2023-12-08T08:38:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fb362026640 1 -- 172.21.15.16:0/1125120851 learned_addr learned my addr 172.21.15.16:0/1125120851 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7f3749ffb640 1 -- 172.21.15.16:0/1754844007 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374402cd30 con 0x7f374c160a00 2023-12-08T08:38:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7f3749ffb640 1 -- 172.21.15.16:0/1754844007 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374402cea0 con 0x7f374c160a00 2023-12-08T08:38:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f19ccb45640 1 -- 172.21.15.16:0/2335551654 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c00061d0 con 0x7f19c00df1e0 2023-12-08T08:38:44.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7f45f24bb640 1 -- 172.21.15.16:0/3333890423 shutdown_connections 2023-12-08T08:38:44.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fb361825640 1 --2- 172.21.15.16:0/1125120851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb35c146460 0x7fb35c06c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7f3607fff640 1 -- 172.21.15.16:0/3722687510 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600031890 con 0x7f361006ba40 2023-12-08T08:38:44.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7f3607fff640 1 -- 172.21.15.16:0/3722687510 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36000319f0 con 0x7f361006ba40 2023-12-08T08:38:44.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f361730f640 1 -- 172.21.15.16:0/3722687510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361006ba40 msgr2=0x7f36100d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f361730f640 1 --2- 172.21.15.16:0/3722687510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361006ba40 0x7f36100d1250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f36100d1e20 tx=0x7f360002b930 comp rx=0 tx=0).stop 2023-12-08T08:38:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f25977fe640 1 -- 172.21.15.16:0/3677282068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25800039a0 msgr2=0x7f2580003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f25977fe640 1 --2- 172.21.15.16:0/3677282068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25800039a0 0x7f2580003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f25977fe640 1 -- 172.21.15.16:0/3677282068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25800047c0 msgr2=0x7f2580004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f25977fe640 1 --2- 172.21.15.16:0/3677282068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25800047c0 0x7f2580004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.282+0000 7f25977fe640 1 -- 172.21.15.16:0/3677282068 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2580003520 con 0x7f2598159580 2023-12-08T08:38:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7f2596ffd640 1 --2- 172.21.15.16:0/3677282068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25800039a0 0x7f2580003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7f19ccb45640 1 --2- 172.21.15.16:0/2335551654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19c00df1e0 0x7f19c0038040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0091e0 tx=0x7f19bc02cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f553e1f5d6e400e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fd631d9b640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd634070390 0x7fd634164eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:34002/0 (socket says 172.21.15.16:34002) 2023-12-08T08:38:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fd631d9b640 1 -- 172.21.15.16:0/1880249146 learned_addr learned my addr 172.21.15.16:0/1880249146 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fd63359e640 1 -- 172.21.15.16:0/1880249146 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd634070d30 con 0x7fd634070390 2023-12-08T08:38:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fd63359e640 1 -- 172.21.15.16:0/1880249146 --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6340b2f60 con 0x7fd634162640 2023-12-08T08:38:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fd63359e640 1 -- 172.21.15.16:0/1880249146 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd634072790 con 0x7fd634161250 2023-12-08T08:38:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7fd632d9d640 1 --2- 172.21.15.16:0/1880249146 >> v2:172.21.15.16:3300/0 conn(0x7fd634162640 0x7fd6340707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f361730f640 1 -- 172.21.15.16:0/3722687510 shutdown_connections 2023-12-08T08:38:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f361730f640 1 --2- 172.21.15.16:0/3722687510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361006ba40 0x7f36100d1250 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f361730f640 1 --2- 172.21.15.16:0/3722687510 >> v2:172.21.15.112:3300/0 conn(0x7f361006b5e0 0x7f36100d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f361730f640 1 --2- 172.21.15.16:0/3722687510 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36100c3d50 0x7f36100c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f361730f640 1 -- 172.21.15.16:0/3722687510 >> 172.21.15.16:0/3722687510 conn(0x7f3610099a10 msgr2=0x7f3610099e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fb362026640 1 -- 172.21.15.16:0/1125120851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb35c146460 msgr2=0x7fb35c06c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fb362026640 1 --2- 172.21.15.16:0/1125120851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb35c146460 0x7fb35c06c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.283+0000 7fb362026640 1 -- 172.21.15.16:0/1125120851 >> v1:172.21.15.16:6789/0 conn(0x7fb35c06c5a0 legacy=0x7fb35c146840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7fb362026640 1 -- 172.21.15.16:0/1125120851 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c0b35e0 con 0x7fb35c146f50 2023-12-08T08:38:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7fd63259c640 1 --2- 172.21.15.16:0/1880249146 >> v2:172.21.15.112:3300/0 conn(0x7fd634161250 0x7fd63406e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f361730f640 1 -- 172.21.15.16:0/3722687510 shutdown_connections 2023-12-08T08:38:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f19c77fe640 1 -- 172.21.15.16:0/2335551654 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc02d9d0 con 0x7f19c00df1e0 2023-12-08T08:38:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f19c77fe640 1 -- 172.21.15.16:0/2335551654 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc02db30 con 0x7f19c00df1e0 2023-12-08T08:38:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f19c77fe640 1 -- 172.21.15.16:0/2335551654 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc035440 con 0x7f19c00df1e0 2023-12-08T08:38:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7fd632d9d640 1 -- 172.21.15.16:0/1880249146 >> v2:172.21.15.112:3300/0 conn(0x7fd634161250 msgr2=0x7fd63406e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7fd632d9d640 1 --2- 172.21.15.16:0/1880249146 >> v2:172.21.15.112:3300/0 conn(0x7fd634161250 0x7fd63406e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f37517dc640 1 -- 172.21.15.16:0/1754844007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f374c160a00 msgr2=0x7f374c0d3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f25977fe640 1 --2- 172.21.15.16:0/3677282068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2598159580 0x7f258000b980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f259000a760 tx=0x7f25800057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=962e8b7e3ad873a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f37517dc640 1 --2- 172.21.15.16:0/1754844007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f374c160a00 0x7f374c0d3a60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3744009d90 tx=0x7f374402be20 comp rx=0 tx=0).stop 2023-12-08T08:38:44.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f2595ffb640 1 -- 172.21.15.16:0/3677282068 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590034630 con 0x7f2598159580 2023-12-08T08:38:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7fb362026640 1 --2- 172.21.15.16:0/1125120851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c146f50 0x7fb35c0cda60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb358009ad0 tx=0x7fb35802bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadad3cc1da5fd89 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f2595ffb640 1 -- 172.21.15.16:0/3677282068 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259002f070 con 0x7f2598159580 2023-12-08T08:38:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f19ceda9640 1 -- 172.21.15.16:0/2335551654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19c00df1e0 msgr2=0x7f19c0038040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f19ceda9640 1 --2- 172.21.15.16:0/2335551654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19c00df1e0 0x7f19c0038040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0091e0 tx=0x7f19bc02cc20 comp rx=0 tx=0).stop 2023-12-08T08:38:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7fb360823640 1 -- 172.21.15.16:0/1125120851 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35802d700 con 0x7fb35c146f50 2023-12-08T08:38:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7fb360823640 1 -- 172.21.15.16:0/1125120851 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35802d860 con 0x7fb35c146f50 2023-12-08T08:38:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7fb360823640 1 -- 172.21.15.16:0/1125120851 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35802dbf0 con 0x7fb35c146f50 2023-12-08T08:38:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f2595ffb640 1 -- 172.21.15.16:0/3677282068 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590034a10 con 0x7f2598159580 2023-12-08T08:38:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f19ceda9640 1 -- 172.21.15.16:0/2335551654 shutdown_connections 2023-12-08T08:38:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f19ceda9640 1 --2- 172.21.15.16:0/2335551654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19c0053c30 0x7f19c004fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7fac51561640 1 Processor -- start 2023-12-08T08:38:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f19ceda9640 1 --2- 172.21.15.16:0/2335551654 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19c00df1e0 0x7f19c0038040 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0091e0 tx=0x7f19bc02cc20 comp rx=0 tx=0).stop 2023-12-08T08:38:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f19ceda9640 1 -- 172.21.15.16:0/2335551654 >> 172.21.15.16:0/2335551654 conn(0x7f19c0009f70 msgr2=0x7f19c000a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f45f24bb640 1 -- 172.21.15.16:0/3333890423 wait complete. 2023-12-08T08:38:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f37517dc640 1 -- 172.21.15.16:0/1754844007 shutdown_connections 2023-12-08T08:38:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f37517dc640 1 --2- 172.21.15.16:0/1754844007 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f374c0cffe0 0x7f374c0b2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f37517dc640 1 --2- 172.21.15.16:0/1754844007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f374c160a00 0x7f374c0d3a60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f37517dc640 1 -- 172.21.15.16:0/1754844007 >> 172.21.15.16:0/1754844007 conn(0x7f374c09c300 msgr2=0x7f374c0d2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f37517dc640 1 -- 172.21.15.16:0/1754844007 shutdown_connections 2023-12-08T08:38:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7f361730f640 1 -- 172.21.15.16:0/3722687510 wait complete. 2023-12-08T08:38:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7fd632d9d640 1 -- 172.21.15.16:0/1880249146 >> v1:172.21.15.112:6789/0 conn(0x7fd634070390 legacy=0x7fd634164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7fd632d9d640 1 -- 172.21.15.16:0/1880249146 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6340b22e0 con 0x7fd634162640 2023-12-08T08:38:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.284+0000 7fd63259c640 1 --2- 172.21.15.16:0/1880249146 >> v2:172.21.15.112:3300/0 conn(0x7fd634161250 0x7fd63406e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f19ceda9640 1 -- 172.21.15.16:0/2335551654 shutdown_connections 2023-12-08T08:38:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f19ceda9640 1 -- 172.21.15.16:0/2335551654 wait complete. 2023-12-08T08:38:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7fac51561640 1 -- start start 2023-12-08T08:38:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.286+0000 7f259d43d640 1 -- 172.21.15.16:0/3677282068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2598159580 msgr2=0x7f258000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.286+0000 7f259d43d640 1 --2- 172.21.15.16:0/3677282068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2598159580 0x7f258000b980 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f259000a760 tx=0x7f25800057f0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.286+0000 7fb363a89640 1 -- 172.21.15.16:0/1125120851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c146f50 msgr2=0x7fb35c0cda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.286+0000 7f25977fe640 1 -- 172.21.15.16:0/3677282068 reap_dead start 2023-12-08T08:38:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.286+0000 7f259d43d640 1 -- 172.21.15.16:0/3677282068 shutdown_connections 2023-12-08T08:38:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.286+0000 7f259d43d640 1 -- 172.21.15.16:0/3677282068 >> 172.21.15.16:0/3677282068 conn(0x7f25980cf470 msgr2=0x7f25980cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.286+0000 7f361730f640 1 Processor -- start 2023-12-08T08:38:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.286+0000 7f259d43d640 1 -- 172.21.15.16:0/3677282068 shutdown_connections 2023-12-08T08:38:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.285+0000 7f37517dc640 1 -- 172.21.15.16:0/1754844007 wait complete. 2023-12-08T08:38:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.286+0000 7f259d43d640 1 -- 172.21.15.16:0/3677282068 wait complete. 2023-12-08T08:38:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.286+0000 7fb363a89640 1 --2- 172.21.15.16:0/1125120851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c146f50 0x7fb35c0cda60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb358009ad0 tx=0x7fb35802bb50 comp rx=0 tx=0).stop 2023-12-08T08:38:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.286+0000 7fac51561640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4c06e2b0 0x7fac4c166810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.286+0000 7fd632d9d640 1 --2- 172.21.15.16:0/1880249146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd634162640 0x7fd6340707f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd634072a80 tx=0x7fd62c02bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67b45a7144a03c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fac51561640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac4c159f80 0x7fac4c0c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fac51561640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac4c15e850 0x7fac4c200490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fac4affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4c06e2b0 0x7fac4c166810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fac4affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4c06e2b0 0x7fac4c166810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52452/0 (socket says 172.21.15.16:52452) 2023-12-08T08:38:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fd63159a640 1 -- 172.21.15.16:0/1880249146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c02ca30 con 0x7fd634162640 2023-12-08T08:38:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fd63159a640 1 -- 172.21.15.16:0/1880249146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c02cb90 con 0x7fd634162640 2023-12-08T08:38:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fd63159a640 1 -- 172.21.15.16:0/1880249146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c034440 con 0x7fd634162640 2023-12-08T08:38:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fac4affd640 1 -- 172.21.15.16:0/202933939 learned_addr learned my addr 172.21.15.16:0/202933939 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7f45f24bb640 1 Processor -- start 2023-12-08T08:38:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7f45f24bb640 1 -- start start 2023-12-08T08:38:44.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.286+0000 7f361730f640 1 -- start start 2023-12-08T08:38:44.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fac51561640 1 -- 172.21.15.16:0/202933939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c166df0 con 0x7fac4c06e2b0 2023-12-08T08:38:44.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fb363a89640 1 -- 172.21.15.16:0/1125120851 shutdown_connections 2023-12-08T08:38:44.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fb363a89640 1 --2- 172.21.15.16:0/1125120851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c146f50 0x7fb35c0cda60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fb363a89640 1 --2- 172.21.15.16:0/1125120851 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb35c146460 0x7fb35c06c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fb363a89640 1 -- 172.21.15.16:0/1125120851 >> 172.21.15.16:0/1125120851 conn(0x7fb35c0b0750 msgr2=0x7fb35c0b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7f361730f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361006b5e0 0x7f361015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7f37517dc640 1 Processor -- start 2023-12-08T08:38:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7f361730f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361006ba40 0x7f36100c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7f361730f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36100c3d50 0x7f36100c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7f361730f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361015d720 con 0x7f361006b5e0 2023-12-08T08:38:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7f361730f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36100c4cf0 con 0x7f361006ba40 2023-12-08T08:38:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7f361730f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36100c4fd0 con 0x7f36100c3d50 2023-12-08T08:38:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f19ceda9640 1 Processor -- start 2023-12-08T08:38:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7fd63359e640 1 -- 172.21.15.16:0/1880249146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd634162640 msgr2=0x7fd6340707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f36150ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361006b5e0 0x7f361015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f36150ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361006b5e0 0x7f361015d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52462/0 (socket says 172.21.15.16:52462) 2023-12-08T08:38:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f36150ab640 1 -- 172.21.15.16:0/2867515219 learned_addr learned my addr 172.21.15.16:0/2867515219 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f19ceda9640 1 -- start start 2023-12-08T08:38:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f36158ac640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36100c3d50 0x7f36100c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f259d43d640 1 Processor -- start 2023-12-08T08:38:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f37517dc640 1 -- start start 2023-12-08T08:38:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7fd63359e640 1 --2- 172.21.15.16:0/1880249146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd634162640 0x7fd6340707f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd634072a80 tx=0x7fd62c02bd00 comp rx=0 tx=0).stop 2023-12-08T08:38:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f259d43d640 1 -- start start 2023-12-08T08:38:44.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f36150ab640 1 -- 172.21.15.16:0/2867515219 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36100c3d50 msgr2=0x7f36100c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f45f24bb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45e40071b0 0x7f45e404f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f45f24bb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45e40e7470 0x7f45e4046d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f45f24bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e40e7850 0x7f45e4044760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f36148aa640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361006ba40 0x7f36100c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f45f24bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e404d680 con 0x7f45e40e7850 2023-12-08T08:38:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7fd63359e640 1 -- 172.21.15.16:0/1880249146 shutdown_connections 2023-12-08T08:38:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7fd63359e640 1 --2- 172.21.15.16:0/1880249146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd634162640 0x7fd6340707f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7fd63359e640 1 --2- 172.21.15.16:0/1880249146 >> v2:172.21.15.112:3300/0 conn(0x7fd634161250 0x7fd63406e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7fd63359e640 1 -- 172.21.15.16:0/1880249146 >> 172.21.15.16:0/1880249146 conn(0x7fd63406c540 msgr2=0x7fd63406c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f45f1cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e40e7850 0x7f45e4044760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f36150ab640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36100c3d50 0x7f36100c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f45f14b9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45e40071b0 0x7f45e404f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f45f1cba640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e40e7850 0x7f45e4044760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52464/0 (socket says 172.21.15.16:52464) 2023-12-08T08:38:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f45f1cba640 1 -- 172.21.15.16:0/925702457 learned_addr learned my addr 172.21.15.16:0/925702457 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f45f24bb640 1 -- 172.21.15.16:0/925702457 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e404fba0 con 0x7f45e40071b0 2023-12-08T08:38:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f45f24bb640 1 -- 172.21.15.16:0/925702457 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e404d890 con 0x7f45e40e7470 2023-12-08T08:38:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7f36150ab640 1 -- 172.21.15.16:0/2867515219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361006ba40 msgr2=0x7f36100c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7fd63359e640 1 -- 172.21.15.16:0/1880249146 shutdown_connections 2023-12-08T08:38:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fac51561640 1 -- 172.21.15.16:0/202933939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c2009d0 con 0x7fac4c15e850 2023-12-08T08:38:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fac51561640 1 -- 172.21.15.16:0/202933939 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac4c15bc50 con 0x7fac4c159f80 2023-12-08T08:38:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.287+0000 7fac4b7fe640 1 --2- 172.21.15.16:0/202933939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac4c15e850 0x7fac4c200490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.288+0000 7fac4a7fc640 1 --2- 172.21.15.16:0/202933939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac4c159f80 0x7fac4c0c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f36150ab640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361006ba40 0x7f36100c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f36150ab640 1 -- 172.21.15.16:0/2867515219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c009400 con 0x7f361006b5e0 2023-12-08T08:38:44.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f37517dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f374c0b1ea0 0x7f374c148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f36158ac640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36100c3d50 0x7f36100c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:44.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f45f1cba640 1 -- 172.21.15.16:0/925702457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45e40e7470 msgr2=0x7f45e4046d50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-08T08:38:44.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f45f1cba640 1 --2- 172.21.15.16:0/925702457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45e40e7470 0x7f45e4046d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f45f1cba640 1 -- 172.21.15.16:0/925702457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45e40071b0 msgr2=0x7f45e404f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f45f1cba640 1 --2- 172.21.15.16:0/925702457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45e40071b0 0x7f45e404f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f45f1cba640 1 -- 172.21.15.16:0/925702457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ec068760 con 0x7f45e40e7850 2023-12-08T08:38:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f36148aa640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f361006ba40 0x7f36100c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7fd63359e640 1 -- 172.21.15.16:0/1880249146 wait complete. 2023-12-08T08:38:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f374affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f374c0b1ea0 0x7f374c148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f37517dc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f374c0cffe0 0x7f374c0c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f37517dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f374c160a00 0x7f374c15b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f37517dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c146a60 con 0x7f374c0b1ea0 2023-12-08T08:38:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f37517dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374c145b90 con 0x7f374c160a00 2023-12-08T08:38:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f37517dc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374c148b40 con 0x7f374c0cffe0 2023-12-08T08:38:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f374affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f374c0b1ea0 0x7f374c148600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52472/0 (socket says 172.21.15.16:52472) 2023-12-08T08:38:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f374affd640 1 -- 172.21.15.16:0/1718088925 learned_addr learned my addr 172.21.15.16:0/1718088925 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f45f1cba640 1 --2- 172.21.15.16:0/925702457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e40e7850 0x7f45e4044760 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0697e0 tx=0x7f45ec069bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f259d43d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25981508e0 0x7f259815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.289+0000 7f19ceda9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19c0033eb0 0x7f19c00d4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7fac4b7fe640 1 -- 172.21.15.16:0/202933939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac4c159f80 msgr2=0x7fac4c0c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7fac4b7fe640 1 --2- 172.21.15.16:0/202933939 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fac4c159f80 0x7fac4c0c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7fac4b7fe640 1 -- 172.21.15.16:0/202933939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4c06e2b0 msgr2=0x7fac4c166810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7fac4b7fe640 1 --2- 172.21.15.16:0/202933939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac4c06e2b0 0x7fac4c166810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7fac4b7fe640 1 -- 172.21.15.16:0/202933939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac44009a70 con 0x7fac4c15e850 2023-12-08T08:38:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f19ceda9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19c0053c30 0x7f19c01754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f259d43d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25980ccaf0 0x7f25980ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f374affd640 1 -- 172.21.15.16:0/1718088925 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f374c0cffe0 msgr2=0x7f374c0c8e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-08T08:38:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f374affd640 1 --2- 172.21.15.16:0/1718088925 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f374c0cffe0 0x7f374c0c8e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f374affd640 1 -- 172.21.15.16:0/1718088925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f374c160a00 msgr2=0x7f374c15b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f374affd640 1 --2- 172.21.15.16:0/1718088925 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f374c160a00 0x7f374c15b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f374affd640 1 -- 172.21.15.16:0/1718088925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3744009a70 con 0x7f374c0b1ea0 2023-12-08T08:38:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f19ccb45640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19c0033eb0 0x7f19c00d4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f19ccb45640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19c0033eb0 0x7f19c00d4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58166/0 (socket says 172.21.15.16:58166) 2023-12-08T08:38:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f19ccb45640 1 -- 172.21.15.16:0/3248288328 learned_addr learned my addr 172.21.15.16:0/3248288328 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7fac4b7fe640 1 --2- 172.21.15.16:0/202933939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac4c15e850 0x7fac4c200490 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fac4c072430 tx=0x7fac44004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f19ceda9640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19c004e490 0x7f19c00daf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f2596ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25981508e0 0x7f259815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f2596ffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25981508e0 0x7f259815baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58170/0 (socket says 172.21.15.16:58170) 2023-12-08T08:38:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f2596ffd640 1 -- 172.21.15.16:0/3706329626 learned_addr learned my addr 172.21.15.16:0/3706329626 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f259d43d640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25981ef150 0x7f25981ef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f25967fc640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25980ccaf0 0x7f25980ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f19c7fff640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19c0053c30 0x7f19c01754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f45e1ffb640 1 -- 172.21.15.16:0/925702457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0adae0 con 0x7f45e40e7850 2023-12-08T08:38:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f19cd346640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19c004e490 0x7f19c00daf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f36150ab640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361006b5e0 0x7f361015d1e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f360800ba50 tx=0x7f360800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f19ceda9640 1 -- 172.21.15.16:0/3248288328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00d88a0 con 0x7f19c0053c30 2023-12-08T08:38:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f259d43d640 1 -- 172.21.15.16:0/3706329626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25980cd490 con 0x7f25981ef150 2023-12-08T08:38:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f19ceda9640 1 -- 172.21.15.16:0/3248288328 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00daaf0 con 0x7f19c004e490 2023-12-08T08:38:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f3605ffb640 1 -- 172.21.15.16:0/2867515219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800cc10 con 0x7f361006b5e0 2023-12-08T08:38:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f259d43d640 1 -- 172.21.15.16:0/3706329626 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2598150cc0 con 0x7f25980ccaf0 2023-12-08T08:38:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f19ceda9640 1 -- 172.21.15.16:0/3248288328 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c00d9c60 con 0x7f19c0033eb0 2023-12-08T08:38:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f259d43d640 1 -- 172.21.15.16:0/3706329626 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25981efec0 con 0x7f25981508e0 2023-12-08T08:38:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f45e1ffb640 1 -- 172.21.15.16:0/925702457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec0adc40 con 0x7f45e40e7850 2023-12-08T08:38:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f45e1ffb640 1 -- 172.21.15.16:0/925702457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0ba450 con 0x7f45e40e7850 2023-12-08T08:38:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f45f24bb640 1 -- 172.21.15.16:0/925702457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e404db70 con 0x7f45e40e7850 2023-12-08T08:38:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.291+0000 7f45f24bb640 1 -- 172.21.15.16:0/925702457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e404e130 con 0x7f45e40e7850 2023-12-08T08:38:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7f361730f640 1 -- 172.21.15.16:0/2867515219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36000094d0 con 0x7f361006b5e0 2023-12-08T08:38:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.291+0000 7f361730f640 1 -- 172.21.15.16:0/2867515219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36100c5590 con 0x7f361006b5e0 2023-12-08T08:38:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.291+0000 7f3605ffb640 1 -- 172.21.15.16:0/2867515219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360800cd70 con 0x7f361006b5e0 2023-12-08T08:38:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7fac337fe640 1 -- 172.21.15.16:0/202933939 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4403e610 con 0x7fac4c15e850 2023-12-08T08:38:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.290+0000 7fac337fe640 1 -- 172.21.15.16:0/202933939 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4403e770 con 0x7fac4c15e850 2023-12-08T08:38:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.291+0000 7fac337fe640 1 -- 172.21.15.16:0/202933939 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4403eb20 con 0x7fac4c15e850 2023-12-08T08:38:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f45f24bb640 1 -- 172.21.15.16:0/925702457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f45e40057b0 con 0x7f45e40e7850 2023-12-08T08:38:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.291+0000 7f374affd640 1 --2- 172.21.15.16:0/1718088925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f374c0b1ea0 0x7f374c148600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3744009d90 tx=0x7f3744004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.291+0000 7fb363a89640 1 -- 172.21.15.16:0/1125120851 shutdown_connections 2023-12-08T08:38:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.291+0000 7fd63359e640 1 Processor -- start 2023-12-08T08:38:44.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.291+0000 7fd63359e640 1 -- start start 2023-12-08T08:38:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.291+0000 7fb363a89640 1 -- 172.21.15.16:0/1125120851 wait complete. 2023-12-08T08:38:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f25977fe640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25981ef150 0x7f25981ef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f2596ffd640 1 -- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25980ccaf0 msgr2=0x7f25980ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f2596ffd640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25980ccaf0 0x7f25980ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f2596ffd640 1 -- 172.21.15.16:0/3706329626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25981ef150 msgr2=0x7f25981ef5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f2596ffd640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25981ef150 0x7f25981ef5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f2596ffd640 1 -- 172.21.15.16:0/3706329626 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259815ace0 con 0x7f25981508e0 2023-12-08T08:38:44.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f25977fe640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25981ef150 0x7f25981ef5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:44.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f3605ffb640 1 -- 172.21.15.16:0/2867515219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608014680 con 0x7f361006b5e0 2023-12-08T08:38:44.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f19ccb45640 1 -- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19c004e490 msgr2=0x7f19c00daf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f19ccb45640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19c004e490 0x7f19c00daf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f19ccb45640 1 -- 172.21.15.16:0/3248288328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19c0053c30 msgr2=0x7f19c01754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f19ccb45640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19c0053c30 0x7f19c01754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f19ccb45640 1 -- 172.21.15.16:0/3248288328 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc009d60 con 0x7f19c0033eb0 2023-12-08T08:38:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7fac51561640 1 -- 172.21.15.16:0/202933939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac4c15bef0 con 0x7fac4c15e850 2023-12-08T08:38:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f361730f640 1 -- 172.21.15.16:0/2867515219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35f0000fc0 con 0x7f361006b5e0 2023-12-08T08:38:44.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7fac51561640 1 -- 172.21.15.16:0/202933939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac4c15c4b0 con 0x7fac4c15e850 2023-12-08T08:38:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f2596ffd640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25981508e0 0x7f259815baa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f258c014de0 tx=0x7f258c008480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f19ccb45640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19c0033eb0 0x7f19c00d4280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19bc02d1b0 tx=0x7f19bc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f257f7fe640 1 -- 172.21.15.16:0/3706329626 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c019070 con 0x7f25981508e0 2023-12-08T08:38:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f257f7fe640 1 -- 172.21.15.16:0/3706329626 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c008da0 con 0x7f25981508e0 2023-12-08T08:38:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.292+0000 7f257f7fe640 1 -- 172.21.15.16:0/3706329626 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c012b20 con 0x7f25981508e0 2023-12-08T08:38:44.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7f259d43d640 1 -- 172.21.15.16:0/3706329626 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259815afc0 con 0x7f25981508e0 2023-12-08T08:38:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7f37337fe640 1 -- 172.21.15.16:0/1718088925 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374402cea0 con 0x7f374c0b1ea0 2023-12-08T08:38:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7f37337fe640 1 -- 172.21.15.16:0/1718088925 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374403e820 con 0x7f374c0b1ea0 2023-12-08T08:38:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7f37337fe640 1 -- 172.21.15.16:0/1718088925 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374403ebb0 con 0x7f374c0b1ea0 2023-12-08T08:38:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7f37517dc640 1 -- 172.21.15.16:0/1718088925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374c0c9390 con 0x7f374c0b1ea0 2023-12-08T08:38:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7f259d43d640 1 -- 172.21.15.16:0/3706329626 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259815b580 con 0x7f25981508e0 2023-12-08T08:38:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7fd63359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd634070390 0x7fd63415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7fd63359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634161250 0x7fd6341583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7fb363a89640 1 Processor -- start 2023-12-08T08:38:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7fb363a89640 1 -- start start 2023-12-08T08:38:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7fac317fa640 1 -- 172.21.15.16:0/202933939 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fac4c0ce090 con 0x7fac4c15e850 2023-12-08T08:38:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7fd63359e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd634155320 0x7fd634154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7f37517dc640 1 -- 172.21.15.16:0/1718088925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f374c1460c0 con 0x7f374c0b1ea0 2023-12-08T08:38:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7f19c57fa640 1 -- 172.21.15.16:0/3248288328 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc03e620 con 0x7f19c0033eb0 2023-12-08T08:38:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7f19c57fa640 1 -- 172.21.15.16:0/3248288328 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc03e7c0 con 0x7f19c0033eb0 2023-12-08T08:38:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.294+0000 7fd631d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634161250 0x7fd6341583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.294+0000 7fd631d9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634161250 0x7fd6341583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:46034/0 (socket says 172.21.15.16:46034) 2023-12-08T08:38:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.293+0000 7f19c57fa640 1 -- 172.21.15.16:0/3248288328 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc03eb70 con 0x7f19c0033eb0 2023-12-08T08:38:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.294+0000 7f19ceda9640 1 -- 172.21.15.16:0/3248288328 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c00d47c0 con 0x7f19c0033eb0 2023-12-08T08:38:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.294+0000 7f19ceda9640 1 -- 172.21.15.16:0/3248288328 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c00d9ee0 con 0x7f19c0033eb0 2023-12-08T08:38:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.294+0000 7f257d7fa640 1 -- 172.21.15.16:0/3706329626 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25980b3f70 con 0x7f25981508e0 2023-12-08T08:38:44.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.294+0000 7f45e1ffb640 1 -- 172.21.15.16:0/925702457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45ec0ae440 con 0x7f45e40e7850 2023-12-08T08:38:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7f3605ffb640 1 -- 172.21.15.16:0/2867515219 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f360801d3b0 con 0x7f361006b5e0 2023-12-08T08:38:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.294+0000 7fd632d9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd634155320 0x7fd634154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.294+0000 7fd632d9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd634155320 0x7fd634154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58178/0 (socket says 172.21.15.16:58178) 2023-12-08T08:38:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.294+0000 7fd63259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd634070390 0x7fd63415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.294+0000 7fd63259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd634070390 0x7fd63415a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52492/0 (socket says 172.21.15.16:52492) 2023-12-08T08:38:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.294+0000 7fd632d9d640 1 -- 172.21.15.16:0/2607502252 learned_addr learned my addr 172.21.15.16:0/2607502252 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.294+0000 7fd63359e640 1 -- 172.21.15.16:0/2607502252 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63415ba30 con 0x7fd634070390 2023-12-08T08:38:44.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.294+0000 7fd63359e640 1 -- 172.21.15.16:0/2607502252 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63415a000 con 0x7fd634161250 2023-12-08T08:38:44.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.294+0000 7fd63359e640 1 -- 172.21.15.16:0/2607502252 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd634158b90 con 0x7fd634155320 2023-12-08T08:38:44.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7fd632d9d640 1 -- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634161250 msgr2=0x7fd6341583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7fd632d9d640 1 --2- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634161250 0x7fd6341583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7fd632d9d640 1 -- 172.21.15.16:0/2607502252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd634070390 msgr2=0x7fd63415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7fd632d9d640 1 --2- 172.21.15.16:0/2607502252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd634070390 0x7fd63415a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7fd632d9d640 1 -- 172.21.15.16:0/2607502252 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c009a70 con 0x7fd634155320 2023-12-08T08:38:44.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7f37517dc640 1 -- 172.21.15.16:0/1718088925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f374c160de0 con 0x7f374c0b1ea0 2023-12-08T08:38:44.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7fac337fe640 1 -- 172.21.15.16:0/202933939 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fac4403b050 con 0x7fac4c15e850 2023-12-08T08:38:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7f3605ffb640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f35f406af10 0x7f35f406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7fac337fe640 1 --2- 172.21.15.16:0/202933939 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fac3806ae90 0x7fac3806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7fb363a89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c06c5a0 0x7fb35c1f7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7f3605ffb640 1 -- 172.21.15.16:0/2867515219 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f360808a0c0 con 0x7f361006b5e0 2023-12-08T08:38:44.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7fd631d9b640 1 --2- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634161250 0x7fd6341583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:44.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7fd632d9d640 1 --2- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd634155320 0x7fd634154550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd62c02c310 tx=0x7fd62c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fb363a89640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb35c146460 0x7fb35c143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7f36148aa640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f35f406af10 0x7f35f406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fb363a89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb35c146f50 0x7fb35c15abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7f3605ffb640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.16:6808/3215779870,v1:172.21.15.16:6809/3215779870] conn(0x7f35f40743e0 0x7f35f4076820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fb363a89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c1f7ea0 con 0x7fb35c06c5a0 2023-12-08T08:38:44.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fb361024640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb35c146460 0x7fb35c143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fb361024640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb35c146460 0x7fb35c143af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58194/0 (socket says 172.21.15.16:58194) 2023-12-08T08:38:44.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fb361024640 1 -- 172.21.15.16:0/3472468881 learned_addr learned my addr 172.21.15.16:0/3472468881 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fb362026640 1 --2- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb35c146f50 0x7fb35c15abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7f45e1ffb640 1 --2- 172.21.15.16:0/925702457 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f45d006af10 0x7f45d006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7f45e1ffb640 1 -- 172.21.15.16:0/925702457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f45ec126200 con 0x7f45e40e7850 2023-12-08T08:38:44.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7f45e1ffb640 1 --2- 172.21.15.16:0/925702457 >> [v2:172.21.15.16:6817/1015436151,v1:172.21.15.16:6819/1015436151] conn(0x7f45d0074550 0x7f45d0076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7f45e1ffb640 1 -- 172.21.15.16:0/925702457 --> [v2:172.21.15.16:6817/1015436151,v1:172.21.15.16:6819/1015436151] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f45d0077060 con 0x7f45d0074550 2023-12-08T08:38:44.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.295+0000 7f45e1ffb640 1 -- 172.21.15.16:0/925702457 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45ec128820 con 0x7f45e40e7850 2023-12-08T08:38:44.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7f45f0cb8640 1 --2- 172.21.15.16:0/925702457 >> [v2:172.21.15.16:6817/1015436151,v1:172.21.15.16:6819/1015436151] conn(0x7f45d0074550 0x7f45d0076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7f36158ac640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.16:6808/3215779870,v1:172.21.15.16:6809/3215779870] conn(0x7f35f40743e0 0x7f35f4076820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fb361825640 1 --2- 172.21.15.16:0/3472468881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c06c5a0 0x7fb35c1f7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fb363a89640 1 -- 172.21.15.16:0/3472468881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c159b40 con 0x7fb35c146f50 2023-12-08T08:38:44.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fb363a89640 1 -- 172.21.15.16:0/3472468881 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c159e20 con 0x7fb35c146460 2023-12-08T08:38:44.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fac337fe640 1 -- 172.21.15.16:0/202933939 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fac440abdd0 con 0x7fac4c15e850 2023-12-08T08:38:44.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fac337fe640 1 --2- 172.21.15.16:0/202933939 >> [v2:172.21.15.16:6816/3573990929,v1:172.21.15.16:6818/3573990929] conn(0x7fac38074360 0x7fac380767a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fac337fe640 1 -- 172.21.15.16:0/202933939 --> [v2:172.21.15.16:6816/3573990929,v1:172.21.15.16:6818/3573990929] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac38076e70 con 0x7fac38074360 2023-12-08T08:38:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fac337fe640 1 -- 172.21.15.16:0/202933939 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac440ae4e0 con 0x7fac4c15e850 2023-12-08T08:38:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fac4a7fc640 1 --2- 172.21.15.16:0/202933939 >> [v2:172.21.15.16:6816/3573990929,v1:172.21.15.16:6818/3573990929] conn(0x7fac38074360 0x7fac380767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7fac4affd640 1 --2- 172.21.15.16:0/202933939 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fac3806ae90 0x7fac3806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7f45f0cb8640 1 --2- 172.21.15.16:0/925702457 >> [v2:172.21.15.16:6817/1015436151,v1:172.21.15.16:6819/1015436151] conn(0x7f45d0074550 0x7f45d0076990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7f45f14b9640 1 --2- 172.21.15.16:0/925702457 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f45d006af10 0x7f45d006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7f3605ffb640 1 -- 172.21.15.16:0/2867515219 --> [v2:172.21.15.16:6808/3215779870,v1:172.21.15.16:6809/3215779870] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35f4076ef0 con 0x7f35f40743e0 2023-12-08T08:38:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7f3605ffb640 1 -- 172.21.15.16:0/2867515219 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f360808a470 con 0x7f361006b5e0 2023-12-08T08:38:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7f36158ac640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.16:6808/3215779870,v1:172.21.15.16:6809/3215779870] conn(0x7f35f40743e0 0x7f35f4076820 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7f36148aa640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f35f406af10 0x7f35f406d3d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f361009eea0 tx=0x7f360c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7fd622ffd640 1 -- 172.21.15.16:0/2607502252 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c03e610 con 0x7fd634155320 2023-12-08T08:38:44.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7fd622ffd640 1 -- 172.21.15.16:0/2607502252 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c03e790 con 0x7fd634155320 2023-12-08T08:38:44.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7fd622ffd640 1 -- 172.21.15.16:0/2607502252 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c03eb40 con 0x7fd634155320 2023-12-08T08:38:44.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7fd63359e640 1 -- 172.21.15.16:0/2607502252 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd634159710 con 0x7fd634155320 2023-12-08T08:38:44.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7f19c57fa640 1 -- 172.21.15.16:0/3248288328 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f19bc03c070 con 0x7f19c0033eb0 2023-12-08T08:38:44.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7f19ceda9640 1 -- 172.21.15.16:0/3248288328 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f19c00df5c0 con 0x7f19c0033eb0 2023-12-08T08:38:44.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.296+0000 7f257f7fe640 1 -- 172.21.15.16:0/3706329626 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f258c012c80 con 0x7f25981508e0 2023-12-08T08:38:44.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7f257f7fe640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f258406af40 0x7f258406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7f257f7fe640 1 -- 172.21.15.16:0/3706329626 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f258c0916d0 con 0x7f25981508e0 2023-12-08T08:38:44.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7f257f7fe640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:6824/898596008,v1:172.21.15.112:6825/898596008] conn(0x7f2584074500 0x7f2584076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7f257f7fe640 1 -- 172.21.15.16:0/3706329626 --> [v2:172.21.15.112:6824/898596008,v1:172.21.15.112:6825/898596008] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2584077010 con 0x7f2584074500 2023-12-08T08:38:44.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7f257f7fe640 1 -- 172.21.15.16:0/3706329626 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f258c05c500 con 0x7f25981508e0 2023-12-08T08:38:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7fd63359e640 1 -- 172.21.15.16:0/2607502252 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6341599b0 con 0x7fd634155320 2023-12-08T08:38:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7f37337fe640 1 -- 172.21.15.16:0/1718088925 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f374403b050 con 0x7f374c0b1ea0 2023-12-08T08:38:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f25977fe640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:6824/898596008,v1:172.21.15.112:6825/898596008] conn(0x7f2584074500 0x7f2584076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f25967fc640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f258406af40 0x7f258406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f45e1ffb640 1 -- 172.21.15.16:0/925702457 <== osd.1 v2:172.21.15.16:6817/1015436151 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f45d0077060 con 0x7f45d0074550 2023-12-08T08:38:44.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f19c57fa640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f19ac06af40 0x7f19ac06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f19c57fa640 1 -- 172.21.15.16:0/3248288328 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f19bc0ab900 con 0x7f19c0033eb0 2023-12-08T08:38:44.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7fb362026640 1 -- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb35c146460 msgr2=0x7fb35c143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7fb362026640 1 --2- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb35c146460 0x7fb35c143af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7fb362026640 1 -- 172.21.15.16:0/3472468881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c06c5a0 msgr2=0x7fb35c1f7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7fb362026640 1 --2- 172.21.15.16:0/3472468881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb35c06c5a0 0x7fb35c1f7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.297+0000 7fb362026640 1 -- 172.21.15.16:0/3472468881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3580094d0 con 0x7fb35c146f50 2023-12-08T08:38:44.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f19c57fa640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:6816/1387317691,v1:172.21.15.112:6817/1387317691] conn(0x7f19ac074570 0x7f19ac0769b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f19c57fa640 1 -- 172.21.15.16:0/3248288328 --> [v2:172.21.15.112:6816/1387317691,v1:172.21.15.112:6817/1387317691] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f19ac077080 con 0x7f19ac074570 2023-12-08T08:38:44.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f19c57fa640 1 -- 172.21.15.16:0/3248288328 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19bc0b0240 con 0x7f19c0033eb0 2023-12-08T08:38:44.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f3605ffb640 1 -- 172.21.15.16:0/2867515219 <== osd.3 v2:172.21.15.16:6808/3215779870 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f35f4076ef0 con 0x7f35f40743e0 2023-12-08T08:38:44.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7fac4a7fc640 1 --2- 172.21.15.16:0/202933939 >> [v2:172.21.15.16:6816/3573990929,v1:172.21.15.16:6818/3573990929] conn(0x7fac38074360 0x7fac380767a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f19c7fff640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f19ac06af40 0x7f19ac06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7fb362026640 1 --2- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb35c146f50 0x7fb35c15abe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb358033000 tx=0x7fb3580046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f37337fe640 1 --2- 172.21.15.16:0/1718088925 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f373806af10 0x7f373806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f37337fe640 1 -- 172.21.15.16:0/1718088925 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f37440ac830 con 0x7f374c0b1ea0 2023-12-08T08:38:44.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f37337fe640 1 --2- 172.21.15.16:0/1718088925 >> [v2:172.21.15.16:6800/3962287192,v1:172.21.15.16:6801/3962287192] conn(0x7f3738074550 0x7f3738076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f37337fe640 1 -- 172.21.15.16:0/1718088925 --> [v2:172.21.15.16:6800/3962287192,v1:172.21.15.16:6801/3962287192] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3738077060 con 0x7f3738074550 2023-12-08T08:38:44.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f37337fe640 1 -- 172.21.15.16:0/1718088925 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37440acbe0 con 0x7f374c0b1ea0 2023-12-08T08:38:44.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.299+0000 7fb3527fc640 1 -- 172.21.15.16:0/3472468881 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358035c60 con 0x7fb35c146f50 2023-12-08T08:38:44.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.299+0000 7fb3527fc640 1 -- 172.21.15.16:0/3472468881 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb358035dc0 con 0x7fb35c146f50 2023-12-08T08:38:44.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.299+0000 7fb3527fc640 1 -- 172.21.15.16:0/3472468881 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358040680 con 0x7fb35c146f50 2023-12-08T08:38:44.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f45f14b9640 1 --2- 172.21.15.16:0/925702457 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f45d006af10 0x7f45d006d3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45e8007fd0 tx=0x7f45e8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.299+0000 7f374a7fc640 1 --2- 172.21.15.16:0/1718088925 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f373806af10 0x7f373806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.299+0000 7fd63359e640 1 -- 172.21.15.16:0/2607502252 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd6340b2b80 con 0x7fd634155320 2023-12-08T08:38:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7f19cd346640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:6816/1387317691,v1:172.21.15.112:6817/1387317691] conn(0x7f19ac074570 0x7f19ac0769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.299+0000 7f19cd346640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:6816/1387317691,v1:172.21.15.112:6817/1387317691] conn(0x7f19ac074570 0x7f19ac0769b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.299+0000 7fb363a89640 1 -- 172.21.15.16:0/3472468881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35c15a100 con 0x7fb35c146f50 2023-12-08T08:38:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.299+0000 7fb363a89640 1 -- 172.21.15.16:0/3472468881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35c15a6c0 con 0x7fb35c146f50 2023-12-08T08:38:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.299+0000 7f374b7fe640 1 --2- 172.21.15.16:0/1718088925 >> [v2:172.21.15.16:6800/3962287192,v1:172.21.15.16:6801/3962287192] conn(0x7f3738074550 0x7f3738076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.300+0000 7f25977fe640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:6824/898596008,v1:172.21.15.112:6825/898596008] conn(0x7f2584074500 0x7f2584076940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.300+0000 7f19c7fff640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f19ac06af40 0x7f19ac06d400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f19c004b090 tx=0x7f19b4009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.300+0000 7f25967fc640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f258406af40 0x7f258406d400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f25980aed30 tx=0x7f2588009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.298+0000 7fac4affd640 1 --2- 172.21.15.16:0/202933939 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fac3806ae90 0x7fac3806d350 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fac3c000f60 tx=0x7fac3c007e10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.300+0000 7fac337fe640 1 -- 172.21.15.16:0/202933939 <== osd.0 v2:172.21.15.16:6816/3573990929 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fac38076e70 con 0x7fac38074360 2023-12-08T08:38:44.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.300+0000 7fb363a89640 1 -- 172.21.15.16:0/3472468881 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb35c0b1af0 con 0x7fb35c146f50 2023-12-08T08:38:44.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.300+0000 7f19c57fa640 1 -- 172.21.15.16:0/3248288328 <== osd.5 v2:172.21.15.112:6816/1387317691 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f19ac077080 con 0x7f19ac074570 2023-12-08T08:38:44.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.300+0000 7f257f7fe640 1 -- 172.21.15.16:0/3706329626 <== osd.4 v2:172.21.15.112:6824/898596008 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f2584077010 con 0x7f2584074500 2023-12-08T08:38:44.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.300+0000 7f374b7fe640 1 --2- 172.21.15.16:0/1718088925 >> [v2:172.21.15.16:6800/3962287192,v1:172.21.15.16:6801/3962287192] conn(0x7f3738074550 0x7f3738076990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.301+0000 7f374a7fc640 1 --2- 172.21.15.16:0/1718088925 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f373806af10 0x7f373806d3d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f374c0cd180 tx=0x7f373c0079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.301+0000 7fd622ffd640 1 -- 172.21.15.16:0/2607502252 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd62c03b050 con 0x7fd634155320 2023-12-08T08:38:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.301+0000 7fb3527fc640 1 -- 172.21.15.16:0/3472468881 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb3580407e0 con 0x7fb35c146f50 2023-12-08T08:38:44.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.301+0000 7fd622ffd640 1 --2- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd61406af10 0x7fd61406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.302+0000 7fd622ffd640 1 -- 172.21.15.16:0/2607502252 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd62c0ac210 con 0x7fd634155320 2023-12-08T08:38:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.302+0000 7fb3527fc640 1 --2- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb34006ae90 0x7fb34006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.302+0000 7fb3527fc640 1 -- 172.21.15.16:0/3472468881 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb358031070 con 0x7fb35c146f50 2023-12-08T08:38:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.302+0000 7fd622ffd640 1 --2- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:6808/1116392641,v1:172.21.15.112:6809/1116392641] conn(0x7fd614074550 0x7fd614076990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.302+0000 7fb361825640 1 --2- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb34006ae90 0x7fb34006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.302+0000 7fd622ffd640 1 -- 172.21.15.16:0/2607502252 --> [v2:172.21.15.112:6808/1116392641,v1:172.21.15.112:6809/1116392641] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd614077060 con 0x7fd614074550 2023-12-08T08:38:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.302+0000 7fb3527fc640 1 --2- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:6800/1165220586,v1:172.21.15.112:6801/1165220586] conn(0x7fb340074450 0x7fb340076890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.302+0000 7fb3527fc640 1 -- 172.21.15.16:0/3472468881 --> [v2:172.21.15.112:6800/1165220586,v1:172.21.15.112:6801/1165220586] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb340076f60 con 0x7fb340074450 2023-12-08T08:38:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.302+0000 7f37337fe640 1 -- 172.21.15.16:0/1718088925 <== osd.2 v2:172.21.15.16:6800/3962287192 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f3738077060 con 0x7f3738074550 2023-12-08T08:38:44.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.302+0000 7fb3527fc640 1 -- 172.21.15.16:0/3472468881 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb358076e40 con 0x7fb35c146f50 2023-12-08T08:38:44.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.302+0000 7fd622ffd640 1 -- 172.21.15.16:0/2607502252 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd62c0ae830 con 0x7fd634155320 2023-12-08T08:38:44.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.302+0000 7fd63259c640 1 --2- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd61406af10 0x7fd61406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.302+0000 7fd631d9b640 1 --2- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:6808/1116392641,v1:172.21.15.112:6809/1116392641] conn(0x7fd614074550 0x7fd614076990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.303+0000 7fb361825640 1 --2- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb34006ae90 0x7fb34006d350 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb35c15cc90 tx=0x7fb34c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.303+0000 7fb361024640 1 --2- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:6800/1165220586,v1:172.21.15.112:6801/1165220586] conn(0x7fb340074450 0x7fb340076890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.303+0000 7fd631d9b640 1 --2- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:6808/1116392641,v1:172.21.15.112:6809/1116392641] conn(0x7fd614074550 0x7fd614076990 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.303+0000 7fd63259c640 1 --2- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd61406af10 0x7fd61406d3d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd6341fa7a0 tx=0x7fd624008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.303+0000 7fb361024640 1 --2- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:6800/1165220586,v1:172.21.15.112:6801/1165220586] conn(0x7fb340074450 0x7fb340076890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.304+0000 7fd622ffd640 1 -- 172.21.15.16:0/2607502252 <== osd.7 v2:172.21.15.112:6808/1116392641 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd614077060 con 0x7fd614074550 2023-12-08T08:38:44.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.305+0000 7fb3527fc640 1 -- 172.21.15.16:0/3472468881 <== osd.6 v2:172.21.15.112:6800/1165220586 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fb340076f60 con 0x7fb340074450 2023-12-08T08:38:44.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.330+0000 7f45f24bb640 1 -- 172.21.15.16:0/925702457 --> [v2:172.21.15.16:6817/1015436151,v1:172.21.15.16:6819/1015436151] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f45e40501f0 con 0x7f45d0074550 2023-12-08T08:38:44.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.331+0000 7f45e1ffb640 1 -- 172.21.15.16:0/925702457 <== osd.1 v2:172.21.15.16:6817/1015436151 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f45e40501f0 con 0x7f45d0074550 2023-12-08T08:38:44.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.331+0000 7f45d77fe640 1 -- 172.21.15.16:0/925702457 >> [v2:172.21.15.16:6817/1015436151,v1:172.21.15.16:6819/1015436151] conn(0x7f45d0074550 msgr2=0x7f45d0076990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.332+0000 7f45d77fe640 1 --2- 172.21.15.16:0/925702457 >> [v2:172.21.15.16:6817/1015436151,v1:172.21.15.16:6819/1015436151] conn(0x7f45d0074550 0x7f45d0076990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.332+0000 7f361730f640 1 -- 172.21.15.16:0/2867515219 --> [v2:172.21.15.16:6808/3215779870,v1:172.21.15.16:6809/3215779870] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f35f0002ca0 con 0x7f35f40743e0 2023-12-08T08:38:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.332+0000 7fac317fa640 1 -- 172.21.15.16:0/202933939 --> [v2:172.21.15.16:6816/3573990929,v1:172.21.15.16:6818/3573990929] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fac4c0cee10 con 0x7fac38074360 2023-12-08T08:38:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.332+0000 7f45d77fe640 1 -- 172.21.15.16:0/925702457 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f45d006af10 msgr2=0x7f45d006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.332+0000 7f45d77fe640 1 --2- 172.21.15.16:0/925702457 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f45d006af10 0x7f45d006d3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f45e8007fd0 tx=0x7f45e8009040 comp rx=0 tx=0).stop 2023-12-08T08:38:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.332+0000 7f45d77fe640 1 -- 172.21.15.16:0/925702457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e40e7850 msgr2=0x7f45e4044760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.332+0000 7f45d77fe640 1 --2- 172.21.15.16:0/925702457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45e40e7850 0x7f45e4044760 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f45ec0697e0 tx=0x7f45ec069bf0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.332+0000 7f45f1cba640 1 -- 172.21.15.16:0/925702457 reap_dead start 2023-12-08T08:38:44.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.332+0000 7f45d77fe640 1 -- 172.21.15.16:0/925702457 shutdown_connections 2023-12-08T08:38:44.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.332+0000 7f45d77fe640 1 -- 172.21.15.16:0/925702457 >> 172.21.15.16:0/925702457 conn(0x7f45e4055ba0 msgr2=0x7f45e4049800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.332+0000 7f45d77fe640 1 -- 172.21.15.16:0/925702457 shutdown_connections 2023-12-08T08:38:44.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.332+0000 7f45d77fe640 1 -- 172.21.15.16:0/925702457 wait complete. 2023-12-08T08:38:44.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.333+0000 7f3605ffb640 1 -- 172.21.15.16:0/2867515219 <== osd.3 v2:172.21.15.16:6808/3215779870 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f35f0002ca0 con 0x7f35f40743e0 2023-12-08T08:38:44.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.333+0000 7f257d7fa640 1 -- 172.21.15.16:0/3706329626 --> [v2:172.21.15.112:6824/898596008,v1:172.21.15.112:6825/898596008] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f259815da70 con 0x7f2584074500 2023-12-08T08:38:44.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.333+0000 7fac337fe640 1 -- 172.21.15.16:0/202933939 <== osd.0 v2:172.21.15.16:6816/3573990929 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fac40007170 con 0x7fac38074360 2023-12-08T08:38:44.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.333+0000 7f361730f640 1 -- 172.21.15.16:0/2867515219 >> [v2:172.21.15.16:6808/3215779870,v1:172.21.15.16:6809/3215779870] conn(0x7f35f40743e0 msgr2=0x7f35f4076820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.333+0000 7f361730f640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.16:6808/3215779870,v1:172.21.15.16:6809/3215779870] conn(0x7f35f40743e0 0x7f35f4076820 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.333+0000 7f257f7fe640 1 -- 172.21.15.16:0/3706329626 <== osd.4 v2:172.21.15.112:6824/898596008 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f25900017b0 con 0x7f2584074500 2023-12-08T08:38:44.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.333+0000 7f361730f640 1 -- 172.21.15.16:0/2867515219 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f35f406af10 msgr2=0x7f35f406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.333+0000 7f361730f640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f35f406af10 0x7f35f406d3d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f361009eea0 tx=0x7f360c008040 comp rx=0 tx=0).stop 2023-12-08T08:38:44.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.333+0000 7f361730f640 1 -- 172.21.15.16:0/2867515219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361006b5e0 msgr2=0x7f361015d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7f361730f640 1 --2- 172.21.15.16:0/2867515219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361006b5e0 0x7f361015d1e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f360800ba50 tx=0x7f360800bf10 comp rx=0 tx=0).stop 2023-12-08T08:38:44.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7f36158ac640 1 -- 172.21.15.16:0/2867515219 reap_dead start 2023-12-08T08:38:44.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7f361730f640 1 -- 172.21.15.16:0/2867515219 shutdown_connections 2023-12-08T08:38:44.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7f361730f640 1 -- 172.21.15.16:0/2867515219 >> 172.21.15.16:0/2867515219 conn(0x7f3610099a10 msgr2=0x7f36100b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7f361730f640 1 -- 172.21.15.16:0/2867515219 shutdown_connections 2023-12-08T08:38:44.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7f361730f640 1 -- 172.21.15.16:0/2867515219 wait complete. 2023-12-08T08:38:44.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7fac317fa640 1 -- 172.21.15.16:0/202933939 >> [v2:172.21.15.16:6816/3573990929,v1:172.21.15.16:6818/3573990929] conn(0x7fac38074360 msgr2=0x7fac380767a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7fac317fa640 1 --2- 172.21.15.16:0/202933939 >> [v2:172.21.15.16:6816/3573990929,v1:172.21.15.16:6818/3573990929] conn(0x7fac38074360 0x7fac380767a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7f259d43d640 1 -- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:6824/898596008,v1:172.21.15.112:6825/898596008] conn(0x7f2584074500 msgr2=0x7f2584076940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7f259d43d640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:6824/898596008,v1:172.21.15.112:6825/898596008] conn(0x7f2584074500 0x7f2584076940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7f19ceda9640 1 -- 172.21.15.16:0/3248288328 --> [v2:172.21.15.112:6816/1387317691,v1:172.21.15.112:6817/1387317691] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f19c00dac30 con 0x7f19ac074570 2023-12-08T08:38:44.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7fac317fa640 1 -- 172.21.15.16:0/202933939 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fac3806ae90 msgr2=0x7fac3806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7fac317fa640 1 --2- 172.21.15.16:0/202933939 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fac3806ae90 0x7fac3806d350 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fac3c000f60 tx=0x7fac3c007e10 comp rx=0 tx=0).stop 2023-12-08T08:38:44.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7fac317fa640 1 -- 172.21.15.16:0/202933939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac4c15e850 msgr2=0x7fac4c200490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.335+0000 7fac317fa640 1 --2- 172.21.15.16:0/202933939 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fac4c15e850 0x7fac4c200490 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fac4c072430 tx=0x7fac44004a40 comp rx=0 tx=0).stop 2023-12-08T08:38:44.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7f259d43d640 1 -- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f258406af40 msgr2=0x7f258406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7f259d43d640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f258406af40 0x7f258406d400 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f25980aed30 tx=0x7f2588009040 comp rx=0 tx=0).stop 2023-12-08T08:38:44.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7f259d43d640 1 -- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25981508e0 msgr2=0x7f259815baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7f259d43d640 1 --2- 172.21.15.16:0/3706329626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25981508e0 0x7f259815baa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f258c014de0 tx=0x7f258c008480 comp rx=0 tx=0).stop 2023-12-08T08:38:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.334+0000 7f25977fe640 1 -- 172.21.15.16:0/3706329626 reap_dead start 2023-12-08T08:38:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.335+0000 7fac4b7fe640 1 -- 172.21.15.16:0/202933939 reap_dead start 2023-12-08T08:38:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.335+0000 7fac317fa640 1 -- 172.21.15.16:0/202933939 shutdown_connections 2023-12-08T08:38:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.335+0000 7f259d43d640 1 -- 172.21.15.16:0/3706329626 shutdown_connections 2023-12-08T08:38:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.335+0000 7fac317fa640 1 -- 172.21.15.16:0/202933939 >> 172.21.15.16:0/202933939 conn(0x7fac4c0b1860 msgr2=0x7fac4c1590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.335+0000 7f259d43d640 1 -- 172.21.15.16:0/3706329626 >> 172.21.15.16:0/3706329626 conn(0x7f25980cf470 msgr2=0x7f2598153ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.335+0000 7fac317fa640 1 -- 172.21.15.16:0/202933939 shutdown_connections 2023-12-08T08:38:44.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.335+0000 7f259d43d640 1 -- 172.21.15.16:0/3706329626 shutdown_connections 2023-12-08T08:38:44.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.335+0000 7fac317fa640 1 -- 172.21.15.16:0/202933939 wait complete. 2023-12-08T08:38:44.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.335+0000 7f259d43d640 1 -- 172.21.15.16:0/3706329626 wait complete. 2023-12-08T08:38:44.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.335+0000 7f19c57fa640 1 -- 172.21.15.16:0/3248288328 <== osd.5 v2:172.21.15.112:6816/1387317691 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f19c804f730 con 0x7f19ac074570 2023-12-08T08:38:44.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.336+0000 7fd63359e640 1 -- 172.21.15.16:0/2607502252 --> [v2:172.21.15.112:6808/1116392641,v1:172.21.15.112:6809/1116392641] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd6340b28a0 con 0x7fd614074550 2023-12-08T08:38:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.336+0000 7fb363a89640 1 -- 172.21.15.16:0/3472468881 --> [v2:172.21.15.112:6800/1165220586,v1:172.21.15.112:6801/1165220586] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb35c1476b0 con 0x7fb340074450 2023-12-08T08:38:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.336+0000 7f37517dc640 1 -- 172.21.15.16:0/1718088925 --> [v2:172.21.15.16:6800/3962287192,v1:172.21.15.16:6801/3962287192] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f374c0b32b0 con 0x7f3738074550 2023-12-08T08:38:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7f19ceda9640 1 -- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:6816/1387317691,v1:172.21.15.112:6817/1387317691] conn(0x7f19ac074570 msgr2=0x7f19ac0769b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7f19ceda9640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:6816/1387317691,v1:172.21.15.112:6817/1387317691] conn(0x7f19ac074570 0x7f19ac0769b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7fd622ffd640 1 -- 172.21.15.16:0/2607502252 <== osd.7 v2:172.21.15.112:6808/1116392641 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd628001410 con 0x7fd614074550 2023-12-08T08:38:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7fb3527fc640 1 -- 172.21.15.16:0/3472468881 <== osd.6 v2:172.21.15.112:6800/1165220586 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb3540053a0 con 0x7fb340074450 2023-12-08T08:38:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7f37337fe640 1 -- 172.21.15.16:0/1718088925 <== osd.2 v2:172.21.15.16:6800/3962287192 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3738077060 con 0x7f3738074550 2023-12-08T08:38:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7f19ceda9640 1 -- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f19ac06af40 msgr2=0x7f19ac06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7f19ceda9640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f19ac06af40 0x7f19ac06d400 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f19c004b090 tx=0x7f19b4009190 comp rx=0 tx=0).stop 2023-12-08T08:38:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7f19ceda9640 1 -- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19c0033eb0 msgr2=0x7f19c00d4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7f19ceda9640 1 --2- 172.21.15.16:0/3248288328 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19c0033eb0 0x7f19c00d4280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19bc02d1b0 tx=0x7f19bc0046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7f19cd346640 1 -- 172.21.15.16:0/3248288328 reap_dead start 2023-12-08T08:38:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7fb363a89640 1 -- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:6800/1165220586,v1:172.21.15.112:6801/1165220586] conn(0x7fb340074450 msgr2=0x7fb340076890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7fb363a89640 1 --2- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:6800/1165220586,v1:172.21.15.112:6801/1165220586] conn(0x7fb340074450 0x7fb340076890 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7f19ceda9640 1 -- 172.21.15.16:0/3248288328 shutdown_connections 2023-12-08T08:38:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7f19ceda9640 1 -- 172.21.15.16:0/3248288328 >> 172.21.15.16:0/3248288328 conn(0x7f19c0009f70 msgr2=0x7f19c00e0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7fb363a89640 1 -- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb34006ae90 msgr2=0x7fb34006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.337+0000 7fb363a89640 1 --2- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb34006ae90 0x7fb34006d350 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb35c15cc90 tx=0x7fb34c009190 comp rx=0 tx=0).stop 2023-12-08T08:38:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fb363a89640 1 -- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb35c146f50 msgr2=0x7fb35c15abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fb363a89640 1 --2- 172.21.15.16:0/3472468881 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb35c146f50 0x7fb35c15abe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb358033000 tx=0x7fb3580046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fb362026640 1 -- 172.21.15.16:0/3472468881 reap_dead start 2023-12-08T08:38:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fd63359e640 1 -- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:6808/1116392641,v1:172.21.15.112:6809/1116392641] conn(0x7fd614074550 msgr2=0x7fd614076990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fd63359e640 1 --2- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:6808/1116392641,v1:172.21.15.112:6809/1116392641] conn(0x7fd614074550 0x7fd614076990 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7f37517dc640 1 -- 172.21.15.16:0/1718088925 >> [v2:172.21.15.16:6800/3962287192,v1:172.21.15.16:6801/3962287192] conn(0x7f3738074550 msgr2=0x7f3738076990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7f37517dc640 1 --2- 172.21.15.16:0/1718088925 >> [v2:172.21.15.16:6800/3962287192,v1:172.21.15.16:6801/3962287192] conn(0x7f3738074550 0x7f3738076990 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fb363a89640 1 -- 172.21.15.16:0/3472468881 shutdown_connections 2023-12-08T08:38:44.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fb363a89640 1 -- 172.21.15.16:0/3472468881 >> 172.21.15.16:0/3472468881 conn(0x7fb35c0b0750 msgr2=0x7fb35c1611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7f37517dc640 1 -- 172.21.15.16:0/1718088925 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f373806af10 msgr2=0x7f373806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7f37517dc640 1 --2- 172.21.15.16:0/1718088925 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f373806af10 0x7f373806d3d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f374c0cd180 tx=0x7f373c0079b0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7f37517dc640 1 -- 172.21.15.16:0/1718088925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f374c0b1ea0 msgr2=0x7f374c148600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7f37517dc640 1 --2- 172.21.15.16:0/1718088925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f374c0b1ea0 0x7f374c148600 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3744009d90 tx=0x7f3744004a40 comp rx=0 tx=0).stop 2023-12-08T08:38:44.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fd63359e640 1 -- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd61406af10 msgr2=0x7fd61406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fd63359e640 1 --2- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd61406af10 0x7fd61406d3d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd6341fa7a0 tx=0x7fd624008040 comp rx=0 tx=0).stop 2023-12-08T08:38:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7f374b7fe640 1 -- 172.21.15.16:0/1718088925 reap_dead start 2023-12-08T08:38:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fd63359e640 1 -- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd634155320 msgr2=0x7fd634154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fd63359e640 1 --2- 172.21.15.16:0/2607502252 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd634155320 0x7fd634154550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd62c02c310 tx=0x7fd62c004a40 comp rx=0 tx=0).stop 2023-12-08T08:38:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fd632d9d640 1 -- 172.21.15.16:0/2607502252 reap_dead start 2023-12-08T08:38:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7f19ceda9640 1 -- 172.21.15.16:0/3248288328 shutdown_connections 2023-12-08T08:38:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fd63359e640 1 -- 172.21.15.16:0/2607502252 shutdown_connections 2023-12-08T08:38:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fd63359e640 1 -- 172.21.15.16:0/2607502252 >> 172.21.15.16:0/2607502252 conn(0x7fd63406c540 msgr2=0x7fd6340b0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7f19ceda9640 1 -- 172.21.15.16:0/3248288328 wait complete. 2023-12-08T08:38:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fb363a89640 1 -- 172.21.15.16:0/3472468881 shutdown_connections 2023-12-08T08:38:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fd63359e640 1 -- 172.21.15.16:0/2607502252 shutdown_connections 2023-12-08T08:38:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.338+0000 7fb363a89640 1 -- 172.21.15.16:0/3472468881 wait complete. 2023-12-08T08:38:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.339+0000 7fd63359e640 1 -- 172.21.15.16:0/2607502252 wait complete. 2023-12-08T08:38:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.339+0000 7f37517dc640 1 -- 172.21.15.16:0/1718088925 shutdown_connections 2023-12-08T08:38:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.339+0000 7f37517dc640 1 -- 172.21.15.16:0/1718088925 >> 172.21.15.16:0/1718088925 conn(0x7f374c09c300 msgr2=0x7f374c15fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.339+0000 7f37517dc640 1 -- 172.21.15.16:0/1718088925 shutdown_connections 2023-12-08T08:38:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.339+0000 7f37517dc640 1 -- 172.21.15.16:0/1718088925 wait complete. 2023-12-08T08:38:44.397 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-12-08T08:38:44.398 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-08T08:38:44.399 INFO:teuthology.orchestra.run.smithi016.stdout:55834574858 2023-12-08T08:38:44.399 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-08T08:38:44.400 INFO:teuthology.orchestra.run.smithi016.stdout:55834574858 2023-12-08T08:38:44.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-08T08:38:44.401 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-12-08T08:38:44.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-08T08:38:44.401 INFO:teuthology.orchestra.run.smithi016.stdout:55834574859 2023-12-08T08:38:44.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-08T08:38:44.402 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-12-08T08:38:44.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-08T08:38:44.403 INFO:teuthology.orchestra.run.smithi016.stdout:55834574858 2023-12-08T08:38:44.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-08T08:38:44.404 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-12-08T08:38:44.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-08T08:38:44.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.619+0000 7f0552fe5640 1 Processor -- start 2023-12-08T08:38:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.619+0000 7f0552fe5640 1 -- start start 2023-12-08T08:38:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.620+0000 7f23f99ec640 1 Processor -- start 2023-12-08T08:38:44.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.620+0000 7f23f99ec640 1 -- start start 2023-12-08T08:38:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.620+0000 7f0552fe5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f054c0cf560 0x7f054c156b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.620+0000 7f0552fe5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f054c06cb80 0x7f054c06cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.620+0000 7f0552fe5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054c06c3b0 con 0x7f054c0b2b60 2023-12-08T08:38:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.620+0000 7f0552fe5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054c159560 con 0x7f054c06cb80 2023-12-08T08:38:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.620+0000 7f0552fe5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c15a530 con 0x7f054c0cf560 2023-12-08T08:38:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.621+0000 7f36b398a640 1 Processor -- start 2023-12-08T08:38:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.621+0000 7f36b398a640 1 -- start start 2023-12-08T08:38:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.621+0000 7fe06cc13640 1 Processor -- start 2023-12-08T08:38:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.621+0000 7fe06cc13640 1 -- start start 2023-12-08T08:38:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f37b8ee7640 1 Processor -- start 2023-12-08T08:38:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.621+0000 7f23f99ec640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23f406e110 0x7f23f413f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.621+0000 7f23f99ec640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23f4070230 0x7f23f4070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f37b8ee7640 1 -- start start 2023-12-08T08:38:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.621+0000 7f054bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f054c06cb80 0x7f054c06cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.621+0000 7f054bfff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f054c06cb80 0x7f054c06cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:46048/0 (socket says 172.21.15.16:46048) 2023-12-08T08:38:44.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.621+0000 7f054bfff640 1 -- 172.21.15.16:0/1667571626 learned_addr learned my addr 172.21.15.16:0/1667571626 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.621+0000 7f0550d81640 1 --2- 172.21.15.16:0/1667571626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f054c0cf560 0x7f054c156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f054bfff640 1 -- 172.21.15.16:0/1667571626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f054c0cf560 msgr2=0x7f054c156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f054bfff640 1 --2- 172.21.15.16:0/1667571626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f054c0cf560 0x7f054c156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f054bfff640 1 -- 172.21.15.16:0/1667571626 >> v1:172.21.15.112:6789/0 conn(0x7f054c0b2b60 legacy=0x7f054c15cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f054bfff640 1 -- 172.21.15.16:0/1667571626 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c06d500 con 0x7f054c06cb80 2023-12-08T08:38:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.621+0000 7f23f99ec640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f4070bd0 con 0x7f23f40b2060 2023-12-08T08:38:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.621+0000 7f23f99ec640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23f40b3910 con 0x7f23f406e110 2023-12-08T08:38:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.621+0000 7f23f99ec640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f415fce0 con 0x7f23f4070230 2023-12-08T08:38:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f23f2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23f406e110 0x7f23f413f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f23f2ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f23f406e110 0x7f23f413f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52502/0 (socket says 172.21.15.16:52502) 2023-12-08T08:38:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f23f2ffd640 1 -- 172.21.15.16:0/1558183063 learned_addr learned my addr 172.21.15.16:0/1558183063 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f36b398a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f36ac160a00 0x7f36ac0d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f36b398a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f36ac0b1ea0 0x7f36ac0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f36b398a640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac0b28e0 con 0x7f36ac0cfe10 2023-12-08T08:38:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f36b398a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ac0d02a0 con 0x7f36ac0b1ea0 2023-12-08T08:38:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f36b398a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ac0b2610 con 0x7f36ac160a00 2023-12-08T08:38:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f36b1f27640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f36ac0cfe10 0x7f36ac0d0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59748/0 (socket says 172.21.15.16:59748) 2023-12-08T08:38:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f36b1f27640 1 -- 172.21.15.16:0/1729988153 learned_addr learned my addr 172.21.15.16:0/1729988153 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.623+0000 7f36b0f25640 1 --2- 172.21.15.16:0/1729988153 >> v2:172.21.15.16:3300/0 conn(0x7f36ac0b1ea0 0x7f36ac0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.623+0000 7f0550d81640 1 --2- 172.21.15.16:0/1667571626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f054c0cf560 0x7f054c156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.623+0000 7f36a3fff640 1 -- 172.21.15.16:0/1729988153 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123461949 0 0) 0x7f36ac0b28e0 con 0x7f36ac0cfe10 2023-12-08T08:38:44.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.623+0000 7f36a3fff640 1 -- 172.21.15.16:0/1729988153 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3698003520 con 0x7f36ac0cfe10 2023-12-08T08:38:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f23f2ffd640 1 -- 172.21.15.16:0/1558183063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23f4070230 msgr2=0x7f23f4070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f23f2ffd640 1 --2- 172.21.15.16:0/1558183063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23f4070230 0x7f23f4070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f23f2ffd640 1 -- 172.21.15.16:0/1558183063 >> v1:172.21.15.112:6789/0 conn(0x7f23f40b2060 legacy=0x7f23f40b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.622+0000 7f23f2ffd640 1 -- 172.21.15.16:0/1558183063 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f40b2fe0 con 0x7f23f406e110 2023-12-08T08:38:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.623+0000 7fe06cc13640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe06806e0d0 0x7fe068071550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.623+0000 7fe06cc13640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe0680ced50 0x7fe0680cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.623+0000 7fe06cc13640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe068070a90 con 0x7fe068071a90 2023-12-08T08:38:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.623+0000 7fe06cc13640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0680cf720 con 0x7fe06806e0d0 2023-12-08T08:38:44.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.623+0000 7fe06cc13640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0680705a0 con 0x7fe0680ced50 2023-12-08T08:38:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.623+0000 7f36b0f25640 1 -- 172.21.15.16:0/1729988153 >> v2:172.21.15.112:3300/0 conn(0x7f36ac160a00 msgr2=0x7f36ac0d3a60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-08T08:38:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.623+0000 7f36b0f25640 1 --2- 172.21.15.16:0/1729988153 >> v2:172.21.15.112:3300/0 conn(0x7f36ac160a00 0x7f36ac0d3a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.623+0000 7f36b0f25640 1 -- 172.21.15.16:0/1729988153 >> v1:172.21.15.16:6789/0 conn(0x7f36ac0cfe10 legacy=0x7f36ac0d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.623+0000 7f36b0f25640 1 -- 172.21.15.16:0/1729988153 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac0ce7c0 con 0x7f36ac0b1ea0 2023-12-08T08:38:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7f37b8ee7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37b40cd240 0x7f37b40cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7f37b8ee7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37b40cf730 0x7f37b415edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7fe0677fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe06806e0d0 0x7fe068071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7fe0677fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe06806e0d0 0x7fe068071550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52512/0 (socket says 172.21.15.16:52512) 2023-12-08T08:38:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7fe0677fe640 1 -- 172.21.15.16:0/2536930429 learned_addr learned my addr 172.21.15.16:0/2536930429 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7fe067fff640 1 --2- 172.21.15.16:0/2536930429 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe0680ced50 0x7fe0680cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7f37b8ee7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f37b415f310 0x7f37b415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7f37b8ee7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b40cd620 con 0x7f37b40cd240 2023-12-08T08:38:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7f23f2ffd640 1 --2- 172.21.15.16:0/1558183063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f406e110 0x7f23f413f320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009870 tx=0x7f23e402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897112a7973d7371 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7f37b2d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f37b415f310 0x7f37b415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7f37b2d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f37b415f310 0x7f37b415a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:46058/0 (socket says 172.21.15.16:46058) 2023-12-08T08:38:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7f37b2d9d640 1 -- 172.21.15.16:0/1835728924 learned_addr learned my addr 172.21.15.16:0/1835728924 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7fe067fff640 1 -- 172.21.15.16:0/2536930429 >> v2:172.21.15.16:3300/0 conn(0x7fe06806e0d0 msgr2=0x7fe068071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7fe067fff640 1 --2- 172.21.15.16:0/2536930429 >> v2:172.21.15.16:3300/0 conn(0x7fe06806e0d0 0x7fe068071550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7fe067fff640 1 -- 172.21.15.16:0/2536930429 >> v1:172.21.15.16:6789/0 conn(0x7fe068071a90 legacy=0x7fe06813f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7fe067fff640 1 -- 172.21.15.16:0/2536930429 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe068070d70 con 0x7fe0680ced50 2023-12-08T08:38:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7f71eade3640 1 Processor -- start 2023-12-08T08:38:44.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f37b8ee7640 1 -- 172.21.15.16:0/1835728924 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b40cfb90 con 0x7f37b415f310 2023-12-08T08:38:44.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f71eade3640 1 -- start start 2023-12-08T08:38:44.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f054bfff640 1 --2- 172.21.15.16:0/1667571626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f054c06cb80 0x7f054c06cfc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0540009870 tx=0x7f054002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3906bb456dc77062 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f37b8ee7640 1 -- 172.21.15.16:0/1835728924 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b415c150 con 0x7f37b40cf730 2023-12-08T08:38:44.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f23f1ffb640 1 -- 172.21.15.16:0/1558183063 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e40316d0 con 0x7f23f406e110 2023-12-08T08:38:44.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f23f1ffb640 1 -- 172.21.15.16:0/1558183063 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e4031830 con 0x7f23f406e110 2023-12-08T08:38:44.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f23f1ffb640 1 -- 172.21.15.16:0/1558183063 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4031c00 con 0x7f23f406e110 2023-12-08T08:38:44.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f36b0f25640 1 --2- 172.21.15.16:0/1729988153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ac0b1ea0 0x7f36ac0cf8d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f36ac08d6b0 tx=0x7f369c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09ec268ab625da7 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7f45d4c87640 1 Processor -- start 2023-12-08T08:38:44.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.624+0000 7f45d4c87640 1 -- start start 2023-12-08T08:38:44.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f45d4c87640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45d00cb6d0 0x7f45d00cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f45d4c87640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f45d006c0c0 0x7f45d006c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f45d4c87640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d00cd970 con 0x7f45d00ae2b0 2023-12-08T08:38:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f45d4c87640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d00cbff0 con 0x7f45d00cb6d0 2023-12-08T08:38:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f45d4c87640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d00cde00 con 0x7f45d006c0c0 2023-12-08T08:38:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f36a3fff640 1 -- 172.21.15.16:0/1729988153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c031640 con 0x7f36ac0b1ea0 2023-12-08T08:38:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f36a3fff640 1 -- 172.21.15.16:0/1729988153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c0317a0 con 0x7f36ac0b1ea0 2023-12-08T08:38:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f36a3fff640 1 -- 172.21.15.16:0/1729988153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c031b30 con 0x7f36ac0b1ea0 2023-12-08T08:38:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f45ced9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f45d006c0c0 0x7f45d006c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7fe067fff640 1 --2- 172.21.15.16:0/2536930429 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe0680ced50 0x7fe0680cf1e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe0600097a0 tx=0x7fe06002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd0cd602b73d3b92 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f45ced9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f45d006c0c0 0x7f45d006c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:46070/0 (socket says 172.21.15.16:46070) 2023-12-08T08:38:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.625+0000 7f45ced9d640 1 -- 172.21.15.16:0/1962801203 learned_addr learned my addr 172.21.15.16:0/1962801203 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f45ce59c640 1 --2- 172.21.15.16:0/1962801203 >> v2:172.21.15.16:3300/0 conn(0x7f45d00cb6d0 0x7f45d00cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7fe0667fc640 1 -- 172.21.15.16:0/2536930429 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060030800 con 0x7fe0680ced50 2023-12-08T08:38:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f45ced9d640 1 -- 172.21.15.16:0/1962801203 >> v2:172.21.15.16:3300/0 conn(0x7f45d00cb6d0 msgr2=0x7f45d00cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f45ced9d640 1 --2- 172.21.15.16:0/1962801203 >> v2:172.21.15.16:3300/0 conn(0x7f45d00cb6d0 0x7f45d00cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f45ced9d640 1 -- 172.21.15.16:0/1962801203 >> v1:172.21.15.16:6789/0 conn(0x7f45d00ae2b0 legacy=0x7f45d00b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7fe0667fc640 1 -- 172.21.15.16:0/2536930429 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe060030960 con 0x7fe0680ced50 2023-12-08T08:38:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f23f99ec640 1 -- 172.21.15.16:0/1558183063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f406e110 msgr2=0x7f23f413f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f23f99ec640 1 --2- 172.21.15.16:0/1558183063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f406e110 0x7f23f413f320 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009870 tx=0x7f23e402b7f0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f71eade3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71dc0df220 0x7f71dc038080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f37b2d9d640 1 -- 172.21.15.16:0/1835728924 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37b40cf730 msgr2=0x7f37b415edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f37b2d9d640 1 --2- 172.21.15.16:0/1835728924 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37b40cf730 0x7f37b415edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f37b2d9d640 1 -- 172.21.15.16:0/1835728924 >> v2:172.21.15.16:3300/0 conn(0x7f37b40cd240 msgr2=0x7f37b40cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f23f99ec640 1 -- 172.21.15.16:0/1558183063 shutdown_connections 2023-12-08T08:38:44.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f36b398a640 1 -- 172.21.15.16:0/1729988153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ac0b1ea0 msgr2=0x7f36ac0cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f36b398a640 1 --2- 172.21.15.16:0/1729988153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ac0b1ea0 0x7f36ac0cf8d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f36ac08d6b0 tx=0x7f369c02b8f0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f71e8b7f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71dc0df220 0x7f71dc038080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f71e8b7f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71dc0df220 0x7f71dc038080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52532/0 (socket says 172.21.15.16:52532) 2023-12-08T08:38:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f37b259c640 1 --2- 172.21.15.16:0/1835728924 >> v2:172.21.15.16:3300/0 conn(0x7f37b40cd240 0x7f37b40cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f71e8b7f640 1 -- 172.21.15.16:0/286307561 learned_addr learned my addr 172.21.15.16:0/286307561 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f71eade3640 1 --2- 172.21.15.16:0/286307561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71dc053c70 0x7f71dc04fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f23f99ec640 1 --2- 172.21.15.16:0/1558183063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23f4070230 0x7f23f4070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f23f99ec640 1 --2- 172.21.15.16:0/1558183063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f406e110 0x7f23f413f320 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f36b398a640 1 -- 172.21.15.16:0/1729988153 shutdown_connections 2023-12-08T08:38:44.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f36b398a640 1 --2- 172.21.15.16:0/1729988153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ac0b1ea0 0x7f36ac0cf8d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f36b398a640 1 --2- 172.21.15.16:0/1729988153 >> v2:172.21.15.112:3300/0 conn(0x7f36ac160a00 0x7f36ac0d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f36b398a640 1 -- 172.21.15.16:0/1729988153 >> 172.21.15.16:0/1729988153 conn(0x7f36ac09c300 msgr2=0x7f36ac0d2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f23f99ec640 1 -- 172.21.15.16:0/1558183063 >> 172.21.15.16:0/1558183063 conn(0x7f23f40b0750 msgr2=0x7f23f40b0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f71eade3640 1 -- 172.21.15.16:0/286307561 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71dc0544c0 con 0x7f71dc033ef0 2023-12-08T08:38:44.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f71eade3640 1 -- 172.21.15.16:0/286307561 --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71dc0541e0 con 0x7f71dc0df220 2023-12-08T08:38:44.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f054b7fe640 1 -- 172.21.15.16:0/1667571626 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540030860 con 0x7f054c06cb80 2023-12-08T08:38:44.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f71eade3640 1 -- 172.21.15.16:0/286307561 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc005f30 con 0x7f71dc053c70 2023-12-08T08:38:44.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f054b7fe640 1 -- 172.21.15.16:0/1667571626 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05400309c0 con 0x7f054c06cb80 2023-12-08T08:38:44.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f054b7fe640 1 -- 172.21.15.16:0/1667571626 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540030d90 con 0x7f054c06cb80 2023-12-08T08:38:44.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7fe0667fc640 1 -- 172.21.15.16:0/2536930429 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe060030cf0 con 0x7fe0680ced50 2023-12-08T08:38:44.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f38e9fe6640 1 Processor -- start 2023-12-08T08:38:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f37b2d9d640 1 --2- 172.21.15.16:0/1835728924 >> v2:172.21.15.16:3300/0 conn(0x7f37b40cd240 0x7f37b40cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f38e9fe6640 1 -- start start 2023-12-08T08:38:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f36b398a640 1 -- 172.21.15.16:0/1729988153 shutdown_connections 2023-12-08T08:38:44.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f37b2d9d640 1 -- 172.21.15.16:0/1835728924 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b415fb00 con 0x7f37b415f310 2023-12-08T08:38:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f37b259c640 1 --2- 172.21.15.16:0/1835728924 >> v2:172.21.15.16:3300/0 conn(0x7f37b40cd240 0x7f37b40cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f0552fe5640 1 -- 172.21.15.16:0/1667571626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f054c06cb80 msgr2=0x7f054c06cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f0552fe5640 1 --2- 172.21.15.16:0/1667571626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f054c06cb80 0x7f054c06cfc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0540009870 tx=0x7f054002b9a0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7fe06cc13640 1 -- 172.21.15.16:0/2536930429 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe0680ced50 msgr2=0x7fe0680cf1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7fe06cc13640 1 --2- 172.21.15.16:0/2536930429 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe0680ced50 0x7fe0680cf1e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe0600097a0 tx=0x7fe06002b940 comp rx=0 tx=0).stop 2023-12-08T08:38:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f71e3fff640 1 --2- 172.21.15.16:0/286307561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71dc053c70 0x7f71dc04fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f71e8b7f640 1 -- 172.21.15.16:0/286307561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71dc053c70 msgr2=0x7f71dc04fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f71e8b7f640 1 --2- 172.21.15.16:0/286307561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71dc053c70 0x7f71dc04fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f71e8b7f640 1 -- 172.21.15.16:0/286307561 >> v1:172.21.15.112:6789/0 conn(0x7f71dc033ef0 legacy=0x7f71dc006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f71e8b7f640 1 -- 172.21.15.16:0/286307561 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc006210 con 0x7f71dc0df220 2023-12-08T08:38:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f71e37fe640 1 -- 172.21.15.16:0/286307561 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 358509434 0 0) 0x7f71dc0544c0 con 0x7f71dc033ef0 2023-12-08T08:38:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7f0552fe5640 1 -- 172.21.15.16:0/1667571626 shutdown_connections 2023-12-08T08:38:44.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7f0552fe5640 1 --2- 172.21.15.16:0/1667571626 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f054c06cb80 0x7f054c06cfc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7f0552fe5640 1 --2- 172.21.15.16:0/1667571626 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f054c0cf560 0x7f054c156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7f0552fe5640 1 -- 172.21.15.16:0/1667571626 >> 172.21.15.16:0/1667571626 conn(0x7f054c0cea40 msgr2=0x7f054c0cee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f45ced9d640 1 -- 172.21.15.16:0/1962801203 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d00ccd60 con 0x7f45d006c0c0 2023-12-08T08:38:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7fe06cc13640 1 -- 172.21.15.16:0/2536930429 shutdown_connections 2023-12-08T08:38:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7fe06cc13640 1 --2- 172.21.15.16:0/2536930429 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe0680ced50 0x7fe0680cf1e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7fe06cc13640 1 --2- 172.21.15.16:0/2536930429 >> v2:172.21.15.16:3300/0 conn(0x7fe06806e0d0 0x7fe068071550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7fe06cc13640 1 -- 172.21.15.16:0/2536930429 >> 172.21.15.16:0/2536930429 conn(0x7fe06806a6b0 msgr2=0x7fe06806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f23f99ec640 1 -- 172.21.15.16:0/1558183063 shutdown_connections 2023-12-08T08:38:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.626+0000 7f23f99ec640 1 -- 172.21.15.16:0/1558183063 wait complete. 2023-12-08T08:38:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7f0552fe5640 1 -- 172.21.15.16:0/1667571626 shutdown_connections 2023-12-08T08:38:44.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7f37b2d9d640 1 --2- 172.21.15.16:0/1835728924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37b415f310 0x7f37b415a9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37b40724c0 tx=0x7f37ac02bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cc69e19145484f server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7f37b159a640 1 -- 172.21.15.16:0/1835728924 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac02ca80 con 0x7f37b415f310 2023-12-08T08:38:44.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7f37b159a640 1 -- 172.21.15.16:0/1835728924 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac02cbe0 con 0x7f37b415f310 2023-12-08T08:38:44.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7f37b159a640 1 -- 172.21.15.16:0/1835728924 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac0343f0 con 0x7f37b415f310 2023-12-08T08:38:44.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7f38e9fe6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38dc0e0060 0x7f38dc034f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.628+0000 7f0552fe5640 1 -- 172.21.15.16:0/1667571626 wait complete. 2023-12-08T08:38:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e9fe6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38dc0508f0 con 0x7f38dc0503f0 2023-12-08T08:38:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f38dc0503f0 0x7f38dc038da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59770/0 (socket says 172.21.15.16:59770) 2023-12-08T08:38:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e3fff640 1 -- 172.21.15.16:0/4227170714 learned_addr learned my addr 172.21.15.16:0/4227170714 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e8fe4640 1 --2- 172.21.15.16:0/4227170714 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38dc0e0060 0x7f38dc034f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e9fe6640 1 -- 172.21.15.16:0/4227170714 --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38dc034cd0 con 0x7f38dc054fc0 2023-12-08T08:38:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e9fe6640 1 -- 172.21.15.16:0/4227170714 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc007cf0 con 0x7f38dc0e0060 2023-12-08T08:38:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881711340 0 0) 0x7f38dc0508f0 con 0x7f38dc0503f0 2023-12-08T08:38:44.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d4003520 con 0x7f38dc0503f0 2023-12-08T08:38:44.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f37b8ee7640 1 -- 172.21.15.16:0/1835728924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37b415f310 msgr2=0x7f37b415a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370081478 0 0) 0x7f38dc034cd0 con 0x7f38dc054fc0 2023-12-08T08:38:44.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38dc0508f0 con 0x7f38dc054fc0 2023-12-08T08:38:44.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f37b8ee7640 1 --2- 172.21.15.16:0/1835728924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37b415f310 0x7f37b415a9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37b40724c0 tx=0x7f37ac02bee0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 44690369 0 0) 0x7f38d4003520 con 0x7f38dc0503f0 2023-12-08T08:38:44.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38dc0e0060 msgr2=0x7f38dc034f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e37fe640 1 --2- 172.21.15.16:0/4227170714 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38dc0e0060 0x7f38dc034f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 >> v1:172.21.15.112:6789/0 conn(0x7f38dc054fc0 legacy=0x7f38dc008ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f23f99ec640 1 Processor -- start 2023-12-08T08:38:44.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc007fd0 con 0x7f38dc0503f0 2023-12-08T08:38:44.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f38d0001670 con 0x7f38dc0503f0 2023-12-08T08:38:44.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f71e3fff640 1 --2- 172.21.15.16:0/286307561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71dc053c70 0x7f71dc04fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:44.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 >> v1:172.21.15.16:6789/0 conn(0x7f38dc0503f0 legacy=0x7f38dc038da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f37b8ee7640 1 -- 172.21.15.16:0/1835728924 shutdown_connections 2023-12-08T08:38:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f37b8ee7640 1 --2- 172.21.15.16:0/1835728924 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37b415f310 0x7f37b415a9f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f45ced9d640 1 --2- 172.21.15.16:0/1962801203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45d006c0c0 0x7f45d006c550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f45d006eca0 tx=0x7f45c802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f662382d7b275484 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f37b8ee7640 1 --2- 172.21.15.16:0/1835728924 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37b40cf730 0x7f37b415edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f37b8ee7640 1 --2- 172.21.15.16:0/1835728924 >> v2:172.21.15.16:3300/0 conn(0x7f37b40cd240 0x7f37b40cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f38e37fe640 1 --2- 172.21.15.16:0/4227170714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38d4003b30 0x7f38d4003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f38e37fe640 1 --2- 172.21.15.16:0/4227170714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d40047c0 0x7f38d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f38e37fe640 1 --2- 172.21.15.16:0/4227170714 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38d4005380 0x7f38d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc007cf0 con 0x7f38d40047c0 2023-12-08T08:38:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f37b8ee7640 1 -- 172.21.15.16:0/1835728924 >> 172.21.15.16:0/1835728924 conn(0x7f37b4099a10 msgr2=0x7f37b4099e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40105b0 con 0x7f38d4003b30 2023-12-08T08:38:44.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4010770 con 0x7f38d4005380 2023-12-08T08:38:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f23f99ec640 1 -- start start 2023-12-08T08:38:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f37b8ee7640 1 -- 172.21.15.16:0/1835728924 shutdown_connections 2023-12-08T08:38:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f45cd59a640 1 -- 172.21.15.16:0/1962801203 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8030860 con 0x7f45d006c0c0 2023-12-08T08:38:44.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f45cd59a640 1 -- 172.21.15.16:0/1962801203 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c80309c0 con 0x7f45d006c0c0 2023-12-08T08:38:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f37b8ee7640 1 -- 172.21.15.16:0/1835728924 wait complete. 2023-12-08T08:38:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f45cd59a640 1 -- 172.21.15.16:0/1962801203 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8030d50 con 0x7f45d006c0c0 2023-12-08T08:38:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f0552fe5640 1 Processor -- start 2023-12-08T08:38:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f38e97e5640 1 --2- 172.21.15.16:0/4227170714 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38d4005380 0x7f38d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f38e8fe4640 1 --2- 172.21.15.16:0/4227170714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38d4003b30 0x7f38d4003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f38e3fff640 1 --2- 172.21.15.16:0/4227170714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d40047c0 0x7f38d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.627+0000 7f36b398a640 1 -- 172.21.15.16:0/1729988153 wait complete. 2023-12-08T08:38:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f36b398a640 1 Processor -- start 2023-12-08T08:38:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.629+0000 7f36b398a640 1 -- start start 2023-12-08T08:38:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f71e8b7f640 1 --2- 172.21.15.16:0/286307561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71dc0df220 0x7f71dc038080 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009870 tx=0x7f71d802fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4947d75371f54a67 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f38e97e5640 1 -- 172.21.15.16:0/4227170714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38d4003b30 msgr2=0x7f38d4003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f38e97e5640 1 --2- 172.21.15.16:0/4227170714 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38d4003b30 0x7f38d4003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f38e97e5640 1 -- 172.21.15.16:0/4227170714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d40047c0 msgr2=0x7f38d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:44.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f38e97e5640 1 --2- 172.21.15.16:0/4227170714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38d40047c0 0x7f38d4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f38e97e5640 1 -- 172.21.15.16:0/4227170714 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc007fd0 con 0x7f38d4005380 2023-12-08T08:38:44.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7fe06cc13640 1 -- 172.21.15.16:0/2536930429 shutdown_connections 2023-12-08T08:38:44.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f71e37fe640 1 -- 172.21.15.16:0/286307561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8030af0 con 0x7f71dc0df220 2023-12-08T08:38:44.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f71e37fe640 1 -- 172.21.15.16:0/286307561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d8030c50 con 0x7f71dc0df220 2023-12-08T08:38:44.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b398a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36ac0b1ea0 0x7f36ac204a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b398a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ac0cfe10 0x7f36ac0ccce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b398a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36ac160a00 0x7f36ac205360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b398a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac0ca330 con 0x7f36ac0cfe10 2023-12-08T08:38:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b398a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac0cd220 con 0x7f36ac0b1ea0 2023-12-08T08:38:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b398a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac0ca5f0 con 0x7f36ac160a00 2023-12-08T08:38:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b1f27640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36ac160a00 0x7f36ac205360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b1f27640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36ac160a00 0x7f36ac205360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58302/0 (socket says 172.21.15.16:58302) 2023-12-08T08:38:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b1f27640 1 -- 172.21.15.16:0/349354641 learned_addr learned my addr 172.21.15.16:0/349354641 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b0f25640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ac0cfe10 0x7f36ac0ccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b1726640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36ac0b1ea0 0x7f36ac204a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b1f27640 1 -- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36ac0b1ea0 msgr2=0x7f36ac204a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b1f27640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36ac0b1ea0 0x7f36ac204a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b1f27640 1 -- 172.21.15.16:0/349354641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ac0cfe10 msgr2=0x7f36ac0ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b1f27640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ac0cfe10 0x7f36ac0ccce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b1f27640 1 -- 172.21.15.16:0/349354641 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c009400 con 0x7f36ac160a00 2023-12-08T08:38:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b0f25640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ac0cfe10 0x7f36ac0ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f36b1726640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36ac0b1ea0 0x7f36ac204a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f71e37fe640 1 -- 172.21.15.16:0/286307561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8038440 con 0x7f71dc0df220 2023-12-08T08:38:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f23f99ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f406e110 0x7f23f4137680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f23f99ec640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23f4070230 0x7f23f406c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f23f99ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f40b2060 0x7f23f406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f23f99ec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f4159ec0 con 0x7f23f40b2060 2023-12-08T08:38:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f23f99ec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f415a1a0 con 0x7f23f406e110 2023-12-08T08:38:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f23f99ec640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f415a480 con 0x7f23f4070230 2023-12-08T08:38:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f406e110 0x7f23f4137680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f23f2ffd640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f406e110 0x7f23f4137680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:46096/0 (socket says 172.21.15.16:46096) 2023-12-08T08:38:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f23f2ffd640 1 -- 172.21.15.16:0/97281473 learned_addr learned my addr 172.21.15.16:0/97281473 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.631+0000 7f23f37fe640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f40b2060 0x7f23f406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f45d4c87640 1 -- 172.21.15.16:0/1962801203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45d006c0c0 msgr2=0x7f45d006c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f45d4c87640 1 --2- 172.21.15.16:0/1962801203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45d006c0c0 0x7f45d006c550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f45d006eca0 tx=0x7f45c802b9a0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f36b1f27640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36ac160a00 0x7f36ac205360 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f36a800c3c0 tx=0x7f36a800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.630+0000 7f0552fe5640 1 -- start start 2023-12-08T08:38:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0552fe5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f054c06cb80 0x7f054c145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0552fe5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f054c0b2b60 0x7f054c13e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0552fe5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054c0cf560 0x7f054c13e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0552fe5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c13ee90 con 0x7f054c0cf560 2023-12-08T08:38:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0552fe5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c1581f0 con 0x7f054c0b2b60 2023-12-08T08:38:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0552fe5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c1575e0 con 0x7f054c06cb80 2023-12-08T08:38:44.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f38e97e5640 1 --2- 172.21.15.16:0/4227170714 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38d4005380 0x7f38d400b980 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38d80018a0 tx=0x7f38d40057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5db31a193c21fe95 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f37b8ee7640 1 Processor -- start 2023-12-08T08:38:44.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0550d81640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f054c06cb80 0x7f054c145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0550d81640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f054c06cb80 0x7f054c145030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58314/0 (socket says 172.21.15.16:58314) 2023-12-08T08:38:44.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0550d81640 1 -- 172.21.15.16:0/1031242014 learned_addr learned my addr 172.21.15.16:0/1031242014 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f71eade3640 1 -- 172.21.15.16:0/286307561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71dc0df220 msgr2=0x7f71dc038080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f71eade3640 1 --2- 172.21.15.16:0/286307561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71dc0df220 0x7f71dc038080 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009870 tx=0x7f71d802fd40 comp rx=0 tx=0).stop 2023-12-08T08:38:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f37b8ee7640 1 -- start start 2023-12-08T08:38:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f45d4c87640 1 -- 172.21.15.16:0/1962801203 shutdown_connections 2023-12-08T08:38:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f45d4c87640 1 --2- 172.21.15.16:0/1962801203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45d006c0c0 0x7f45d006c550 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f45d4c87640 1 --2- 172.21.15.16:0/1962801203 >> v2:172.21.15.16:3300/0 conn(0x7f45d00cb6d0 0x7f45d00cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f45d4c87640 1 -- 172.21.15.16:0/1962801203 >> 172.21.15.16:0/1962801203 conn(0x7f45d00b09f0 msgr2=0x7f45d00b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f71eade3640 1 -- 172.21.15.16:0/286307561 shutdown_connections 2023-12-08T08:38:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f71eade3640 1 --2- 172.21.15.16:0/286307561 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71dc053c70 0x7f71dc04fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f71eade3640 1 --2- 172.21.15.16:0/286307561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71dc0df220 0x7f71dc038080 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f71eade3640 1 -- 172.21.15.16:0/286307561 >> 172.21.15.16:0/286307561 conn(0x7f71dc009fb0 msgr2=0x7f71dc00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0551582640 1 --2- 172.21.15.16:0/1031242014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054c0cf560 0x7f054c13e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d802e070 con 0x7f38d4005380 2023-12-08T08:38:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d80050d0 con 0x7f38d4005380 2023-12-08T08:38:44.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f38e37fe640 1 -- 172.21.15.16:0/4227170714 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8005460 con 0x7f38d4005380 2023-12-08T08:38:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f23f27fc640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23f4070230 0x7f23f406c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f71eade3640 1 -- 172.21.15.16:0/286307561 shutdown_connections 2023-12-08T08:38:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f45d4c87640 1 -- 172.21.15.16:0/1962801203 shutdown_connections 2023-12-08T08:38:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0550d81640 1 -- 172.21.15.16:0/1031242014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f054c0b2b60 msgr2=0x7f054c13e410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-08T08:38:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0550d81640 1 --2- 172.21.15.16:0/1031242014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f054c0b2b60 0x7f054c13e410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0550d81640 1 -- 172.21.15.16:0/1031242014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054c0cf560 msgr2=0x7f054c13e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0550d81640 1 --2- 172.21.15.16:0/1031242014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054c0cf560 0x7f054c13e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f0550d81640 1 -- 172.21.15.16:0/1031242014 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0544009a70 con 0x7f054c06cb80 2023-12-08T08:38:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f0550d81640 1 --2- 172.21.15.16:0/1031242014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f054c06cb80 0x7f054c145030 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f054400efc0 tx=0x7f054400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f0551582640 1 --2- 172.21.15.16:0/1031242014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054c0cf560 0x7f054c13e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f23f2ffd640 1 -- 172.21.15.16:0/97281473 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23f4070230 msgr2=0x7f23f406c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f23f2ffd640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23f4070230 0x7f23f406c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f23f2ffd640 1 -- 172.21.15.16:0/97281473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f40b2060 msgr2=0x7f23f406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f23f2ffd640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f40b2060 0x7f23f406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f23f2ffd640 1 -- 172.21.15.16:0/97281473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e40094d0 con 0x7f23f406e110 2023-12-08T08:38:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f38e9fe6640 1 -- 172.21.15.16:0/4227170714 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38d4005380 msgr2=0x7f38d400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f23f37fe640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f40b2060 0x7f23f406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7fe06cc13640 1 -- 172.21.15.16:0/2536930429 wait complete. 2023-12-08T08:38:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f38e9fe6640 1 --2- 172.21.15.16:0/4227170714 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38d4005380 0x7f38d400b980 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f38d80018a0 tx=0x7f38d40057f0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f71eade3640 1 -- 172.21.15.16:0/286307561 wait complete. 2023-12-08T08:38:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f36a1ffb640 1 -- 172.21.15.16:0/349354641 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a80074e0 con 0x7f36ac160a00 2023-12-08T08:38:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f36a1ffb640 1 -- 172.21.15.16:0/349354641 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a8007640 con 0x7f36ac160a00 2023-12-08T08:38:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f36a1ffb640 1 -- 172.21.15.16:0/349354641 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a8004a20 con 0x7f36ac160a00 2023-12-08T08:38:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f37b8ee7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40cd240 0x7f37b40b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f37b8ee7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37b40cf730 0x7f37b40b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f37b8ee7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37b415f310 0x7f37b40b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f36b398a640 1 -- 172.21.15.16:0/349354641 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ac0ca8d0 con 0x7f36ac160a00 2023-12-08T08:38:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f36b398a640 1 -- 172.21.15.16:0/349354641 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ac0cae90 con 0x7f36ac160a00 2023-12-08T08:38:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f37b8ee7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40b0800 con 0x7f37b40cd240 2023-12-08T08:38:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f37b8ee7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40b2700 con 0x7f37b40cf730 2023-12-08T08:38:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f37b8ee7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40c7550 con 0x7f37b415f310 2023-12-08T08:38:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.633+0000 7f05497fa640 1 -- 172.21.15.16:0/1031242014 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544014680 con 0x7f054c06cb80 2023-12-08T08:38:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f05497fa640 1 -- 172.21.15.16:0/1031242014 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054400f040 con 0x7f054c06cb80 2023-12-08T08:38:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f05497fa640 1 -- 172.21.15.16:0/1031242014 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05440149e0 con 0x7f054c06cb80 2023-12-08T08:38:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.632+0000 7f45d4c87640 1 -- 172.21.15.16:0/1962801203 wait complete. 2023-12-08T08:38:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f37b2d9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37b415f310 0x7f37b40b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f0552fe5640 1 -- 172.21.15.16:0/1031242014 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05400094d0 con 0x7f054c06cb80 2023-12-08T08:38:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f37b259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40cd240 0x7f37b40b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f37b2d9d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37b415f310 0x7f37b40b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58336/0 (socket says 172.21.15.16:58336) 2023-12-08T08:38:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f37b2d9d640 1 -- 172.21.15.16:0/3324583284 learned_addr learned my addr 172.21.15.16:0/3324583284 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f0552fe5640 1 -- 172.21.15.16:0/1031242014 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054c157ba0 con 0x7f054c06cb80 2023-12-08T08:38:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f38e97e5640 1 -- 172.21.15.16:0/4227170714 reap_dead start 2023-12-08T08:38:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7fe06cc13640 1 Processor -- start 2023-12-08T08:38:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f38e9fe6640 1 -- 172.21.15.16:0/4227170714 shutdown_connections 2023-12-08T08:38:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f37b1d9b640 1 --2- 172.21.15.16:0/3324583284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37b40cf730 0x7f37b40b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f38e9fe6640 1 -- 172.21.15.16:0/4227170714 >> 172.21.15.16:0/4227170714 conn(0x7f38dc055b90 msgr2=0x7f38dc055fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f37b259c640 1 -- 172.21.15.16:0/3324583284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37b415f310 msgr2=0x7f37b40b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f37b259c640 1 --2- 172.21.15.16:0/3324583284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37b415f310 0x7f37b40b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f37b259c640 1 -- 172.21.15.16:0/3324583284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37b40cf730 msgr2=0x7f37b40b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f37b259c640 1 --2- 172.21.15.16:0/3324583284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37b40cf730 0x7f37b40b4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f37b259c640 1 -- 172.21.15.16:0/3324583284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac009b30 con 0x7f37b40cd240 2023-12-08T08:38:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f71eade3640 1 Processor -- start 2023-12-08T08:38:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7fe06cc13640 1 -- start start 2023-12-08T08:38:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.634+0000 7f23f2ffd640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f406e110 0x7f23f4137680 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23e402b830 tx=0x7f23e40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f23dbfff640 1 -- 172.21.15.16:0/97281473 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4034ab0 con 0x7f23f406e110 2023-12-08T08:38:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f36b398a640 1 -- 172.21.15.16:0/349354641 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ac0cb0b0 con 0x7f36ac160a00 2023-12-08T08:38:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f37b259c640 1 --2- 172.21.15.16:0/3324583284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40cd240 0x7f37b40b21c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f37a400c830 tx=0x7f37a400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f38e9fe6640 1 -- 172.21.15.16:0/4227170714 shutdown_connections 2023-12-08T08:38:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f37a2ffd640 1 -- 172.21.15.16:0/3324583284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4007820 con 0x7f37b40cd240 2023-12-08T08:38:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f37a2ffd640 1 -- 172.21.15.16:0/3324583284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a4007980 con 0x7f37b40cd240 2023-12-08T08:38:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f37a2ffd640 1 -- 172.21.15.16:0/3324583284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4007d50 con 0x7f37b40cd240 2023-12-08T08:38:44.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f38e9fe6640 1 -- 172.21.15.16:0/4227170714 wait complete. 2023-12-08T08:38:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7fe06cc13640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe06806e0d0 0x7fe0681487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7f0552fe5640 1 -- 172.21.15.16:0/1031242014 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054c0673c0 con 0x7f054c06cb80 2023-12-08T08:38:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7fe06cc13640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe068071a90 0x7fe068169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f71eade3640 1 -- start start 2023-12-08T08:38:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7fe0677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe06806e0d0 0x7fe0681487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7fe0677fe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe06806e0d0 0x7fe0681487b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:46128/0 (socket says 172.21.15.16:46128) 2023-12-08T08:38:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7fe0677fe640 1 -- 172.21.15.16:0/1840069295 learned_addr learned my addr 172.21.15.16:0/1840069295 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7fe06cc13640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0680ced50 0x7fe0680cde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7f71eade3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc033ef0 0x7f71dc0d92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7f71eade3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71dc053c70 0x7f71dc04a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7f71eade3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71dc0df220 0x7f71dc04a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7f71eade3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc04d1e0 con 0x7f71dc0df220 2023-12-08T08:38:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7f71eade3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc049370 con 0x7f71dc033ef0 2023-12-08T08:38:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7f71eade3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc049650 con 0x7f71dc053c70 2023-12-08T08:38:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f37b8ee7640 1 -- 172.21.15.16:0/3324583284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b40b0a10 con 0x7f37b40cd240 2023-12-08T08:38:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f37b8ee7640 1 -- 172.21.15.16:0/3324583284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b40b0fd0 con 0x7f37b40cd240 2023-12-08T08:38:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7f05497fa640 1 -- 172.21.15.16:0/1031242014 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0544014b40 con 0x7f054c06cb80 2023-12-08T08:38:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7fe066ffd640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe068071a90 0x7fe068169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f71e9380640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71dc0df220 0x7f71dc04a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f45d4c87640 1 Processor -- start 2023-12-08T08:38:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7f45d4c87640 1 -- start start 2023-12-08T08:38:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f45d4c87640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45d006c0c0 0x7f45d01545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f45d4c87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d00ae2b0 0x7f45d015fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f45d4c87640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45d00cb6d0 0x7f45d015ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f45d4c87640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d015c130 con 0x7f45d00ae2b0 2023-12-08T08:38:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f45d4c87640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d0154b00 con 0x7f45d00cb6d0 2023-12-08T08:38:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f45d4c87640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d015b520 con 0x7f45d006c0c0 2023-12-08T08:38:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f71e9380640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71dc0df220 0x7f71dc04a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52562/0 (socket says 172.21.15.16:52562) 2023-12-08T08:38:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f71e9380640 1 -- 172.21.15.16:0/2262180625 learned_addr learned my addr 172.21.15.16:0/2262180625 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f71e8b7f640 1 --2- 172.21.15.16:0/2262180625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc033ef0 0x7f71dc0d92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7fe06cc13640 1 -- 172.21.15.16:0/1840069295 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe068148cf0 con 0x7fe0680ced50 2023-12-08T08:38:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7fe06cc13640 1 -- 172.21.15.16:0/1840069295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe068147710 con 0x7fe06806e0d0 2023-12-08T08:38:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7fe06cc13640 1 -- 172.21.15.16:0/1840069295 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0681479f0 con 0x7fe068071a90 2023-12-08T08:38:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7fe067fff640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0680ced50 0x7fe0680cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f23f99ec640 1 -- 172.21.15.16:0/97281473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f415a760 con 0x7f23f406e110 2023-12-08T08:38:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.635+0000 7f23f99ec640 1 -- 172.21.15.16:0/97281473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f4159b00 con 0x7f23f406e110 2023-12-08T08:38:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f23f99ec640 1 -- 172.21.15.16:0/97281473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f415ac80 con 0x7f23f406e110 2023-12-08T08:38:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f23dbfff640 1 -- 172.21.15.16:0/97281473 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e4034c10 con 0x7f23f406e110 2023-12-08T08:38:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f23dbfff640 1 -- 172.21.15.16:0/97281473 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e403e470 con 0x7f23f406e110 2023-12-08T08:38:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.636+0000 7f36a1ffb640 1 -- 172.21.15.16:0/349354641 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f36a8004b80 con 0x7f36ac160a00 2023-12-08T08:38:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f36a1ffb640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f369006ae90 0x7f369006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f36a1ffb640 1 -- 172.21.15.16:0/349354641 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f36a8004e30 con 0x7f36ac160a00 2023-12-08T08:38:44.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f36b1726640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f369006ae90 0x7f369006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f05497fa640 1 --2- 172.21.15.16:0/1031242014 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f053806af40 0x7f053806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f05497fa640 1 -- 172.21.15.16:0/1031242014 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f054408a860 con 0x7f054c06cb80 2023-12-08T08:38:44.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f054bfff640 1 --2- 172.21.15.16:0/1031242014 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f053806af40 0x7f053806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7f45ced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45d00cb6d0 0x7f45d015ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7fe0677fe640 1 -- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe068071a90 msgr2=0x7fe068169bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7fe0677fe640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe068071a90 0x7fe068169bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7f45ce59c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45d006c0c0 0x7f45d01545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7fe0677fe640 1 -- 172.21.15.16:0/1840069295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0680ced50 msgr2=0x7fe0680cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7fe0677fe640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0680ced50 0x7fe0680cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7fe0677fe640 1 -- 172.21.15.16:0/1840069295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe060009400 con 0x7fe06806e0d0 2023-12-08T08:38:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7fe066ffd640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe068071a90 0x7fe068169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7f45ced9d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45d00cb6d0 0x7f45d015ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:46146/0 (socket says 172.21.15.16:46146) 2023-12-08T08:38:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7f45ced9d640 1 -- 172.21.15.16:0/4002870634 learned_addr learned my addr 172.21.15.16:0/4002870634 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7f36b1726640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f369006ae90 0x7f369006d350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f36ac15c9f0 tx=0x7f36a401a580 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.637+0000 7f38e9fe6640 1 Processor -- start 2023-12-08T08:38:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7f38e9fe6640 1 -- start start 2023-12-08T08:38:44.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7f71e8b7f640 1 -- 172.21.15.16:0/2262180625 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71dc053c70 msgr2=0x7f71dc04a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7f71e8b7f640 1 --2- 172.21.15.16:0/2262180625 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71dc053c70 0x7f71dc04a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7f71e8b7f640 1 -- 172.21.15.16:0/2262180625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71dc0df220 msgr2=0x7f71dc04a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7f71e8b7f640 1 --2- 172.21.15.16:0/2262180625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71dc0df220 0x7f71dc04a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7f71e8b7f640 1 -- 172.21.15.16:0/2262180625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d80094d0 con 0x7f71dc033ef0 2023-12-08T08:38:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7f37b8ee7640 1 -- 172.21.15.16:0/3324583284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b415bda0 con 0x7f37b40cd240 2023-12-08T08:38:44.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7fe0677fe640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe06806e0d0 0x7fe0681487b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe05800efc0 tx=0x7fe05800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.638+0000 7f45cdd9b640 1 --2- 172.21.15.16:0/4002870634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d00ae2b0 0x7f45d015fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f45ced9d640 1 -- 172.21.15.16:0/4002870634 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45d006c0c0 msgr2=0x7f45d01545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f45ced9d640 1 --2- 172.21.15.16:0/4002870634 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45d006c0c0 0x7f45d01545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f45ced9d640 1 -- 172.21.15.16:0/4002870634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d00ae2b0 msgr2=0x7f45d015fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f45ced9d640 1 --2- 172.21.15.16:0/4002870634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d00ae2b0 0x7f45d015fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f45ced9d640 1 -- 172.21.15.16:0/4002870634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c80094d0 con 0x7f45d00cb6d0 2023-12-08T08:38:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f71e8b7f640 1 --2- 172.21.15.16:0/2262180625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc033ef0 0x7f71dc0d92b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f71d8037000 tx=0x7f71d80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f45ced9d640 1 --2- 172.21.15.16:0/4002870634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45d00cb6d0 0x7f45d015ff90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009960 tx=0x7f45c8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e9fe6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38d4005380 0x7f38dc0359f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e9fe6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc0d5020 0x7f38dc0372e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f054bfff640 1 --2- 172.21.15.16:0/1031242014 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f053806af40 0x7f053806d400 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0540036000 tx=0x7f054003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e9fe6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc0ddf60 0x7f38dc0de410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e9fe6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc0de950 con 0x7f38dc0d5020 2023-12-08T08:38:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e9fe6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc0d5a70 con 0x7f38dc0ddf60 2023-12-08T08:38:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e9fe6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc037b80 con 0x7f38d4005380 2023-12-08T08:38:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc0d5020 0x7f38dc0372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc0d5020 0x7f38dc0372e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52598/0 (socket says 172.21.15.16:52598) 2023-12-08T08:38:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e3fff640 1 -- 172.21.15.16:0/2176947362 learned_addr learned my addr 172.21.15.16:0/2176947362 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e8fe4640 1 --2- 172.21.15.16:0/2176947362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38d4005380 0x7f38dc0359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e97e5640 1 --2- 172.21.15.16:0/2176947362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc0ddf60 0x7f38dc0de410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f37a2ffd640 1 -- 172.21.15.16:0/3324583284 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f37a40113f0 con 0x7f37b40cd240 2023-12-08T08:38:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f23dbfff640 1 -- 172.21.15.16:0/97281473 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f23e403b040 con 0x7f23f406e110 2023-12-08T08:38:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e3fff640 1 -- 172.21.15.16:0/2176947362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38d4005380 msgr2=0x7f38dc0359f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e3fff640 1 --2- 172.21.15.16:0/2176947362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38d4005380 0x7f38dc0359f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e3fff640 1 -- 172.21.15.16:0/2176947362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc0ddf60 msgr2=0x7f38dc0de410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e3fff640 1 --2- 172.21.15.16:0/2176947362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc0ddf60 0x7f38dc0de410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.639+0000 7f38e3fff640 1 -- 172.21.15.16:0/2176947362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc037e60 con 0x7f38dc0d5020 2023-12-08T08:38:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.640+0000 7f38e3fff640 1 --2- 172.21.15.16:0/2176947362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc0d5020 0x7f38dc0372e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f38d0007310 tx=0x7f38d00077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.640+0000 7f23dbfff640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f23e006ae90 0x7f23e006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.640+0000 7f37a2ffd640 1 --2- 172.21.15.16:0/3324583284 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f379406ae90 0x7f379406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.640+0000 7f23dbfff640 1 -- 172.21.15.16:0/97281473 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f23e40abad0 con 0x7f23f406e110 2023-12-08T08:38:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.640+0000 7f23f27fc640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f23e006ae90 0x7f23e006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.640+0000 7f37a2ffd640 1 -- 172.21.15.16:0/3324583284 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f37a4089830 con 0x7f37b40cd240 2023-12-08T08:38:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.640+0000 7f37b1d9b640 1 --2- 172.21.15.16:0/3324583284 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f379406ae90 0x7f379406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.641+0000 7f37b1d9b640 1 --2- 172.21.15.16:0/3324583284 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f379406ae90 0x7f379406d350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f37b40c4810 tx=0x7f37a8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.641+0000 7f23f27fc640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f23e006ae90 0x7f23e006d350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f23f415ba70 tx=0x7f23e8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.641+0000 7f36a1ffb640 1 -- 172.21.15.16:0/349354641 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36a8055090 con 0x7f36ac160a00 2023-12-08T08:38:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.643+0000 7f45beffd640 1 -- 172.21.15.16:0/4002870634 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8039c10 con 0x7f45d00cb6d0 2023-12-08T08:38:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.643+0000 7fe04ffff640 1 -- 172.21.15.16:0/1840069295 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe05800cda0 con 0x7fe06806e0d0 2023-12-08T08:38:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.643+0000 7fe04ffff640 1 -- 172.21.15.16:0/1840069295 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe05800f040 con 0x7fe06806e0d0 2023-12-08T08:38:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.643+0000 7fe06cc13640 1 -- 172.21.15.16:0/1840069295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe068147cd0 con 0x7fe06806e0d0 2023-12-08T08:38:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.643+0000 7f45beffd640 1 -- 172.21.15.16:0/4002870634 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c8039d70 con 0x7f45d00cb6d0 2023-12-08T08:38:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.643+0000 7f45d4c87640 1 -- 172.21.15.16:0/4002870634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d015b800 con 0x7f45d00cb6d0 2023-12-08T08:38:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.643+0000 7f71e17fa640 1 -- 172.21.15.16:0/2262180625 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8030af0 con 0x7f71dc033ef0 2023-12-08T08:38:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.643+0000 7f23dbfff640 1 -- 172.21.15.16:0/97281473 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23e4076b20 con 0x7f23f406e110 2023-12-08T08:38:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.644+0000 7f71eade3640 1 -- 172.21.15.16:0/2262180625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71dc049930 con 0x7f71dc033ef0 2023-12-08T08:38:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.644+0000 7f71e17fa640 1 -- 172.21.15.16:0/2262180625 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d8043460 con 0x7f71dc033ef0 2023-12-08T08:38:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.644+0000 7f71eade3640 1 -- 172.21.15.16:0/2262180625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71dc049000 con 0x7f71dc033ef0 2023-12-08T08:38:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.644+0000 7f71e17fa640 1 -- 172.21.15.16:0/2262180625 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8043810 con 0x7f71dc033ef0 2023-12-08T08:38:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.644+0000 7fe06cc13640 1 -- 172.21.15.16:0/1840069295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe068148290 con 0x7fe06806e0d0 2023-12-08T08:38:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.644+0000 7f45d4c87640 1 -- 172.21.15.16:0/4002870634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d015bdc0 con 0x7f45d00cb6d0 2023-12-08T08:38:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.645+0000 7f45beffd640 1 -- 172.21.15.16:0/4002870634 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8043620 con 0x7f45d00cb6d0 2023-12-08T08:38:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.645+0000 7fe04ffff640 1 -- 172.21.15.16:0/1840069295 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0580146a0 con 0x7fe06806e0d0 2023-12-08T08:38:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.646+0000 7fe04dffb640 1 -- 172.21.15.16:0/1840069295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0680673c0 con 0x7fe06806e0d0 2023-12-08T08:38:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.645+0000 7f71eade3640 1 -- 172.21.15.16:0/2262180625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71dc04ccd0 con 0x7f71dc033ef0 2023-12-08T08:38:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.646+0000 7fe04ffff640 1 -- 172.21.15.16:0/1840069295 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe058014800 con 0x7fe06806e0d0 2023-12-08T08:38:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.646+0000 7f38e17fa640 1 -- 172.21.15.16:0/2176947362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d000f9c0 con 0x7f38dc0d5020 2023-12-08T08:38:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.646+0000 7f38e17fa640 1 -- 172.21.15.16:0/2176947362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d000fb40 con 0x7f38dc0d5020 2023-12-08T08:38:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.646+0000 7f38e17fa640 1 -- 172.21.15.16:0/2176947362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d00133f0 con 0x7f38dc0d5020 2023-12-08T08:38:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.646+0000 7f45d4c87640 1 -- 172.21.15.16:0/4002870634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d015c2e0 con 0x7f45d00cb6d0 2023-12-08T08:38:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.646+0000 7f37a2ffd640 1 -- 172.21.15.16:0/3324583284 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f37a4054780 con 0x7f37b40cd240 2023-12-08T08:38:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.646+0000 7f05497fa640 1 -- 172.21.15.16:0/1031242014 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0544055690 con 0x7f054c06cb80 2023-12-08T08:38:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.646+0000 7f38e9fe6640 1 -- 172.21.15.16:0/2176947362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38dc0d5cc0 con 0x7f38dc0d5020 2023-12-08T08:38:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.646+0000 7f38e9fe6640 1 -- 172.21.15.16:0/2176947362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38dc0d5480 con 0x7f38dc0d5020 2023-12-08T08:38:44.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.646+0000 7f45beffd640 1 -- 172.21.15.16:0/4002870634 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45c80437c0 con 0x7f45d00cb6d0 2023-12-08T08:38:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.647+0000 7f45beffd640 1 --2- 172.21.15.16:0/4002870634 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f45b006ae70 0x7f45b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.647+0000 7fe04ffff640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe05406ae70 0x7fe05406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.647+0000 7fe04ffff640 1 -- 172.21.15.16:0/1840069295 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe058089f90 con 0x7fe06806e0d0 2023-12-08T08:38:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.648+0000 7fe066ffd640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe05406ae70 0x7fe05406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.648+0000 7f45beffd640 1 -- 172.21.15.16:0/4002870634 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f45c803e070 con 0x7f45d00cb6d0 2023-12-08T08:38:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.648+0000 7f45ce59c640 1 --2- 172.21.15.16:0/4002870634 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f45b006ae70 0x7f45b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.648+0000 7f71e17fa640 1 -- 172.21.15.16:0/2262180625 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f71d80423c0 con 0x7f71dc033ef0 2023-12-08T08:38:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.648+0000 7f38e9fe6640 1 -- 172.21.15.16:0/2176947362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38dc006860 con 0x7f38dc0d5020 2023-12-08T08:38:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.648+0000 7f38e17fa640 1 -- 172.21.15.16:0/2176947362 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f38d0013550 con 0x7f38dc0d5020 2023-12-08T08:38:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.649+0000 7f71e17fa640 1 --2- 172.21.15.16:0/2262180625 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f71c806af10 0x7f71c806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.649+0000 7f38e17fa640 1 --2- 172.21.15.16:0/2176947362 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f38c806aec0 0x7f38c806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.649+0000 7f38e17fa640 1 -- 172.21.15.16:0/2176947362 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f38d00899d0 con 0x7f38dc0d5020 2023-12-08T08:38:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.649+0000 7f71e17fa640 1 -- 172.21.15.16:0/2262180625 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f71d80abd00 con 0x7f71dc033ef0 2023-12-08T08:38:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.649+0000 7f45ce59c640 1 --2- 172.21.15.16:0/4002870634 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f45b006ae70 0x7f45b006d330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009aa0 tx=0x7f45c4009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.649+0000 7f38e8fe4640 1 --2- 172.21.15.16:0/2176947362 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f38c806aec0 0x7f38c806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.649+0000 7f71e3fff640 1 --2- 172.21.15.16:0/2262180625 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f71c806af10 0x7f71c806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.650+0000 7fe066ffd640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe05406ae70 0x7fe05406d330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe068146dd0 tx=0x7fe05c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.650+0000 7f38e8fe4640 1 --2- 172.21.15.16:0/2176947362 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f38c806aec0 0x7f38c806d380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f38e4067a70 tx=0x7f38e408c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.650+0000 7f71e3fff640 1 --2- 172.21.15.16:0/2262180625 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f71c806af10 0x7f71c806d3d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f71d0006fd0 tx=0x7f71d0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.652+0000 7fe04ffff640 1 -- 172.21.15.16:0/1840069295 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe058054db0 con 0x7fe06806e0d0 2023-12-08T08:38:44.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.655+0000 7f38e17fa640 1 -- 172.21.15.16:0/2176947362 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38d0054890 con 0x7f38dc0d5020 2023-12-08T08:38:44.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.656+0000 7f71e17fa640 1 -- 172.21.15.16:0/2262180625 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f71dc04ccd0 con 0x7f71dc033ef0 2023-12-08T08:38:44.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:44.660+0000 7f45beffd640 1 -- 172.21.15.16:0/4002870634 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45d015c2e0 con 0x7f45d00cb6d0 2023-12-08T08:38:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.050+0000 7f36b398a640 1 -- 172.21.15.16:0/349354641 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f36ac067ad0 con 0x7f36ac160a00 2023-12-08T08:38:45.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.051+0000 7f36a1ffb640 1 -- 172.21.15.16:0/349354641 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f36a8055090 con 0x7f36ac160a00 2023-12-08T08:38:45.053 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-12-08T08:38:45.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.054+0000 7f368f7fe640 1 -- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f369006ae90 msgr2=0x7f369006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.054+0000 7f368f7fe640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f369006ae90 0x7f369006d350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f36ac15c9f0 tx=0x7f36a401a580 comp rx=0 tx=0).stop 2023-12-08T08:38:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.054+0000 7f368f7fe640 1 -- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36ac160a00 msgr2=0x7f36ac205360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.054+0000 7f368f7fe640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36ac160a00 0x7f36ac205360 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f36a800c3c0 tx=0x7f36a800c880 comp rx=0 tx=0).stop 2023-12-08T08:38:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.054+0000 7f368f7fe640 1 -- 172.21.15.16:0/349354641 shutdown_connections 2023-12-08T08:38:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.054+0000 7f368f7fe640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f369006ae90 0x7f369006d350 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.055+0000 7f368f7fe640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f36ac160a00 0x7f36ac205360 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.055+0000 7f368f7fe640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36ac0cfe10 0x7f36ac0ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.055+0000 7f368f7fe640 1 --2- 172.21.15.16:0/349354641 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f36ac0b1ea0 0x7f36ac204a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.055+0000 7f368f7fe640 1 -- 172.21.15.16:0/349354641 >> 172.21.15.16:0/349354641 conn(0x7f36ac09c300 msgr2=0x7f36ac06b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.055+0000 7f368f7fe640 1 -- 172.21.15.16:0/349354641 shutdown_connections 2023-12-08T08:38:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.055+0000 7f368f7fe640 1 -- 172.21.15.16:0/349354641 wait complete. 2023-12-08T08:38:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.059+0000 7f0552fe5640 1 -- 172.21.15.16:0/1031242014 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f054c0b3840 con 0x7f054c06cb80 2023-12-08T08:38:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.059+0000 7f05497fa640 1 -- 172.21.15.16:0/1031242014 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0544018090 con 0x7f054c06cb80 2023-12-08T08:38:45.062 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-12-08T08:38:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.063+0000 7f0532ffd640 1 -- 172.21.15.16:0/1031242014 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f053806af40 msgr2=0x7f053806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.063+0000 7f0532ffd640 1 --2- 172.21.15.16:0/1031242014 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f053806af40 0x7f053806d400 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0540036000 tx=0x7f054003b040 comp rx=0 tx=0).stop 2023-12-08T08:38:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.063+0000 7f0532ffd640 1 -- 172.21.15.16:0/1031242014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f054c06cb80 msgr2=0x7f054c145030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.063+0000 7f0532ffd640 1 --2- 172.21.15.16:0/1031242014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f054c06cb80 0x7f054c145030 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f054400efc0 tx=0x7f054400c490 comp rx=0 tx=0).stop 2023-12-08T08:38:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.063+0000 7f0532ffd640 1 -- 172.21.15.16:0/1031242014 shutdown_connections 2023-12-08T08:38:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.063+0000 7f0532ffd640 1 --2- 172.21.15.16:0/1031242014 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f053806af40 0x7f053806d400 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.063+0000 7f0532ffd640 1 --2- 172.21.15.16:0/1031242014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054c0cf560 0x7f054c13e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.063+0000 7f0532ffd640 1 --2- 172.21.15.16:0/1031242014 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f054c0b2b60 0x7f054c13e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.063+0000 7f0532ffd640 1 --2- 172.21.15.16:0/1031242014 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f054c06cb80 0x7f054c145030 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.064+0000 7f0532ffd640 1 -- 172.21.15.16:0/1031242014 >> 172.21.15.16:0/1031242014 conn(0x7f054c0cea40 msgr2=0x7f054c0afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.064+0000 7f0532ffd640 1 -- 172.21.15.16:0/1031242014 shutdown_connections 2023-12-08T08:38:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.064+0000 7f0532ffd640 1 -- 172.21.15.16:0/1031242014 wait complete. 2023-12-08T08:38:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.065+0000 7fe04dffb640 1 -- 172.21.15.16:0/1840069295 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe06813e6c0 con 0x7fe06806e0d0 2023-12-08T08:38:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.066+0000 7fe04ffff640 1 -- 172.21.15.16:0/1840069295 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe0580583d0 con 0x7fe06806e0d0 2023-12-08T08:38:45.070 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-12-08T08:38:45.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7fe06cc13640 1 -- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe05406ae70 msgr2=0x7fe05406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7fe06cc13640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe05406ae70 0x7fe05406d330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe068146dd0 tx=0x7fe05c009040 comp rx=0 tx=0).stop 2023-12-08T08:38:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7f37b8ee7640 1 -- 172.21.15.16:0/3324583284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f37b40b11f0 con 0x7f37b40cd240 2023-12-08T08:38:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7fe06cc13640 1 -- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe06806e0d0 msgr2=0x7fe0681487b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7fe06cc13640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe06806e0d0 0x7fe0681487b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe05800efc0 tx=0x7fe05800c490 comp rx=0 tx=0).stop 2023-12-08T08:38:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7f23f99ec640 1 -- 172.21.15.16:0/97281473 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f23f415ac80 con 0x7f23f406e110 2023-12-08T08:38:45.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7fe06cc13640 1 -- 172.21.15.16:0/1840069295 shutdown_connections 2023-12-08T08:38:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7fe06cc13640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe05406ae70 0x7fe05406d330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7fe06cc13640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0680ced50 0x7fe0680cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7fe06cc13640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe068071a90 0x7fe068169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7fe06cc13640 1 --2- 172.21.15.16:0/1840069295 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe06806e0d0 0x7fe0681487b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7fe06cc13640 1 -- 172.21.15.16:0/1840069295 >> 172.21.15.16:0/1840069295 conn(0x7fe06806a6b0 msgr2=0x7fe06813dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7fe06cc13640 1 -- 172.21.15.16:0/1840069295 shutdown_connections 2023-12-08T08:38:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7f37a2ffd640 1 -- 172.21.15.16:0/3324583284 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f37a401f020 con 0x7f37b40cd240 2023-12-08T08:38:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7fe06cc13640 1 -- 172.21.15.16:0/1840069295 wait complete. 2023-12-08T08:38:45.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.070+0000 7f23dbfff640 1 -- 172.21.15.16:0/97281473 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f23f415ac80 con 0x7f23f406e110 2023-12-08T08:38:45.076 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-12-08T08:38:45.076 INFO:teuthology.orchestra.run.smithi016.stdout:55834574858 2023-12-08T08:38:45.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2023-12-08T08:38:45.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.073+0000 7f38e9fe6640 1 -- 172.21.15.16:0/2176947362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f38dc0d61e0 con 0x7f38dc0d5020 2023-12-08T08:38:45.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f71eade3640 1 -- 172.21.15.16:0/2262180625 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f71dc04ccd0 con 0x7f71dc033ef0 2023-12-08T08:38:45.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f37b8ee7640 1 -- 172.21.15.16:0/3324583284 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f379406ae90 msgr2=0x7f379406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f37b8ee7640 1 --2- 172.21.15.16:0/3324583284 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f379406ae90 0x7f379406d350 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f37b40c4810 tx=0x7f37a8008040 comp rx=0 tx=0).stop 2023-12-08T08:38:45.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f38e17fa640 1 -- 172.21.15.16:0/2176947362 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f38dc0d61e0 con 0x7f38dc0d5020 2023-12-08T08:38:45.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f37b8ee7640 1 -- 172.21.15.16:0/3324583284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40cd240 msgr2=0x7f37b40b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f37b8ee7640 1 --2- 172.21.15.16:0/3324583284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40cd240 0x7f37b40b21c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f37a400c830 tx=0x7f37a400ccf0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.079 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-12-08T08:38:45.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f71e17fa640 1 -- 172.21.15.16:0/2262180625 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f71d8048020 con 0x7f71dc033ef0 2023-12-08T08:38:45.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f37b8ee7640 1 -- 172.21.15.16:0/3324583284 shutdown_connections 2023-12-08T08:38:45.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f37b8ee7640 1 --2- 172.21.15.16:0/3324583284 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f379406ae90 0x7f379406d350 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f37b8ee7640 1 --2- 172.21.15.16:0/3324583284 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f37b415f310 0x7f37b40b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f37b8ee7640 1 --2- 172.21.15.16:0/3324583284 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f37b40cf730 0x7f37b40b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f37b8ee7640 1 --2- 172.21.15.16:0/3324583284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b40cd240 0x7f37b40b21c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f37b8ee7640 1 -- 172.21.15.16:0/3324583284 >> 172.21.15.16:0/3324583284 conn(0x7f37b4099a10 msgr2=0x7f37b40cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:45.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f37b8ee7640 1 -- 172.21.15.16:0/3324583284 shutdown_connections 2023-12-08T08:38:45.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f23f99ec640 1 -- 172.21.15.16:0/97281473 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f23e006ae90 msgr2=0x7f23e006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f23f99ec640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f23e006ae90 0x7f23e006d350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f23f415ba70 tx=0x7f23e8008040 comp rx=0 tx=0).stop 2023-12-08T08:38:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f23f99ec640 1 -- 172.21.15.16:0/97281473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f406e110 msgr2=0x7f23f4137680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f23f99ec640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f406e110 0x7f23f4137680 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23e402b830 tx=0x7f23e40046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f23f99ec640 1 -- 172.21.15.16:0/97281473 shutdown_connections 2023-12-08T08:38:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f23f99ec640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f23e006ae90 0x7f23e006d350 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f23f99ec640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23f40b2060 0x7f23f406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f23f99ec640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f23f4070230 0x7f23f406c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f23f99ec640 1 --2- 172.21.15.16:0/97281473 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f23f406e110 0x7f23f4137680 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23e402b830 tx=0x7f23e40046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f23f99ec640 1 -- 172.21.15.16:0/97281473 >> 172.21.15.16:0/97281473 conn(0x7f23f40b0750 msgr2=0x7f23f40cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f23f99ec640 1 -- 172.21.15.16:0/97281473 shutdown_connections 2023-12-08T08:38:45.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f23f99ec640 1 -- 172.21.15.16:0/97281473 wait complete. 2023-12-08T08:38:45.081 INFO:teuthology.orchestra.run.smithi016.stdout:55834574856 2023-12-08T08:38:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.074+0000 7f37b8ee7640 1 -- 172.21.15.16:0/3324583284 wait complete. 2023-12-08T08:38:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.075+0000 7f45d4c87640 1 -- 172.21.15.16:0/4002870634 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f45d0067430 con 0x7f45d00cb6d0 2023-12-08T08:38:45.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.076+0000 7f45beffd640 1 -- 172.21.15.16:0/4002870634 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f45d015c2e0 con 0x7f45d00cb6d0 2023-12-08T08:38:45.082 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-12-08T08:38:45.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.078+0000 7f71eade3640 1 -- 172.21.15.16:0/2262180625 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f71c806af10 msgr2=0x7f71c806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.078+0000 7f71eade3640 1 --2- 172.21.15.16:0/2262180625 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f71c806af10 0x7f71c806d3d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f71d0006fd0 tx=0x7f71d0008040 comp rx=0 tx=0).stop 2023-12-08T08:38:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.078+0000 7f71eade3640 1 -- 172.21.15.16:0/2262180625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc033ef0 msgr2=0x7f71dc0d92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.078+0000 7f71eade3640 1 --2- 172.21.15.16:0/2262180625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc033ef0 0x7f71dc0d92b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f71d8037000 tx=0x7f71d80046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.078+0000 7f71eade3640 1 -- 172.21.15.16:0/2262180625 shutdown_connections 2023-12-08T08:38:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.078+0000 7f71eade3640 1 --2- 172.21.15.16:0/2262180625 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f71c806af10 0x7f71c806d3d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.078+0000 7f71eade3640 1 --2- 172.21.15.16:0/2262180625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71dc0df220 0x7f71dc04a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.078+0000 7f71eade3640 1 --2- 172.21.15.16:0/2262180625 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71dc053c70 0x7f71dc04a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.078+0000 7f71eade3640 1 --2- 172.21.15.16:0/2262180625 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71dc033ef0 0x7f71dc0d92b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.078+0000 7f71eade3640 1 -- 172.21.15.16:0/2262180625 >> 172.21.15.16:0/2262180625 conn(0x7f71dc009fb0 msgr2=0x7f71dc0e0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:45.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.078+0000 7f38e9fe6640 1 -- 172.21.15.16:0/2176947362 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f38c806aec0 msgr2=0x7f38c806d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.078+0000 7f38e9fe6640 1 --2- 172.21.15.16:0/2176947362 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f38c806aec0 0x7f38c806d380 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f38e4067a70 tx=0x7f38e408c040 comp rx=0 tx=0).stop 2023-12-08T08:38:45.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.078+0000 7f71eade3640 1 -- 172.21.15.16:0/2262180625 shutdown_connections 2023-12-08T08:38:45.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.079+0000 7f71eade3640 1 -- 172.21.15.16:0/2262180625 wait complete. 2023-12-08T08:38:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.079+0000 7f38e9fe6640 1 -- 172.21.15.16:0/2176947362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc0d5020 msgr2=0x7f38dc0372e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.079+0000 7f38e9fe6640 1 --2- 172.21.15.16:0/2176947362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc0d5020 0x7f38dc0372e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f38d0007310 tx=0x7f38d00077d0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.079+0000 7f38e9fe6640 1 -- 172.21.15.16:0/2176947362 shutdown_connections 2023-12-08T08:38:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.079+0000 7f38e9fe6640 1 --2- 172.21.15.16:0/2176947362 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f38c806aec0 0x7f38c806d380 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.079+0000 7f38e9fe6640 1 --2- 172.21.15.16:0/2176947362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38dc0ddf60 0x7f38dc0de410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.079+0000 7f38e9fe6640 1 --2- 172.21.15.16:0/2176947362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc0d5020 0x7f38dc0372e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.079+0000 7f38e9fe6640 1 --2- 172.21.15.16:0/2176947362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38d4005380 0x7f38dc0359f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.079+0000 7f38e9fe6640 1 -- 172.21.15.16:0/2176947362 >> 172.21.15.16:0/2176947362 conn(0x7f38dc055b90 msgr2=0x7f38dc006f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.079+0000 7f38e9fe6640 1 -- 172.21.15.16:0/2176947362 shutdown_connections 2023-12-08T08:38:45.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.079+0000 7f38e9fe6640 1 -- 172.21.15.16:0/2176947362 wait complete. 2023-12-08T08:38:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.080+0000 7f45d4c87640 1 -- 172.21.15.16:0/4002870634 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f45b006ae70 msgr2=0x7f45b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.080+0000 7f45d4c87640 1 --2- 172.21.15.16:0/4002870634 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f45b006ae70 0x7f45b006d330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009aa0 tx=0x7f45c4009190 comp rx=0 tx=0).stop 2023-12-08T08:38:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.080+0000 7f45d4c87640 1 -- 172.21.15.16:0/4002870634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45d00cb6d0 msgr2=0x7f45d015ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.080+0000 7f45d4c87640 1 --2- 172.21.15.16:0/4002870634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45d00cb6d0 0x7f45d015ff90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009960 tx=0x7f45c8004730 comp rx=0 tx=0).stop 2023-12-08T08:38:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.080+0000 7f45d4c87640 1 -- 172.21.15.16:0/4002870634 shutdown_connections 2023-12-08T08:38:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.080+0000 7f45d4c87640 1 --2- 172.21.15.16:0/4002870634 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f45b006ae70 0x7f45b006d330 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.080+0000 7f45d4c87640 1 --2- 172.21.15.16:0/4002870634 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f45d00cb6d0 0x7f45d015ff90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.080+0000 7f45d4c87640 1 --2- 172.21.15.16:0/4002870634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45d00ae2b0 0x7f45d015fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.080+0000 7f45d4c87640 1 --2- 172.21.15.16:0/4002870634 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f45d006c0c0 0x7f45d01545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.080+0000 7f45d4c87640 1 -- 172.21.15.16:0/4002870634 >> 172.21.15.16:0/4002870634 conn(0x7f45d00b09f0 msgr2=0x7f45d015aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.080+0000 7f45d4c87640 1 -- 172.21.15.16:0/4002870634 shutdown_connections 2023-12-08T08:38:45.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:45.080+0000 7f45d4c87640 1 -- 172.21.15.16:0/4002870634 wait complete. 2023-12-08T08:38:45.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2023-12-08T08:38:45.097 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2023-12-08T08:38:45.097 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.1 2023-12-08T08:38:45.100 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.2 2023-12-08T08:38:45.100 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2023-12-08T08:38:45.102 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.3 2023-12-08T08:38:45.102 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2023-12-08T08:38:46.078 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-08T08:38:46.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-08T08:38:46.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-08T08:38:46.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-08T08:38:46.100 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-08T08:38:46.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-08T08:38:46.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-08T08:38:46.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-08T08:38:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.254+0000 7fd5656cf640 1 Processor -- start 2023-12-08T08:38:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.254+0000 7fd5656cf640 1 -- start start 2023-12-08T08:38:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.256+0000 7fd5656cf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5600ceed0 0x7fd560159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.256+0000 7fd5656cf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5600cbd50 0x7fd56015a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.256+0000 7fd5656cf640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5600cd490 con 0x7fd560159bc0 2023-12-08T08:38:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.256+0000 7fd5656cf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd56006e100 con 0x7fd5600ceed0 2023-12-08T08:38:46.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.256+0000 7fd5656cf640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56015aa20 con 0x7fd5600cbd50 2023-12-08T08:38:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.256+0000 7fd55f7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5600cbd50 0x7fd56015a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.256+0000 7fd55f7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5600cbd50 0x7fd56015a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58388/0 (socket says 172.21.15.16:58388) 2023-12-08T08:38:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.256+0000 7fd55f7fe640 1 -- 172.21.15.16:0/2681518088 learned_addr learned my addr 172.21.15.16:0/2681518088 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.256+0000 7fd55e7fc640 1 --2- 172.21.15.16:0/2681518088 >> v2:172.21.15.16:3300/0 conn(0x7fd5600ceed0 0x7fd560159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.257+0000 7fd55f7fe640 1 -- 172.21.15.16:0/2681518088 >> v2:172.21.15.16:3300/0 conn(0x7fd5600ceed0 msgr2=0x7fd560159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.257+0000 7fd55f7fe640 1 --2- 172.21.15.16:0/2681518088 >> v2:172.21.15.16:3300/0 conn(0x7fd5600ceed0 0x7fd560159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.257+0000 7fd55f7fe640 1 -- 172.21.15.16:0/2681518088 >> v1:172.21.15.16:6789/0 conn(0x7fd560159bc0 legacy=0x7fd56006d1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-08T08:38:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.257+0000 7fd55f7fe640 1 -- 172.21.15.16:0/2681518088 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd56006d8b0 con 0x7fd5600cbd50 2023-12-08T08:38:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.258+0000 7fd55e7fc640 1 --2- 172.21.15.16:0/2681518088 >> v2:172.21.15.16:3300/0 conn(0x7fd5600ceed0 0x7fd560159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.259+0000 7fd55f7fe640 1 --2- 172.21.15.16:0/2681518088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5600cbd50 0x7fd56015a4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd558009e50 tx=0x7fd55802bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d367d40f3d92977 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.260+0000 7fd55dffb640 1 -- 172.21.15.16:0/2681518088 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55802caa0 con 0x7fd5600cbd50 2023-12-08T08:38:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.260+0000 7fd55dffb640 1 -- 172.21.15.16:0/2681518088 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55802cc00 con 0x7fd5600cbd50 2023-12-08T08:38:46.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.261+0000 7fd5656cf640 1 -- 172.21.15.16:0/2681518088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5600cbd50 msgr2=0x7fd56015a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.261+0000 7fd5656cf640 1 --2- 172.21.15.16:0/2681518088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5600cbd50 0x7fd56015a4e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd558009e50 tx=0x7fd55802bf00 comp rx=0 tx=0).stop 2023-12-08T08:38:46.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.261+0000 7fd5656cf640 1 -- 172.21.15.16:0/2681518088 shutdown_connections 2023-12-08T08:38:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.261+0000 7fd5656cf640 1 --2- 172.21.15.16:0/2681518088 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5600cbd50 0x7fd56015a4e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.261+0000 7fd5656cf640 1 --2- 172.21.15.16:0/2681518088 >> v2:172.21.15.16:3300/0 conn(0x7fd5600ceed0 0x7fd560159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.261+0000 7fd5656cf640 1 -- 172.21.15.16:0/2681518088 >> 172.21.15.16:0/2681518088 conn(0x7fd560071860 msgr2=0x7fd56006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.262+0000 7fd5656cf640 1 -- 172.21.15.16:0/2681518088 shutdown_connections 2023-12-08T08:38:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.262+0000 7fd5656cf640 1 -- 172.21.15.16:0/2681518088 wait complete. 2023-12-08T08:38:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.264+0000 7fd5656cf640 1 Processor -- start 2023-12-08T08:38:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.264+0000 7fd5656cf640 1 -- start start 2023-12-08T08:38:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.266+0000 7fd5656cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5600cbd50 0x7fd56014d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.266+0000 7fd55effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5600cbd50 0x7fd56014d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.266+0000 7fd55effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5600cbd50 0x7fd56014d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52620/0 (socket says 172.21.15.16:52620) 2023-12-08T08:38:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.266+0000 7fd55effd640 1 -- 172.21.15.16:0/3410705471 learned_addr learned my addr 172.21.15.16:0/3410705471 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.266+0000 7fd5656cf640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5600ceed0 0x7fd5601657d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.267+0000 7fd5656cf640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd560159bc0 0x7fd56014e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.267+0000 7fd5656cf640 1 -- 172.21.15.16:0/3410705471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56015b1e0 con 0x7fd5600cbd50 2023-12-08T08:38:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.267+0000 7fd5656cf640 1 -- 172.21.15.16:0/3410705471 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56015b4c0 con 0x7fd560159bc0 2023-12-08T08:38:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.267+0000 7fd55f7fe640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd560159bc0 0x7fd56014e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.267+0000 7fd5656cf640 1 -- 172.21.15.16:0/3410705471 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd56015b7a0 con 0x7fd5600ceed0 2023-12-08T08:38:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.268+0000 7fd55effd640 1 -- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5600ceed0 msgr2=0x7fd5601657d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-08T08:38:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.268+0000 7fd55effd640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5600ceed0 0x7fd5601657d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.268+0000 7fd55effd640 1 -- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd560159bc0 msgr2=0x7fd56014e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.268+0000 7fd55effd640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd560159bc0 0x7fd56014e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.268+0000 7fd55effd640 1 -- 172.21.15.16:0/3410705471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd550009400 con 0x7fd5600cbd50 2023-12-08T08:38:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.269+0000 7fd55effd640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5600cbd50 0x7fd56014d370 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd55400b6d0 tx=0x7fd55400bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.269+0000 7fd5477fe640 1 -- 172.21.15.16:0/3410705471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55400c720 con 0x7fd5600cbd50 2023-12-08T08:38:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.269+0000 7fd5656cf640 1 -- 172.21.15.16:0/3410705471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd558009b30 con 0x7fd5600cbd50 2023-12-08T08:38:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.269+0000 7fd5656cf640 1 -- 172.21.15.16:0/3410705471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56015bd60 con 0x7fd5600cbd50 2023-12-08T08:38:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.273+0000 7fd5477fe640 1 -- 172.21.15.16:0/3410705471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55400c8a0 con 0x7fd5600cbd50 2023-12-08T08:38:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.274+0000 7fd5477fe640 1 -- 172.21.15.16:0/3410705471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55400cc50 con 0x7fd5600cbd50 2023-12-08T08:38:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.275+0000 7fd5477fe640 1 -- 172.21.15.16:0/3410705471 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd55401d3b0 con 0x7fd5600cbd50 2023-12-08T08:38:46.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.275+0000 7fd5477fe640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd54806b130 0x7fd54806d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.276+0000 7fd5477fe640 1 -- 172.21.15.16:0/3410705471 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd554089200 con 0x7fd5600cbd50 2023-12-08T08:38:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.276+0000 7fd5656cf640 1 -- 172.21.15.16:0/3410705471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd540005250 con 0x7fd5600cbd50 2023-12-08T08:38:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.277+0000 7fd55e7fc640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd54806b130 0x7fd54806d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.286+0000 7fd5477fe640 1 -- 172.21.15.16:0/3410705471 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd554054050 con 0x7fd5600cbd50 2023-12-08T08:38:46.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.294+0000 7fd55e7fc640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd54806b130 0x7fd54806d5f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd560072fc0 tx=0x7fd550009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.308+0000 7f216f0b2640 1 Processor -- start 2023-12-08T08:38:46.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.308+0000 7f216f0b2640 1 -- start start 2023-12-08T08:38:46.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.309+0000 7f216f0b2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f216806d530 0x7f216806d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.309+0000 7f216f0b2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21680b31a0 0x7f216806c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.309+0000 7f216f0b2640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21681595a0 0x7f216806c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.309+0000 7f216f0b2640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216815a860 con 0x7f21681595a0 2023-12-08T08:38:46.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.309+0000 7f216f0b2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216806de50 con 0x7f216806d530 2023-12-08T08:38:46.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.309+0000 7f216f0b2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2168070190 con 0x7f21680b31a0 2023-12-08T08:38:46.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.310+0000 7f216d64f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21681595a0 0x7f216806c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.310+0000 7f216d64f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21681595a0 0x7f216806c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52630/0 (socket says 172.21.15.16:52630) 2023-12-08T08:38:46.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.310+0000 7f216d64f640 1 -- 172.21.15.16:0/1969404037 learned_addr learned my addr 172.21.15.16:0/1969404037 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.310+0000 7f2167fff640 1 --2- 172.21.15.16:0/1969404037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21680b31a0 0x7f216806c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.310+0000 7f216d64f640 1 -- 172.21.15.16:0/1969404037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21680b31a0 msgr2=0x7f216806c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:46.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.310+0000 7f216d64f640 1 --2- 172.21.15.16:0/1969404037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21680b31a0 0x7f216806c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.310+0000 7f216d64f640 1 -- 172.21.15.16:0/1969404037 >> v2:172.21.15.112:3300/0 conn(0x7f216806d530 msgr2=0x7f216806d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:46.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.310+0000 7f216ce4e640 1 --2- 172.21.15.16:0/1969404037 >> v2:172.21.15.112:3300/0 conn(0x7f216806d530 0x7f216806d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.310+0000 7f216d64f640 1 --2- 172.21.15.16:0/1969404037 >> v2:172.21.15.112:3300/0 conn(0x7f216806d530 0x7f216806d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.310+0000 7f216d64f640 1 -- 172.21.15.16:0/1969404037 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168159aa0 con 0x7f21681595a0 2023-12-08T08:38:46.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.310+0000 7f216ce4e640 1 --2- 172.21.15.16:0/1969404037 >> v2:172.21.15.112:3300/0 conn(0x7f216806d530 0x7f216806d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.311+0000 7fe04ca1a640 1 Processor -- start 2023-12-08T08:38:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.311+0000 7feaf959e640 1 Processor -- start 2023-12-08T08:38:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.311+0000 7ff4c759e640 1 Processor -- start 2023-12-08T08:38:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.312+0000 7f216d64f640 1 --2- 172.21.15.16:0/1969404037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21681595a0 0x7f216806c600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f216806bcd0 tx=0x7f215c02c190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c58ed1db322997 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.312+0000 7ff4c759e640 1 -- start start 2023-12-08T08:38:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.311+0000 7fe04ca1a640 1 -- start start 2023-12-08T08:38:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.312+0000 7fd58698a640 1 Processor -- start 2023-12-08T08:38:46.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.312+0000 7fe04ca1a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe04000aa70 0x7fe04004fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.312+0000 7feaf959e640 1 -- start start 2023-12-08T08:38:46.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7fe04ca1a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0400df360 0x7fe04004ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7feaf959e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaf40b3860 0x7feaf4158590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.312+0000 7fd58698a640 1 -- start start 2023-12-08T08:38:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7feaf959e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf40b27c0 con 0x7feaf406da30 2023-12-08T08:38:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7feaf959e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf40b2aa0 con 0x7feaf40b49c0 2023-12-08T08:38:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7feaf959e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf40b3440 con 0x7feaf40b3860 2023-12-08T08:38:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7feaf2ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7feaf406da30 0x7feaf40b42b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59786/0 (socket says 172.21.15.16:59786) 2023-12-08T08:38:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7feaf2ffd640 1 -- 172.21.15.16:0/414104214 learned_addr learned my addr 172.21.15.16:0/414104214 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7fa3f5a56640 1 Processor -- start 2023-12-08T08:38:46.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7fa3f5a56640 1 -- start start 2023-12-08T08:38:46.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7f21677fe640 1 -- 172.21.15.16:0/1969404037 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c034440 con 0x7f21681595a0 2023-12-08T08:38:46.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7ff4c759e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4c0009150 0x7ff4c0052ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7ff4c759e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4c0048c00 0x7ff4c0049020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7ff4c759e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4c0049560 con 0x7ff4c00eb260 2023-12-08T08:38:46.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7ff4c759e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4c0036b50 con 0x7ff4c0048c00 2023-12-08T08:38:46.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7ff4c759e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c0009530 con 0x7ff4c0009150 2023-12-08T08:38:46.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7f21677fe640 1 -- 172.21.15.16:0/1969404037 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c0345a0 con 0x7f21681595a0 2023-12-08T08:38:46.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7ff4c659c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4c0009150 0x7ff4c0052ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7ff4c659c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4c0009150 0x7ff4c0052ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58414/0 (socket says 172.21.15.16:58414) 2023-12-08T08:38:46.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7ff4c659c640 1 -- 172.21.15.16:0/4276944703 learned_addr learned my addr 172.21.15.16:0/4276944703 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7fd58698a640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd58006d580 0x7fd58006c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7fd58698a640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd58006c920 0x7fd5800709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7fd58698a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd580070230 con 0x7fd580161860 2023-12-08T08:38:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7fd58698a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd580161c40 con 0x7fd58006c920 2023-12-08T08:38:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7fd58698a640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5800b39b0 con 0x7fd58006d580 2023-12-08T08:38:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7fd57f7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd58006d580 0x7fd58006c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7fd57f7fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd58006d580 0x7fd58006c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:46208/0 (socket says 172.21.15.16:46208) 2023-12-08T08:38:46.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7fd57f7fe640 1 -- 172.21.15.16:0/2262786358 learned_addr learned my addr 172.21.15.16:0/2262786358 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7feaf27fc640 1 -- 172.21.15.16:0/414104214 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2785874954 0 0) 0x7feaf40b27c0 con 0x7feaf406da30 2023-12-08T08:38:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7feaf37fe640 1 --2- 172.21.15.16:0/414104214 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaf40b3860 0x7feaf4158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7feaf27fc640 1 -- 172.21.15.16:0/414104214 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feae0003520 con 0x7feaf406da30 2023-12-08T08:38:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7ff4c5d9b640 1 --2- 172.21.15.16:0/4276944703 >> v2:172.21.15.16:3300/0 conn(0x7ff4c0048c00 0x7ff4c0049020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7ff4c659c640 1 -- 172.21.15.16:0/4276944703 >> v2:172.21.15.16:3300/0 conn(0x7ff4c0048c00 msgr2=0x7ff4c0049020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:46.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7fe046ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0400df360 0x7fe04004ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7fe04ca1a640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe040006870 con 0x7fe040053230 2023-12-08T08:38:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7fe04ca1a640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe040007740 con 0x7fe0400df360 2023-12-08T08:38:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.313+0000 7fe0477fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe04000aa70 0x7fe04004fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7fe0477fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe04000aa70 0x7fe04004fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:46194/0 (socket says 172.21.15.16:46194) 2023-12-08T08:38:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7fe0477fe640 1 -- 172.21.15.16:0/3711997162 learned_addr learned my addr 172.21.15.16:0/3711997162 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7f21677fe640 1 -- 172.21.15.16:0/1969404037 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c034930 con 0x7f21681595a0 2023-12-08T08:38:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7f216f0b2640 1 -- 172.21.15.16:0/1969404037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21681595a0 msgr2=0x7f216806c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7f216f0b2640 1 --2- 172.21.15.16:0/1969404037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21681595a0 0x7f216806c600 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f216806bcd0 tx=0x7f215c02c190 comp rx=0 tx=0).stop 2023-12-08T08:38:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7f216f0b2640 1 -- 172.21.15.16:0/1969404037 shutdown_connections 2023-12-08T08:38:46.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7f216f0b2640 1 --2- 172.21.15.16:0/1969404037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21681595a0 0x7f216806c600 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7f216f0b2640 1 --2- 172.21.15.16:0/1969404037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21680b31a0 0x7f216806c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7f216f0b2640 1 --2- 172.21.15.16:0/1969404037 >> v2:172.21.15.112:3300/0 conn(0x7f216806d530 0x7f216806d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7f216f0b2640 1 -- 172.21.15.16:0/1969404037 >> 172.21.15.16:0/1969404037 conn(0x7f21680b09f0 msgr2=0x7f21680b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7f216f0b2640 1 -- 172.21.15.16:0/1969404037 shutdown_connections 2023-12-08T08:38:46.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fd57f7fe640 1 -- 172.21.15.16:0/2262786358 >> v2:172.21.15.16:3300/0 conn(0x7fd58006c920 msgr2=0x7fd5800709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:46.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fd57f7fe640 1 --2- 172.21.15.16:0/2262786358 >> v2:172.21.15.16:3300/0 conn(0x7fd58006c920 0x7fd5800709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fd57f7fe640 1 -- 172.21.15.16:0/2262786358 >> v1:172.21.15.112:6789/0 conn(0x7fd580161860 legacy=0x7fd58006ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:46.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fd57f7fe640 1 -- 172.21.15.16:0/2262786358 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5800b3c90 con 0x7fd58006d580 2023-12-08T08:38:46.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7fa3f5a56640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa3f0160a00 0x7fa3f00d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fa3f5a56640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3f00cffe0 0x7fa3f00b2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fa3f5a56640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f00b28e0 con 0x7fa3f00b1ea0 2023-12-08T08:38:46.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fa3f5a56640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f00b2610 con 0x7fa3f0160a00 2023-12-08T08:38:46.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fa3f5a56640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00ce7c0 con 0x7fa3f00cffe0 2023-12-08T08:38:46.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fa3ef7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3f00cffe0 0x7fa3f00b2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7ff4c659c640 1 --2- 172.21.15.16:0/4276944703 >> v2:172.21.15.16:3300/0 conn(0x7ff4c0048c00 0x7ff4c0049020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7ff4c659c640 1 -- 172.21.15.16:0/4276944703 >> v1:172.21.15.16:6789/0 conn(0x7ff4c00eb260 legacy=0x7ff4c00eb670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-08T08:38:46.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7ff4c659c640 1 -- 172.21.15.16:0/4276944703 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c0035930 con 0x7ff4c0009150 2023-12-08T08:38:46.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.314+0000 7ff4c5d9b640 1 --2- 172.21.15.16:0/4276944703 >> v2:172.21.15.16:3300/0 conn(0x7ff4c0048c00 0x7ff4c0049020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:46.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fe0467fc640 1 -- 172.21.15.16:0/3711997162 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3483564818 0 0) 0x7fe040006870 con 0x7fe040053230 2023-12-08T08:38:46.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fe04ca1a640 1 -- 172.21.15.16:0/3711997162 --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe040006ac0 con 0x7fe04000aa70 2023-12-08T08:38:46.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fe0467fc640 1 -- 172.21.15.16:0/3711997162 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe038007930 con 0x7fe040053230 2023-12-08T08:38:46.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7f216f0b2640 1 -- 172.21.15.16:0/1969404037 wait complete. 2023-12-08T08:38:46.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fa3ef7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3f00cffe0 0x7fa3f00b2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58422/0 (socket says 172.21.15.16:58422) 2023-12-08T08:38:46.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fa3ef7fe640 1 -- 172.21.15.16:0/1218181062 learned_addr learned my addr 172.21.15.16:0/1218181062 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7feaf37fe640 1 -- 172.21.15.16:0/414104214 >> v1:172.21.15.112:6789/0 conn(0x7feaf40b49c0 legacy=0x7feaf4157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7feaf37fe640 1 -- 172.21.15.16:0/414104214 >> v1:172.21.15.16:6789/0 conn(0x7feaf406da30 legacy=0x7feaf40b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7feaf37fe640 1 -- 172.21.15.16:0/414104214 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf40b2d80 con 0x7feaf40b3860 2023-12-08T08:38:46.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fa3e7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa3f00b1ea0 0x7fa3f00cf8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59790/0 (socket says 172.21.15.16:59790) 2023-12-08T08:38:46.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fa3eeffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa3f0160a00 0x7fa3f00d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.316+0000 7fa3ef7fe640 1 -- 172.21.15.16:0/1218181062 >> v2:172.21.15.112:3300/0 conn(0x7fa3f0160a00 msgr2=0x7fa3f00d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:46.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fe0477fe640 1 -- 172.21.15.16:0/3711997162 >> v2:172.21.15.16:3300/0 conn(0x7fe0400df360 msgr2=0x7fe04004ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:46.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fe0477fe640 1 --2- 172.21.15.16:0/3711997162 >> v2:172.21.15.16:3300/0 conn(0x7fe0400df360 0x7fe04004ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.315+0000 7fe0477fe640 1 -- 172.21.15.16:0/3711997162 >> v1:172.21.15.112:6789/0 conn(0x7fe040053230 legacy=0x7fe0400536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.316+0000 7fe0477fe640 1 -- 172.21.15.16:0/3711997162 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe040006da0 con 0x7fe04000aa70 2023-12-08T08:38:46.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.316+0000 7fe046ffd640 1 --2- 172.21.15.16:0/3711997162 >> v2:172.21.15.16:3300/0 conn(0x7fe0400df360 0x7fe04004ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.316+0000 7fa3ef7fe640 1 --2- 172.21.15.16:0/1218181062 >> v2:172.21.15.112:3300/0 conn(0x7fa3f0160a00 0x7fa3f00d3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.316+0000 7fa3ef7fe640 1 -- 172.21.15.16:0/1218181062 >> v1:172.21.15.16:6789/0 conn(0x7fa3f00b1ea0 legacy=0x7fa3f00cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.316+0000 7ff4c659c640 1 --2- 172.21.15.16:0/4276944703 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4c0009150 0x7ff4c0052ec0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009870 tx=0x7ff4b802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb7913d146270e9d server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.316+0000 7fd57f7fe640 1 --2- 172.21.15.16:0/2262786358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd58006d580 0x7fd58006c2d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd58006fe80 tx=0x7fd57402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2a995c68cad370b server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.316+0000 7fa3ef7fe640 1 -- 172.21.15.16:0/1218181062 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f00ceaa0 con 0x7fa3f00cffe0 2023-12-08T08:38:46.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.316+0000 7fa3eeffd640 1 --2- 172.21.15.16:0/1218181062 >> v2:172.21.15.112:3300/0 conn(0x7fa3f0160a00 0x7fa3f00d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:46.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.316+0000 7fd57effd640 1 -- 172.21.15.16:0/2262786358 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574031870 con 0x7fd58006d580 2023-12-08T08:38:46.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.316+0000 7fd57effd640 1 -- 172.21.15.16:0/2262786358 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5740319d0 con 0x7fd58006d580 2023-12-08T08:38:46.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.317+0000 7f216f0b2640 1 Processor -- start 2023-12-08T08:38:46.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.317+0000 7fd57effd640 1 -- 172.21.15.16:0/2262786358 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574031870 con 0x7fd58006d580 2023-12-08T08:38:46.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.317+0000 7feaf37fe640 1 --2- 172.21.15.16:0/414104214 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaf40b3860 0x7feaf4158590 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7feaec009dc0 tx=0x7feaec02bc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d0ec343743268a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.317+0000 7ff4c559a640 1 -- 172.21.15.16:0/4276944703 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8031640 con 0x7ff4c0009150 2023-12-08T08:38:46.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.317+0000 7feaf27fc640 1 -- 172.21.15.16:0/414104214 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec02c980 con 0x7feaf40b3860 2023-12-08T08:38:46.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.317+0000 7feaf27fc640 1 -- 172.21.15.16:0/414104214 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec02cae0 con 0x7feaf40b3860 2023-12-08T08:38:46.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.317+0000 7feaf27fc640 1 -- 172.21.15.16:0/414104214 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec02ce50 con 0x7feaf40b3860 2023-12-08T08:38:46.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.317+0000 7ff4c559a640 1 -- 172.21.15.16:0/4276944703 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b80317a0 con 0x7ff4c0009150 2023-12-08T08:38:46.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7ff4c559a640 1 -- 172.21.15.16:0/4276944703 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8031b30 con 0x7ff4c0009150 2023-12-08T08:38:46.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7fa3ef7fe640 1 --2- 172.21.15.16:0/1218181062 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3f00cffe0 0x7fa3f00b2d70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009e50 tx=0x7fa3e802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2401176ef74f3b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.317+0000 7f216f0b2640 1 -- start start 2023-12-08T08:38:46.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7ff4c759e640 1 -- 172.21.15.16:0/4276944703 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4c0009150 msgr2=0x7ff4c0052ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7feaf959e640 1 -- 172.21.15.16:0/414104214 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaf40b3860 msgr2=0x7feaf4158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7feaf959e640 1 --2- 172.21.15.16:0/414104214 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaf40b3860 0x7feaf4158590 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7feaec009dc0 tx=0x7feaec02bc50 comp rx=0 tx=0).stop 2023-12-08T08:38:46.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7feaf959e640 1 -- 172.21.15.16:0/414104214 shutdown_connections 2023-12-08T08:38:46.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7feaf959e640 1 --2- 172.21.15.16:0/414104214 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaf40b3860 0x7feaf4158590 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7feaf959e640 1 -- 172.21.15.16:0/414104214 >> 172.21.15.16:0/414104214 conn(0x7feaf4071860 msgr2=0x7feaf406a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7feaf959e640 1 -- 172.21.15.16:0/414104214 shutdown_connections 2023-12-08T08:38:46.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7f216f0b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f216806d530 0x7f21681ffb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7feaf959e640 1 -- 172.21.15.16:0/414104214 wait complete. 2023-12-08T08:38:46.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7f216f0b2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21680b31a0 0x7f216809be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7ff4c759e640 1 --2- 172.21.15.16:0/4276944703 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4c0009150 0x7ff4c0052ec0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8009870 tx=0x7ff4b802b8f0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7fe0477fe640 1 --2- 172.21.15.16:0/3711997162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04000aa70 0x7fe04004fa60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe040052980 tx=0x7fe03c02ba60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b23bcc2d1ec6d005 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7f216f0b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21681595a0 0x7f216809c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7f216f0b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2168099ab0 con 0x7f216806d530 2023-12-08T08:38:46.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7fa3ee7fc640 1 -- 172.21.15.16:0/1218181062 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e802cb00 con 0x7fa3f00cffe0 2023-12-08T08:38:46.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7f216ce4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f216806d530 0x7f21681ffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7fa3ee7fc640 1 -- 172.21.15.16:0/1218181062 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e802cc60 con 0x7fa3f00cffe0 2023-12-08T08:38:46.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7ff4c759e640 1 -- 172.21.15.16:0/4276944703 shutdown_connections 2023-12-08T08:38:46.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7ff4c759e640 1 --2- 172.21.15.16:0/4276944703 >> v2:172.21.15.16:3300/0 conn(0x7ff4c0048c00 0x7ff4c0049020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7ff4c759e640 1 --2- 172.21.15.16:0/4276944703 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4c0009150 0x7ff4c0052ec0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7ff4c759e640 1 -- 172.21.15.16:0/4276944703 >> 172.21.15.16:0/4276944703 conn(0x7ff4c001ef00 msgr2=0x7ff4c0055c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7f2167fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21680b31a0 0x7f216809be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7f2167fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21680b31a0 0x7f216809be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58438/0 (socket says 172.21.15.16:58438) 2023-12-08T08:38:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7f216ce4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f216806d530 0x7f21681ffb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52656/0 (socket says 172.21.15.16:52656) 2023-12-08T08:38:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7f2167fff640 1 -- 172.21.15.16:0/2749689361 learned_addr learned my addr 172.21.15.16:0/2749689361 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.319+0000 7f216f0b2640 1 -- 172.21.15.16:0/2749689361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21680ad140 con 0x7f21681595a0 2023-12-08T08:38:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.319+0000 7f216f0b2640 1 -- 172.21.15.16:0/2749689361 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21680ad420 con 0x7f21680b31a0 2023-12-08T08:38:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7fd58698a640 1 -- 172.21.15.16:0/2262786358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd58006d580 msgr2=0x7fd58006c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7fd58698a640 1 --2- 172.21.15.16:0/2262786358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd58006d580 0x7fd58006c2d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd58006fe80 tx=0x7fd57402b910 comp rx=0 tx=0).stop 2023-12-08T08:38:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7fd58698a640 1 -- 172.21.15.16:0/2262786358 shutdown_connections 2023-12-08T08:38:46.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7fd58698a640 1 --2- 172.21.15.16:0/2262786358 >> v2:172.21.15.16:3300/0 conn(0x7fd58006c920 0x7fd5800709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7fd58698a640 1 --2- 172.21.15.16:0/2262786358 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd58006d580 0x7fd58006c2d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7fd58698a640 1 -- 172.21.15.16:0/2262786358 >> 172.21.15.16:0/2262786358 conn(0x7fd5800b09f0 msgr2=0x7fd5800b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7fd58698a640 1 -- 172.21.15.16:0/2262786358 shutdown_connections 2023-12-08T08:38:46.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.318+0000 7fd58698a640 1 -- 172.21.15.16:0/2262786358 wait complete. 2023-12-08T08:38:46.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.319+0000 7f216d64f640 1 --2- 172.21.15.16:0/2749689361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21681595a0 0x7f216809c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.319+0000 7fe0467fc640 1 -- 172.21.15.16:0/3711997162 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c02d820 con 0x7fe04000aa70 2023-12-08T08:38:46.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.319+0000 7fe0467fc640 1 -- 172.21.15.16:0/3711997162 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c02d980 con 0x7fe04000aa70 2023-12-08T08:38:46.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.319+0000 7f216d64f640 1 -- 172.21.15.16:0/2749689361 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21680b31a0 msgr2=0x7f216809be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.319+0000 7f216d64f640 1 --2- 172.21.15.16:0/2749689361 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21680b31a0 0x7f216809be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.319+0000 7f216d64f640 1 -- 172.21.15.16:0/2749689361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f216806d530 msgr2=0x7f21681ffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.319+0000 7f216d64f640 1 --2- 172.21.15.16:0/2749689361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f216806d530 0x7f21681ffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.319+0000 7fa3ee7fc640 1 -- 172.21.15.16:0/1218181062 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e80343f0 con 0x7fa3f00cffe0 2023-12-08T08:38:46.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.319+0000 7fa3f5a56640 1 -- 172.21.15.16:0/1218181062 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3f00cffe0 msgr2=0x7fa3f00b2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.319+0000 7fa3f5a56640 1 --2- 172.21.15.16:0/1218181062 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3f00cffe0 0x7fa3f00b2d70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009e50 tx=0x7fa3e802bf60 comp rx=0 tx=0).stop 2023-12-08T08:38:46.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fa3f5a56640 1 -- 172.21.15.16:0/1218181062 shutdown_connections 2023-12-08T08:38:46.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fa3f5a56640 1 --2- 172.21.15.16:0/1218181062 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3f00cffe0 0x7fa3f00b2d70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fa3f5a56640 1 --2- 172.21.15.16:0/1218181062 >> v2:172.21.15.112:3300/0 conn(0x7fa3f0160a00 0x7fa3f00d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fa3f5a56640 1 -- 172.21.15.16:0/1218181062 >> 172.21.15.16:0/1218181062 conn(0x7fa3f009c300 msgr2=0x7fa3f00d2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7ff4c759e640 1 -- 172.21.15.16:0/4276944703 shutdown_connections 2023-12-08T08:38:46.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.319+0000 7f216d64f640 1 -- 172.21.15.16:0/2749689361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c009d60 con 0x7f21681595a0 2023-12-08T08:38:46.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.319+0000 7f216ce4e640 1 --2- 172.21.15.16:0/2749689361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f216806d530 0x7f21681ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:46.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7feaf959e640 1 Processor -- start 2023-12-08T08:38:46.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fe04ca1a640 1 -- 172.21.15.16:0/3711997162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04000aa70 msgr2=0x7fe04004fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7f216d64f640 1 --2- 172.21.15.16:0/2749689361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21681595a0 0x7f216809c350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f215c033000 tx=0x7f215c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fe0467fc640 1 -- 172.21.15.16:0/3711997162 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c02d820 con 0x7fe04000aa70 2023-12-08T08:38:46.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fa3f5a56640 1 -- 172.21.15.16:0/1218181062 shutdown_connections 2023-12-08T08:38:46.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fe04ca1a640 1 --2- 172.21.15.16:0/3711997162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04000aa70 0x7fe04004fa60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe040052980 tx=0x7fe03c02ba60 comp rx=0 tx=0).stop 2023-12-08T08:38:46.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fd58698a640 1 Processor -- start 2023-12-08T08:38:46.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fd58698a640 1 -- start start 2023-12-08T08:38:46.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7feaf959e640 1 -- start start 2023-12-08T08:38:46.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fa3f5a56640 1 -- 172.21.15.16:0/1218181062 wait complete. 2023-12-08T08:38:46.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7f21657fa640 1 -- 172.21.15.16:0/2749689361 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c034440 con 0x7f21681595a0 2023-12-08T08:38:46.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7f216f0b2640 1 -- 172.21.15.16:0/2749689361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2168099d50 con 0x7f21681595a0 2023-12-08T08:38:46.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7f216f0b2640 1 -- 172.21.15.16:0/2749689361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2168152630 con 0x7f21681595a0 2023-12-08T08:38:46.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fe04ca1a640 1 -- 172.21.15.16:0/3711997162 shutdown_connections 2023-12-08T08:38:46.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fe04ca1a640 1 --2- 172.21.15.16:0/3711997162 >> v2:172.21.15.16:3300/0 conn(0x7fe0400df360 0x7fe04004ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fe04ca1a640 1 --2- 172.21.15.16:0/3711997162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe04000aa70 0x7fe04004fa60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fe04ca1a640 1 -- 172.21.15.16:0/3711997162 >> 172.21.15.16:0/3711997162 conn(0x7fe040009f80 msgr2=0x7fe04000a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7f21657fa640 1 -- 172.21.15.16:0/2749689361 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c034e90 con 0x7f21681595a0 2023-12-08T08:38:46.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7f21657fa640 1 -- 172.21.15.16:0/2749689361 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c03e680 con 0x7f21681595a0 2023-12-08T08:38:46.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.321+0000 7fd58698a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58006c920 0x7fd580154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.321+0000 7fd58698a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd58006d580 0x7fd5800acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.321+0000 7fd58698a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd580161860 0x7fd5800ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.321+0000 7fd58698a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5801f70f0 con 0x7fd58006c920 2023-12-08T08:38:46.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.321+0000 7fd58698a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58009a6d0 con 0x7fd580161860 2023-12-08T08:38:46.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.321+0000 7fd58698a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd580154e80 con 0x7fd58006d580 2023-12-08T08:38:46.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.321+0000 7fd57f7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd58006d580 0x7fd5800acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.321+0000 7fd57f7fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd58006d580 0x7fd5800acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58450/0 (socket says 172.21.15.16:58450) 2023-12-08T08:38:46.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7fd57f7fe640 1 -- 172.21.15.16:0/1205777432 learned_addr learned my addr 172.21.15.16:0/1205777432 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7fd57ffff640 1 --2- 172.21.15.16:0/1205777432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58006c920 0x7fd580154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7fd57f7fe640 1 -- 172.21.15.16:0/1205777432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd580161860 msgr2=0x7fd5800ad4c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-08T08:38:46.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7fd57f7fe640 1 --2- 172.21.15.16:0/1205777432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd580161860 0x7fd5800ad4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7fd57f7fe640 1 -- 172.21.15.16:0/1205777432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58006c920 msgr2=0x7fd580154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7fd57f7fe640 1 --2- 172.21.15.16:0/1205777432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58006c920 0x7fd580154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7fd57f7fe640 1 -- 172.21.15.16:0/1205777432 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5740094d0 con 0x7fd58006d580 2023-12-08T08:38:46.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7fd57ffff640 1 --2- 172.21.15.16:0/1205777432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58006c920 0x7fd580154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:46.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.321+0000 7feaf959e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaf406da30 0x7feaf4146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7feaf2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaf406da30 0x7feaf4146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7feaf2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaf406da30 0x7feaf4146e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52662/0 (socket says 172.21.15.16:52662) 2023-12-08T08:38:46.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7feaf2ffd640 1 -- 172.21.15.16:0/1091263266 learned_addr learned my addr 172.21.15.16:0/1091263266 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7feaf959e640 1 --2- 172.21.15.16:0/1091263266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaf40b3860 0x7feaf41431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7feaf959e640 1 --2- 172.21.15.16:0/1091263266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaf40b49c0 0x7feaf41eae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7feaf959e640 1 -- 172.21.15.16:0/1091263266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4143710 con 0x7feaf406da30 2023-12-08T08:38:46.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7feaf959e640 1 -- 172.21.15.16:0/1091263266 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4144610 con 0x7feaf40b3860 2023-12-08T08:38:46.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7feaf959e640 1 -- 172.21.15.16:0/1091263266 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf415ba80 con 0x7feaf40b49c0 2023-12-08T08:38:46.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7fa3f5a56640 1 Processor -- start 2023-12-08T08:38:46.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7ff4c759e640 1 -- 172.21.15.16:0/4276944703 wait complete. 2023-12-08T08:38:46.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7ff4c759e640 1 Processor -- start 2023-12-08T08:38:46.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7ff4c759e640 1 -- start start 2023-12-08T08:38:46.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fe04ca1a640 1 -- 172.21.15.16:0/3711997162 shutdown_connections 2023-12-08T08:38:46.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7feaf2ffd640 1 -- 172.21.15.16:0/1091263266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaf40b49c0 msgr2=0x7feaf41eae70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-08T08:38:46.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7feaf2ffd640 1 --2- 172.21.15.16:0/1091263266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaf40b49c0 0x7feaf41eae70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7feaf2ffd640 1 -- 172.21.15.16:0/1091263266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaf40b3860 msgr2=0x7feaf41431d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-08T08:38:46.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7feaf2ffd640 1 --2- 172.21.15.16:0/1091263266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaf40b3860 0x7feaf41431d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7feaf2ffd640 1 -- 172.21.15.16:0/1091263266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec009a70 con 0x7feaf406da30 2023-12-08T08:38:46.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7fa3f5a56640 1 -- start start 2023-12-08T08:38:46.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.320+0000 7fe04ca1a640 1 -- 172.21.15.16:0/3711997162 wait complete. 2023-12-08T08:38:46.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7f21657fa640 1 -- 172.21.15.16:0/2749689361 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f215c03e860 con 0x7f21681595a0 2023-12-08T08:38:46.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7fd57f7fe640 1 --2- 172.21.15.16:0/1205777432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd58006d580 0x7fd5800acf80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd574000c00 tx=0x7fd5740046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7f21657fa640 1 --2- 172.21.15.16:0/2749689361 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f215406ae90 0x7f215406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7f21657fa640 1 -- 172.21.15.16:0/2749689361 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f215c0aba10 con 0x7f21681595a0 2023-12-08T08:38:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7f216f0b2640 1 -- 172.21.15.16:0/2749689361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2148005250 con 0x7f21681595a0 2023-12-08T08:38:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7f216ce4e640 1 --2- 172.21.15.16:0/2749689361 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f215406ae90 0x7f215406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7fd57cff9640 1 -- 172.21.15.16:0/1205777432 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574031870 con 0x7fd58006d580 2023-12-08T08:38:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7fd57cff9640 1 -- 172.21.15.16:0/1205777432 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5740305d0 con 0x7fd58006d580 2023-12-08T08:38:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fd57cff9640 1 -- 172.21.15.16:0/1205777432 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574030940 con 0x7fd58006d580 2023-12-08T08:38:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7f9c6359e640 1 Processor -- start 2023-12-08T08:38:46.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7f9c6359e640 1 -- start start 2023-12-08T08:38:46.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7f9c6359e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0e3080 0x7f9c5c057d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7f9c6359e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c5c03e800 0x7f9c5c053bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7f9c6359e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5c058290 con 0x7f9c5c0540f0 2023-12-08T08:38:46.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7f9c6359e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5c054580 con 0x7f9c5c03e800 2023-12-08T08:38:46.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7f9c6359e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c006160 con 0x7f9c5c0e3080 2023-12-08T08:38:46.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7f9c61d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c5c03e800 0x7f9c5c053bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7f9c61d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c5c03e800 0x7f9c5c053bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52694/0 (socket says 172.21.15.16:52694) 2023-12-08T08:38:46.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7f9c61d9b640 1 -- 172.21.15.16:0/67438261 learned_addr learned my addr 172.21.15.16:0/67438261 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7f9c6259c640 1 --2- 172.21.15.16:0/67438261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0e3080 0x7f9c5c057d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7feaf2ffd640 1 --2- 172.21.15.16:0/1091263266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaf406da30 0x7feaf4146e50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feae400c440 tx=0x7feae400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7feaebfff640 1 -- 172.21.15.16:0/1091263266 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4005ce0 con 0x7feaf406da30 2023-12-08T08:38:46.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7feaf959e640 1 -- 172.21.15.16:0/1091263266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf4143a00 con 0x7feaf406da30 2023-12-08T08:38:46.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7feaebfff640 1 -- 172.21.15.16:0/1091263266 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae4005e40 con 0x7feaf406da30 2023-12-08T08:38:46.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7feaf959e640 1 -- 172.21.15.16:0/1091263266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf4143fc0 con 0x7feaf406da30 2023-12-08T08:38:46.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7feaebfff640 1 -- 172.21.15.16:0/1091263266 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae40072b0 con 0x7feaf406da30 2023-12-08T08:38:46.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7fa3f5a56640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3f00b1ea0 0x7fa3f0201410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7fa3f5a56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f00cffe0 0x7fa3f01f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7fa3f5a56640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0160a00 0x7fa3f00c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7fa3f5a56640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f02026c0 con 0x7fa3f00cffe0 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7fa3f5a56640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f01f3070 con 0x7fa3f0160a00 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7fa3f5a56640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f00c9170 con 0x7fa3f00b1ea0 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7fa3eeffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3f00b1ea0 0x7fa3f0201410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fa3eeffd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3f00b1ea0 0x7fa3f0201410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58478/0 (socket says 172.21.15.16:58478) 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fa3eeffd640 1 -- 172.21.15.16:0/142530932 learned_addr learned my addr 172.21.15.16:0/142530932 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fa3ef7fe640 1 --2- 172.21.15.16:0/142530932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0160a00 0x7fa3f00c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fa3eeffd640 1 -- 172.21.15.16:0/142530932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0160a00 msgr2=0x7fa3f00c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fa3eeffd640 1 --2- 172.21.15.16:0/142530932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0160a00 0x7fa3f00c8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fa3eeffd640 1 -- 172.21.15.16:0/142530932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f00cffe0 msgr2=0x7fa3f01f2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fa3eeffd640 1 --2- 172.21.15.16:0/142530932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f00cffe0 0x7fa3f01f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fa3eeffd640 1 -- 172.21.15.16:0/142530932 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e8009b30 con 0x7fa3f00b1ea0 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fa3eeffd640 1 --2- 172.21.15.16:0/142530932 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3f00b1ea0 0x7fa3f0201410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000d7d0 tx=0x7fa3e000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7fa3ecff9640 1 -- 172.21.15.16:0/142530932 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0004030 con 0x7fa3f00b1ea0 2023-12-08T08:38:46.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7fa3f5a56640 1 -- 172.21.15.16:0/142530932 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f0201ab0 con 0x7fa3f00b1ea0 2023-12-08T08:38:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7ff4c759e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c0009150 0x7ff4c00e60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7ff4c759e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4c0048c00 0x7ff4c004d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7ff4c759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c00eb260 0x7ff4c004da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7ff4c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c00eb260 0x7ff4c004da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7ff4c6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c00eb260 0x7ff4c004da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52692/0 (socket says 172.21.15.16:52692) 2023-12-08T08:38:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7ff4c5d9b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4c0048c00 0x7ff4c004d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7ff4c5d9b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4c0048c00 0x7ff4c004d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58480/0 (socket says 172.21.15.16:58480) 2023-12-08T08:38:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7ff4c6d9d640 1 -- 172.21.15.16:0/2214954236 learned_addr learned my addr 172.21.15.16:0/2214954236 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7ff4c759e640 1 -- 172.21.15.16:0/2214954236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c00e8b70 con 0x7ff4c00eb260 2023-12-08T08:38:46.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7ff4c759e640 1 -- 172.21.15.16:0/2214954236 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c00e6c40 con 0x7ff4c0009150 2023-12-08T08:38:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7ff4c759e640 1 -- 172.21.15.16:0/2214954236 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c00e7ca0 con 0x7ff4c0048c00 2023-12-08T08:38:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.322+0000 7fe04ca1a640 1 Processor -- start 2023-12-08T08:38:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.323+0000 7fe04ca1a640 1 -- start start 2023-12-08T08:38:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fe04ca1a640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe04000aa70 0x7fe040188a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fe04ca1a640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe040053230 0x7fe04004cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fe04ca1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0400df360 0x7fe040039260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fe04ca1a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0400e0750 con 0x7fe0400df360 2023-12-08T08:38:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fe04ca1a640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0400dfb40 con 0x7fe040053230 2023-12-08T08:38:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.324+0000 7fe04ca1a640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0400dfe20 con 0x7fe04000aa70 2023-12-08T08:38:46.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7fe0477fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe04000aa70 0x7fe040188a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7fe0477fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe04000aa70 0x7fe040188a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:58496/0 (socket says 172.21.15.16:58496) 2023-12-08T08:38:46.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7fe0477fe640 1 -- 172.21.15.16:0/1101219061 learned_addr learned my addr 172.21.15.16:0/1101219061 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7fa3f5a56640 1 -- 172.21.15.16:0/142530932 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f0202070 con 0x7fa3f00b1ea0 2023-12-08T08:38:46.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.326+0000 7fa3ecff9640 1 -- 172.21.15.16:0/142530932 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e0004190 con 0x7fa3f00b1ea0 2023-12-08T08:38:46.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.326+0000 7f9c61d9b640 1 -- 172.21.15.16:0/67438261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0e3080 msgr2=0x7f9c5c057d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:46.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.326+0000 7f9c61d9b640 1 --2- 172.21.15.16:0/67438261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0e3080 0x7f9c5c057d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.326+0000 7f9c61d9b640 1 -- 172.21.15.16:0/67438261 >> v1:172.21.15.16:6789/0 conn(0x7f9c5c0540f0 legacy=0x7f9c5c0e6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:46.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.326+0000 7fa3ecff9640 1 -- 172.21.15.16:0/142530932 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0004520 con 0x7fa3f00b1ea0 2023-12-08T08:38:46.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.326+0000 7feaf959e640 1 -- 172.21.15.16:0/1091263266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead8005250 con 0x7feaf406da30 2023-12-08T08:38:46.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.326+0000 7fe046ffd640 1 --2- 172.21.15.16:0/1101219061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe040053230 0x7fe04004cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.326+0000 7fe047fff640 1 --2- 172.21.15.16:0/1101219061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0400df360 0x7fe040039260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7fd58698a640 1 -- 172.21.15.16:0/1205777432 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58009be30 con 0x7fd58006d580 2023-12-08T08:38:46.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7fd58698a640 1 -- 172.21.15.16:0/1205777432 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58009c3f0 con 0x7fd58006d580 2023-12-08T08:38:46.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7ff4c6d9d640 1 -- 172.21.15.16:0/2214954236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4c0048c00 msgr2=0x7ff4c004d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7fd58698a640 1 -- 172.21.15.16:0/1205777432 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5800ae450 con 0x7fd58006d580 2023-12-08T08:38:46.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7ff4c6d9d640 1 --2- 172.21.15.16:0/2214954236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4c0048c00 0x7ff4c004d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7ff4c6d9d640 1 -- 172.21.15.16:0/2214954236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c0009150 msgr2=0x7ff4c00e60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7ff4c6d9d640 1 --2- 172.21.15.16:0/2214954236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c0009150 0x7ff4c00e60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.325+0000 7ff4c6d9d640 1 -- 172.21.15.16:0/2214954236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b80094d0 con 0x7ff4c00eb260 2023-12-08T08:38:46.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.326+0000 7ff4c6d9d640 1 --2- 172.21.15.16:0/2214954236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c00eb260 0x7ff4c004da30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8000b60 tx=0x7ff4c8073190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7ff4b6ffd640 1 -- 172.21.15.16:0/2214954236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c806bc90 con 0x7ff4c00eb260 2023-12-08T08:38:46.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.326+0000 7f9c61d9b640 1 -- 172.21.15.16:0/67438261 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c058530 con 0x7f9c5c03e800 2023-12-08T08:38:46.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7ff4c759e640 1 -- 172.21.15.16:0/2214954236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4c00e65f0 con 0x7ff4c00eb260 2023-12-08T08:38:46.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7ff4c759e640 1 -- 172.21.15.16:0/2214954236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4c00e81d0 con 0x7ff4c00eb260 2023-12-08T08:38:46.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7fe0477fe640 1 -- 172.21.15.16:0/1101219061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe040053230 msgr2=0x7fe04004cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.326+0000 7feaebfff640 1 -- 172.21.15.16:0/1091263266 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feae4007410 con 0x7feaf406da30 2023-12-08T08:38:46.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7fe0477fe640 1 --2- 172.21.15.16:0/1101219061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe040053230 0x7fe04004cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7fe0477fe640 1 -- 172.21.15.16:0/1101219061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0400df360 msgr2=0x7fe040039260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7fe0477fe640 1 --2- 172.21.15.16:0/1101219061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0400df360 0x7fe040039260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7feaebfff640 1 --2- 172.21.15.16:0/1091263266 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fead406ae40 0x7fead406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7feaebfff640 1 -- 172.21.15.16:0/1091263266 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7feae4088e10 con 0x7feaf406da30 2023-12-08T08:38:46.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7ff4b6ffd640 1 -- 172.21.15.16:0/2214954236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c806bdf0 con 0x7ff4c00eb260 2023-12-08T08:38:46.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7fe0477fe640 1 -- 172.21.15.16:0/1101219061 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c0094d0 con 0x7fe04000aa70 2023-12-08T08:38:46.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7feaea7fc640 1 --2- 172.21.15.16:0/1091263266 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fead406ae40 0x7fead406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7f216ce4e640 1 --2- 172.21.15.16:0/2749689361 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f215406ae90 0x7f215406d350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f216814f8b0 tx=0x7f21600077c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7ff4b6ffd640 1 -- 172.21.15.16:0/2214954236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c80723e0 con 0x7ff4c00eb260 2023-12-08T08:38:46.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.327+0000 7fa3f5a56640 1 -- 172.21.15.16:0/142530932 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d4005250 con 0x7fa3f00b1ea0 2023-12-08T08:38:46.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.328+0000 7f9c61d9b640 1 --2- 172.21.15.16:0/67438261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c03e800 0x7f9c5c053bb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009870 tx=0x7f9c5802ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faba7cbc67c0574f server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.328+0000 7feaea7fc640 1 --2- 172.21.15.16:0/1091263266 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fead406ae40 0x7fead406d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feaf4156d90 tx=0x7feadc007930 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.328+0000 7fe0477fe640 1 --2- 172.21.15.16:0/1101219061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe04000aa70 0x7fe040188a10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe03c02baa0 tx=0x7fe03c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.328+0000 7fa3ecff9640 1 -- 172.21.15.16:0/142530932 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa3e000b6c0 con 0x7fa3f00b1ea0 2023-12-08T08:38:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.329+0000 7ff4c759e640 1 -- 172.21.15.16:0/2214954236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ac005250 con 0x7ff4c00eb260 2023-12-08T08:38:46.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.329+0000 7fd57cff9640 1 -- 172.21.15.16:0/1205777432 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd574030aa0 con 0x7fd58006d580 2023-12-08T08:38:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.329+0000 7fa3ecff9640 1 --2- 172.21.15.16:0/142530932 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3d006ae90 0x7fa3d006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.329+0000 7fa3ecff9640 1 -- 172.21.15.16:0/142530932 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa3e0089960 con 0x7fa3f00b1ea0 2023-12-08T08:38:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.329+0000 7fa3e7fff640 1 --2- 172.21.15.16:0/142530932 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3d006ae90 0x7fa3d006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.329+0000 7fd57cff9640 1 --2- 172.21.15.16:0/1205777432 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd56c06af10 0x7fd56c06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.329+0000 7fd57cff9640 1 -- 172.21.15.16:0/1205777432 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd57402d040 con 0x7fd58006d580 2023-12-08T08:38:46.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.330+0000 7fd57ffff640 1 --2- 172.21.15.16:0/1205777432 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd56c06af10 0x7fd56c06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.330+0000 7fa3e7fff640 1 --2- 172.21.15.16:0/142530932 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3d006ae90 0x7fa3d006d350 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa3f01f4290 tx=0x7fa3d80079c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.330+0000 7f21657fa640 1 -- 172.21.15.16:0/2749689361 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f215c07a2d0 con 0x7f21681595a0 2023-12-08T08:38:46.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.330+0000 7fd57ffff640 1 --2- 172.21.15.16:0/1205777432 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd56c06af10 0x7fd56c06d3d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd578009fc0 tx=0x7fd578009550 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.332+0000 7fe033fff640 1 -- 172.21.15.16:0/1101219061 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c004450 con 0x7fe04000aa70 2023-12-08T08:38:46.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.332+0000 7fe04ca1a640 1 -- 172.21.15.16:0/1101219061 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0400e0100 con 0x7fe04000aa70 2023-12-08T08:38:46.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.332+0000 7fd57cff9640 1 -- 172.21.15.16:0/1205777432 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd5740777a0 con 0x7fd58006d580 2023-12-08T08:38:46.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.333+0000 7fe04ca1a640 1 -- 172.21.15.16:0/1101219061 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0400df7d0 con 0x7fe04000aa70 2023-12-08T08:38:46.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.333+0000 7f9c6159a640 1 -- 172.21.15.16:0/67438261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5802f850 con 0x7f9c5c03e800 2023-12-08T08:38:46.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.333+0000 7f9c6159a640 1 -- 172.21.15.16:0/67438261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5802f9b0 con 0x7f9c5c03e800 2023-12-08T08:38:46.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.333+0000 7f9c6159a640 1 -- 172.21.15.16:0/67438261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5802fd80 con 0x7f9c5c03e800 2023-12-08T08:38:46.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.334+0000 7fe031ffb640 1 -- 172.21.15.16:0/1101219061 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe028005250 con 0x7fe04000aa70 2023-12-08T08:38:46.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.334+0000 7feaebfff640 1 -- 172.21.15.16:0/1091263266 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feae4057710 con 0x7feaf406da30 2023-12-08T08:38:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.335+0000 7f9c6359e640 1 -- 172.21.15.16:0/67438261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c03e800 msgr2=0x7f9c5c053bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.335+0000 7f9c6359e640 1 --2- 172.21.15.16:0/67438261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c03e800 0x7f9c5c053bb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009870 tx=0x7f9c5802ba30 comp rx=0 tx=0).stop 2023-12-08T08:38:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.335+0000 7ff4b6ffd640 1 -- 172.21.15.16:0/2214954236 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff4c8072600 con 0x7ff4c00eb260 2023-12-08T08:38:46.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.335+0000 7f9c6359e640 1 -- 172.21.15.16:0/67438261 shutdown_connections 2023-12-08T08:38:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.336+0000 7ff4b6ffd640 1 --2- 172.21.15.16:0/2214954236 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4a806ae40 0x7ff4a806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.335+0000 7f9c6359e640 1 --2- 172.21.15.16:0/67438261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c03e800 0x7f9c5c053bb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.335+0000 7f9c6359e640 1 --2- 172.21.15.16:0/67438261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0e3080 0x7f9c5c057d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.335+0000 7f9c6359e640 1 -- 172.21.15.16:0/67438261 >> 172.21.15.16:0/67438261 conn(0x7f9c5c0559a0 msgr2=0x7f9c5c055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.336+0000 7f9c6359e640 1 -- 172.21.15.16:0/67438261 shutdown_connections 2023-12-08T08:38:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.336+0000 7f9c6359e640 1 -- 172.21.15.16:0/67438261 wait complete. 2023-12-08T08:38:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.336+0000 7ff4b6ffd640 1 -- 172.21.15.16:0/2214954236 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff4c81039a0 con 0x7ff4c00eb260 2023-12-08T08:38:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.336+0000 7ff4c659c640 1 --2- 172.21.15.16:0/2214954236 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4a806ae40 0x7ff4a806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.336+0000 7ff4c659c640 1 --2- 172.21.15.16:0/2214954236 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4a806ae40 0x7ff4a806d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8036000 tx=0x7ff4b8030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.337+0000 7fe033fff640 1 -- 172.21.15.16:0/1101219061 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c003610 con 0x7fe04000aa70 2023-12-08T08:38:46.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.337+0000 7fe033fff640 1 -- 172.21.15.16:0/1101219061 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c0039a0 con 0x7fe04000aa70 2023-12-08T08:38:46.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.338+0000 7f9c6359e640 1 Processor -- start 2023-12-08T08:38:46.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.338+0000 7f9c6359e640 1 -- start start 2023-12-08T08:38:46.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.337+0000 7fa3ecff9640 1 -- 172.21.15.16:0/142530932 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3e0054800 con 0x7fa3f00b1ea0 2023-12-08T08:38:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.339+0000 7fe033fff640 1 -- 172.21.15.16:0/1101219061 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe03c003b00 con 0x7fe04000aa70 2023-12-08T08:38:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.339+0000 7fe033fff640 1 --2- 172.21.15.16:0/1101219061 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe02c06ae40 0x7fe02c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.339+0000 7f9c6359e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c03e800 0x7f9c5c052a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.339+0000 7f9c6359e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0540f0 0x7f9c5c0e66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.339+0000 7fe033fff640 1 -- 172.21.15.16:0/1101219061 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe03c03e070 con 0x7fe04000aa70 2023-12-08T08:38:46.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7f9c6359e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c0e3080 0x7f9c5c0db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7f9c6359e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0e4d50 con 0x7f9c5c03e800 2023-12-08T08:38:46.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7f9c6359e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0dbfa0 con 0x7f9c5c0e3080 2023-12-08T08:38:46.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7f9c6359e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c0e46b0 con 0x7f9c5c0540f0 2023-12-08T08:38:46.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7f9c6259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c03e800 0x7f9c5c052a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7f9c6259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c03e800 0x7f9c5c052a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52706/0 (socket says 172.21.15.16:52706) 2023-12-08T08:38:46.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7f9c6259c640 1 -- 172.21.15.16:0/3211212479 learned_addr learned my addr 172.21.15.16:0/3211212479 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7f9c61d9b640 1 --2- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0540f0 0x7f9c5c0e66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7f9c62d9d640 1 --2- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c0e3080 0x7f9c5c0db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7fe046ffd640 1 --2- 172.21.15.16:0/1101219061 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe02c06ae40 0x7fe02c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7f9c61d9b640 1 -- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c0e3080 msgr2=0x7f9c5c0db3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7f9c61d9b640 1 --2- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c0e3080 0x7f9c5c0db3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7f9c61d9b640 1 -- 172.21.15.16:0/3211212479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c03e800 msgr2=0x7f9c5c052a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7f9c61d9b640 1 --2- 172.21.15.16:0/3211212479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c03e800 0x7f9c5c052a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.340+0000 7f9c61d9b640 1 -- 172.21.15.16:0/3211212479 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c580094d0 con 0x7f9c5c0540f0 2023-12-08T08:38:46.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.341+0000 7f9c61d9b640 1 --2- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0540f0 0x7f9c5c0e66c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9c58033000 tx=0x7f9c58004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.342+0000 7fe046ffd640 1 --2- 172.21.15.16:0/1101219061 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe02c06ae40 0x7fe02c06d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe040050b20 tx=0x7fe034008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.342+0000 7f9c52ffd640 1 -- 172.21.15.16:0/3211212479 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58037d10 con 0x7f9c5c0540f0 2023-12-08T08:38:46.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.342+0000 7f9c52ffd640 1 -- 172.21.15.16:0/3211212479 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c58037e70 con 0x7f9c5c0540f0 2023-12-08T08:38:46.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.342+0000 7f9c52ffd640 1 -- 172.21.15.16:0/3211212479 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58041600 con 0x7f9c5c0540f0 2023-12-08T08:38:46.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.342+0000 7fe033fff640 1 -- 172.21.15.16:0/1101219061 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe03c076a10 con 0x7fe04000aa70 2023-12-08T08:38:46.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.342+0000 7f9c6359e640 1 -- 172.21.15.16:0/3211212479 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c0e6c00 con 0x7f9c5c0540f0 2023-12-08T08:38:46.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.342+0000 7f9c6359e640 1 -- 172.21.15.16:0/3211212479 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c0e71a0 con 0x7f9c5c0540f0 2023-12-08T08:38:46.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.343+0000 7ff4b6ffd640 1 -- 172.21.15.16:0/2214954236 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4c80ce750 con 0x7ff4c00eb260 2023-12-08T08:38:46.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.344+0000 7f9c52ffd640 1 -- 172.21.15.16:0/3211212479 <== mon.2 v2:172.21.15.112:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9c58041760 con 0x7f9c5c0540f0 2023-12-08T08:38:46.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.345+0000 7f9c52ffd640 1 --2- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9c4406af10 0x7f9c4406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.345+0000 7f9c52ffd640 1 -- 172.21.15.16:0/3211212479 <== mon.2 v2:172.21.15.112:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9c5803c070 con 0x7f9c5c0540f0 2023-12-08T08:38:46.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.345+0000 7f9c6259c640 1 --2- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9c4406af10 0x7f9c4406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.346+0000 7f9c6359e640 1 -- 172.21.15.16:0/3211212479 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c48005250 con 0x7f9c5c0540f0 2023-12-08T08:38:46.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.347+0000 7f9c6259c640 1 --2- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9c4406af10 0x7f9c4406d3d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400aae0 tx=0x7f9c54008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.351+0000 7f9c52ffd640 1 -- 172.21.15.16:0/3211212479 <== mon.2 v2:172.21.15.112:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9c58081030 con 0x7f9c5c0540f0 2023-12-08T08:38:46.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.693+0000 7fd5656cf640 1 -- 172.21.15.16:0/3410705471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd540005500 con 0x7fd5600cbd50 2023-12-08T08:38:46.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.694+0000 7fd5477fe640 1 -- 172.21.15.16:0/3410705471 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd554019070 con 0x7fd5600cbd50 2023-12-08T08:38:46.697 INFO:teuthology.orchestra.run.smithi016.stdout:55834574858 2023-12-08T08:38:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.698+0000 7fd5457fa640 1 -- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd54806b130 msgr2=0x7fd54806d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.698+0000 7fd5457fa640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd54806b130 0x7fd54806d5f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd560072fc0 tx=0x7fd550009190 comp rx=0 tx=0).stop 2023-12-08T08:38:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.698+0000 7fd5457fa640 1 -- 172.21.15.16:0/3410705471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5600cbd50 msgr2=0x7fd56014d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.698+0000 7fd5457fa640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5600cbd50 0x7fd56014d370 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd55400b6d0 tx=0x7fd55400bb90 comp rx=0 tx=0).stop 2023-12-08T08:38:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.698+0000 7fd5457fa640 1 -- 172.21.15.16:0/3410705471 shutdown_connections 2023-12-08T08:38:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.698+0000 7fd5457fa640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd54806b130 0x7fd54806d5f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.699+0000 7fd5457fa640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd560159bc0 0x7fd56014e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.699+0000 7fd5457fa640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5600ceed0 0x7fd5601657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.699+0000 7fd5457fa640 1 --2- 172.21.15.16:0/3410705471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5600cbd50 0x7fd56014d370 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.699+0000 7fd5457fa640 1 -- 172.21.15.16:0/3410705471 >> 172.21.15.16:0/3410705471 conn(0x7fd560071860 msgr2=0x7fd56006c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.699+0000 7fd5457fa640 1 -- 172.21.15.16:0/3410705471 shutdown_connections 2023-12-08T08:38:46.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.699+0000 7fd5457fa640 1 -- 172.21.15.16:0/3410705471 wait complete. 2023-12-08T08:38:46.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2023-12-08T08:38:46.719 DEBUG:teuthology.parallel:result is None 2023-12-08T08:38:46.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.731+0000 7ff4c759e640 1 -- 172.21.15.16:0/2214954236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff4ac0057d0 con 0x7ff4c00eb260 2023-12-08T08:38:46.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.732+0000 7ff4b6ffd640 1 -- 172.21.15.16:0/2214954236 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff4c80d1d70 con 0x7ff4c00eb260 2023-12-08T08:38:46.735 INFO:teuthology.orchestra.run.smithi016.stdout:55834574858 2023-12-08T08:38:46.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.734+0000 7ff4c759e640 1 -- 172.21.15.16:0/2214954236 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4a806ae40 msgr2=0x7ff4a806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.734+0000 7ff4c759e640 1 --2- 172.21.15.16:0/2214954236 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4a806ae40 0x7ff4a806d300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8036000 tx=0x7ff4b8030000 comp rx=0 tx=0).stop 2023-12-08T08:38:46.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.734+0000 7ff4c759e640 1 -- 172.21.15.16:0/2214954236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c00eb260 msgr2=0x7ff4c004da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.734+0000 7ff4c759e640 1 --2- 172.21.15.16:0/2214954236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c00eb260 0x7ff4c004da30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8000b60 tx=0x7ff4c8073190 comp rx=0 tx=0).stop 2023-12-08T08:38:46.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.734+0000 7ff4c759e640 1 -- 172.21.15.16:0/2214954236 shutdown_connections 2023-12-08T08:38:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.734+0000 7ff4c759e640 1 --2- 172.21.15.16:0/2214954236 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4a806ae40 0x7ff4a806d300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.734+0000 7ff4c759e640 1 --2- 172.21.15.16:0/2214954236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4c00eb260 0x7ff4c004da30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.734+0000 7ff4c759e640 1 --2- 172.21.15.16:0/2214954236 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4c0048c00 0x7ff4c004d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.734+0000 7ff4c759e640 1 --2- 172.21.15.16:0/2214954236 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4c0009150 0x7ff4c00e60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.734+0000 7ff4c759e640 1 -- 172.21.15.16:0/2214954236 >> 172.21.15.16:0/2214954236 conn(0x7ff4c001ef00 msgr2=0x7ff4c000c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.734+0000 7ff4c759e640 1 -- 172.21.15.16:0/2214954236 shutdown_connections 2023-12-08T08:38:46.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.734+0000 7ff4c759e640 1 -- 172.21.15.16:0/2214954236 wait complete. 2023-12-08T08:38:46.750 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2023-12-08T08:38:46.750 DEBUG:teuthology.parallel:result is None 2023-12-08T08:38:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.748+0000 7fa3f5a56640 1 -- 172.21.15.16:0/142530932 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa3d40057d0 con 0x7fa3f00b1ea0 2023-12-08T08:38:46.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.748+0000 7fa3ecff9640 1 -- 172.21.15.16:0/142530932 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa3e0057e20 con 0x7fa3f00b1ea0 2023-12-08T08:38:46.751 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-12-08T08:38:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.749+0000 7fd58698a640 1 -- 172.21.15.16:0/1205777432 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd580067430 con 0x7fd58006d580 2023-12-08T08:38:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.749+0000 7f216f0b2640 1 -- 172.21.15.16:0/2749689361 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2148005500 con 0x7f21681595a0 2023-12-08T08:38:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.750+0000 7f21657fa640 1 -- 172.21.15.16:0/2749689361 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f215c039030 con 0x7f21681595a0 2023-12-08T08:38:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.750+0000 7fd57cff9640 1 -- 172.21.15.16:0/1205777432 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd57407adc0 con 0x7fd58006d580 2023-12-08T08:38:46.754 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-12-08T08:38:46.754 INFO:teuthology.orchestra.run.smithi016.stdout:55834574858 2023-12-08T08:38:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.752+0000 7fa3f5a56640 1 -- 172.21.15.16:0/142530932 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3d006ae90 msgr2=0x7fa3d006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.752+0000 7fa3f5a56640 1 --2- 172.21.15.16:0/142530932 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3d006ae90 0x7fa3d006d350 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa3f01f4290 tx=0x7fa3d80079c0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.752+0000 7fa3f5a56640 1 -- 172.21.15.16:0/142530932 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3f00b1ea0 msgr2=0x7fa3f0201410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.752+0000 7fa3f5a56640 1 --2- 172.21.15.16:0/142530932 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3f00b1ea0 0x7fa3f0201410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000d7d0 tx=0x7fa3e000dc90 comp rx=0 tx=0).stop 2023-12-08T08:38:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.752+0000 7fa3f5a56640 1 -- 172.21.15.16:0/142530932 shutdown_connections 2023-12-08T08:38:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.752+0000 7fa3f5a56640 1 --2- 172.21.15.16:0/142530932 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3d006ae90 0x7fa3d006d350 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.752+0000 7fa3f5a56640 1 --2- 172.21.15.16:0/142530932 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3f0160a00 0x7fa3f00c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.752+0000 7fa3f5a56640 1 --2- 172.21.15.16:0/142530932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f00cffe0 0x7fa3f01f2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.752+0000 7fa3f5a56640 1 --2- 172.21.15.16:0/142530932 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3f00b1ea0 0x7fa3f0201410 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.752+0000 7fa3f5a56640 1 -- 172.21.15.16:0/142530932 >> 172.21.15.16:0/142530932 conn(0x7fa3f009c300 msgr2=0x7fa3f0066230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.752+0000 7fa3f5a56640 1 -- 172.21.15.16:0/142530932 shutdown_connections 2023-12-08T08:38:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.752+0000 7fa3f5a56640 1 -- 172.21.15.16:0/142530932 wait complete. 2023-12-08T08:38:46.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.752+0000 7feaf959e640 1 -- 172.21.15.16:0/1091263266 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fead8005500 con 0x7feaf406da30 2023-12-08T08:38:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7f216f0b2640 1 -- 172.21.15.16:0/2749689361 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f215406ae90 msgr2=0x7f215406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7f216f0b2640 1 --2- 172.21.15.16:0/2749689361 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f215406ae90 0x7f215406d350 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f216814f8b0 tx=0x7f21600077c0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7f216f0b2640 1 -- 172.21.15.16:0/2749689361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21681595a0 msgr2=0x7f216809c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7f216f0b2640 1 --2- 172.21.15.16:0/2749689361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21681595a0 0x7f216809c350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f215c033000 tx=0x7f215c0046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7f216f0b2640 1 -- 172.21.15.16:0/2749689361 shutdown_connections 2023-12-08T08:38:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7f216f0b2640 1 --2- 172.21.15.16:0/2749689361 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f215406ae90 0x7f215406d350 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7f216f0b2640 1 --2- 172.21.15.16:0/2749689361 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f21681595a0 0x7f216809c350 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7fd58698a640 1 -- 172.21.15.16:0/1205777432 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd56c06af10 msgr2=0x7fd56c06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7fd58698a640 1 --2- 172.21.15.16:0/1205777432 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd56c06af10 0x7fd56c06d3d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd578009fc0 tx=0x7fd578009550 comp rx=0 tx=0).stop 2023-12-08T08:38:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7fd58698a640 1 -- 172.21.15.16:0/1205777432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd58006d580 msgr2=0x7fd5800acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7fd58698a640 1 --2- 172.21.15.16:0/1205777432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd58006d580 0x7fd5800acf80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd574000c00 tx=0x7fd5740046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7fd58698a640 1 -- 172.21.15.16:0/1205777432 shutdown_connections 2023-12-08T08:38:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7fd58698a640 1 --2- 172.21.15.16:0/1205777432 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd56c06af10 0x7fd56c06d3d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7fd58698a640 1 --2- 172.21.15.16:0/1205777432 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd580161860 0x7fd5800ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7fd58698a640 1 --2- 172.21.15.16:0/1205777432 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd58006d580 0x7fd5800acf80 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd574000c00 tx=0x7fd5740046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7fd58698a640 1 --2- 172.21.15.16:0/1205777432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58006c920 0x7fd580154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7fd58698a640 1 -- 172.21.15.16:0/1205777432 >> 172.21.15.16:0/1205777432 conn(0x7fd5800b09f0 msgr2=0x7fd5800abe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7f216f0b2640 1 --2- 172.21.15.16:0/2749689361 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f21680b31a0 0x7f216809be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7f216f0b2640 1 --2- 172.21.15.16:0/2749689361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f216806d530 0x7f21681ffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7f216f0b2640 1 -- 172.21.15.16:0/2749689361 >> 172.21.15.16:0/2749689361 conn(0x7f21680b09f0 msgr2=0x7f216815bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7feaebfff640 1 -- 172.21.15.16:0/1091263266 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feae4017030 con 0x7feaf406da30 2023-12-08T08:38:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.753+0000 7f216f0b2640 1 -- 172.21.15.16:0/2749689361 shutdown_connections 2023-12-08T08:38:46.761 INFO:teuthology.orchestra.run.smithi016.stdout:55834574859 2023-12-08T08:38:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.754+0000 7f216f0b2640 1 -- 172.21.15.16:0/2749689361 wait complete. 2023-12-08T08:38:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.754+0000 7fd58698a640 1 -- 172.21.15.16:0/1205777432 shutdown_connections 2023-12-08T08:38:46.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.754+0000 7fd58698a640 1 -- 172.21.15.16:0/1205777432 wait complete. 2023-12-08T08:38:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.757+0000 7feaf959e640 1 -- 172.21.15.16:0/1091263266 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fead406ae40 msgr2=0x7fead406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.757+0000 7feaf959e640 1 --2- 172.21.15.16:0/1091263266 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fead406ae40 0x7fead406d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feaf4156d90 tx=0x7feadc007930 comp rx=0 tx=0).stop 2023-12-08T08:38:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.757+0000 7feaf959e640 1 -- 172.21.15.16:0/1091263266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaf406da30 msgr2=0x7feaf4146e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.757+0000 7feaf959e640 1 --2- 172.21.15.16:0/1091263266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaf406da30 0x7feaf4146e50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feae400c440 tx=0x7feae400c900 comp rx=0 tx=0).stop 2023-12-08T08:38:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.757+0000 7feaf959e640 1 -- 172.21.15.16:0/1091263266 shutdown_connections 2023-12-08T08:38:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.757+0000 7feaf959e640 1 --2- 172.21.15.16:0/1091263266 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fead406ae40 0x7fead406d300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.757+0000 7feaf959e640 1 --2- 172.21.15.16:0/1091263266 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaf40b49c0 0x7feaf41eae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.757+0000 7feaf959e640 1 --2- 172.21.15.16:0/1091263266 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaf40b3860 0x7feaf41431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.757+0000 7feaf959e640 1 --2- 172.21.15.16:0/1091263266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaf406da30 0x7feaf4146e50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.757+0000 7feaf959e640 1 -- 172.21.15.16:0/1091263266 >> 172.21.15.16:0/1091263266 conn(0x7feaf4071860 msgr2=0x7feaf4160cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.757+0000 7feaf959e640 1 -- 172.21.15.16:0/1091263266 shutdown_connections 2023-12-08T08:38:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.757+0000 7feaf959e640 1 -- 172.21.15.16:0/1091263266 wait complete. 2023-12-08T08:38:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.759+0000 7fe031ffb640 1 -- 172.21.15.16:0/1101219061 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe0280050c0 con 0x7fe04000aa70 2023-12-08T08:38:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.759+0000 7fe033fff640 1 -- 172.21.15.16:0/1101219061 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe03c076a10 con 0x7fe04000aa70 2023-12-08T08:38:46.764 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-12-08T08:38:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.762+0000 7fe031ffb640 1 -- 172.21.15.16:0/1101219061 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe02c06ae40 msgr2=0x7fe02c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.762+0000 7f9c6359e640 1 -- 172.21.15.16:0/3211212479 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9c48005500 con 0x7f9c5c0540f0 2023-12-08T08:38:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.762+0000 7fe031ffb640 1 --2- 172.21.15.16:0/1101219061 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe02c06ae40 0x7fe02c06d300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe040050b20 tx=0x7fe034008040 comp rx=0 tx=0).stop 2023-12-08T08:38:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.762+0000 7fe031ffb640 1 -- 172.21.15.16:0/1101219061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe04000aa70 msgr2=0x7fe040188a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.762+0000 7fe031ffb640 1 --2- 172.21.15.16:0/1101219061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe04000aa70 0x7fe040188a10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe03c02baa0 tx=0x7fe03c0046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.762+0000 7fe031ffb640 1 -- 172.21.15.16:0/1101219061 shutdown_connections 2023-12-08T08:38:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.762+0000 7fe031ffb640 1 --2- 172.21.15.16:0/1101219061 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe02c06ae40 0x7fe02c06d300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.762+0000 7fe031ffb640 1 --2- 172.21.15.16:0/1101219061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0400df360 0x7fe040039260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.762+0000 7fe031ffb640 1 --2- 172.21.15.16:0/1101219061 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe040053230 0x7fe04004cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.762+0000 7fe031ffb640 1 --2- 172.21.15.16:0/1101219061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe04000aa70 0x7fe040188a10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.762+0000 7fe031ffb640 1 -- 172.21.15.16:0/1101219061 >> 172.21.15.16:0/1101219061 conn(0x7fe040009f80 msgr2=0x7fe04000a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.762+0000 7fe031ffb640 1 -- 172.21.15.16:0/1101219061 shutdown_connections 2023-12-08T08:38:46.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.762+0000 7fe031ffb640 1 -- 172.21.15.16:0/1101219061 wait complete. 2023-12-08T08:38:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.762+0000 7f9c52ffd640 1 -- 172.21.15.16:0/3211212479 <== mon.2 v2:172.21.15.112:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9c58081020 con 0x7f9c5c0540f0 2023-12-08T08:38:46.767 INFO:teuthology.orchestra.run.smithi016.stdout:55834574857 2023-12-08T08:38:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.765+0000 7f9c6359e640 1 -- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9c4406af10 msgr2=0x7f9c4406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.765+0000 7f9c6359e640 1 --2- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9c4406af10 0x7f9c4406d3d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9c5400aae0 tx=0x7f9c54008040 comp rx=0 tx=0).stop 2023-12-08T08:38:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.765+0000 7f9c6359e640 1 -- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0540f0 msgr2=0x7f9c5c0e66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.765+0000 7f9c6359e640 1 --2- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0540f0 0x7f9c5c0e66c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9c58033000 tx=0x7f9c58004730 comp rx=0 tx=0).stop 2023-12-08T08:38:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.765+0000 7f9c6359e640 1 -- 172.21.15.16:0/3211212479 shutdown_connections 2023-12-08T08:38:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.765+0000 7f9c6359e640 1 --2- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9c4406af10 0x7f9c4406d3d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.765+0000 7f9c6359e640 1 --2- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9c5c0e3080 0x7f9c5c0db3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.765+0000 7f9c6359e640 1 --2- 172.21.15.16:0/3211212479 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9c5c0540f0 0x7f9c5c0e66c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.765+0000 7f9c6359e640 1 --2- 172.21.15.16:0/3211212479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c5c03e800 0x7f9c5c052a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.765+0000 7f9c6359e640 1 -- 172.21.15.16:0/3211212479 >> 172.21.15.16:0/3211212479 conn(0x7f9c5c0559a0 msgr2=0x7f9c5c057ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.765+0000 7f9c6359e640 1 -- 172.21.15.16:0/3211212479 shutdown_connections 2023-12-08T08:38:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.765+0000 7f9c6359e640 1 -- 172.21.15.16:0/3211212479 wait complete. 2023-12-08T08:38:46.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2023-12-08T08:38:46.775 DEBUG:teuthology.parallel:result is None 2023-12-08T08:38:46.777 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2023-12-08T08:38:46.777 DEBUG:teuthology.parallel:result is None 2023-12-08T08:38:46.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2023-12-08T08:38:46.778 DEBUG:teuthology.parallel:result is None 2023-12-08T08:38:46.780 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2023-12-08T08:38:46.780 DEBUG:teuthology.parallel:result is None 2023-12-08T08:38:46.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2023-12-08T08:38:46.783 DEBUG:teuthology.parallel:result is None 2023-12-08T08:38:46.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2023-12-08T08:38:46.783 DEBUG:teuthology.parallel:result is None 2023-12-08T08:38:46.784 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-08T08:38:46.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-08T08:38:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.926+0000 7f4131089640 1 Processor -- start 2023-12-08T08:38:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.927+0000 7f4131089640 1 -- start start 2023-12-08T08:38:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.927+0000 7f4131089640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f412c0ce3e0 0x7f412c0b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.927+0000 7f4131089640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412c070db0 con 0x7f412c140a10 2023-12-08T08:38:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.927+0000 7f4131089640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412c071090 con 0x7f412c071e50 2023-12-08T08:38:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.927+0000 7f4131089640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412c070ba0 con 0x7f412c0ce3e0 2023-12-08T08:38:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.927+0000 7f412ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f412c140a10 0x7f412c0cdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37496/0 (socket says 172.21.15.16:37496) 2023-12-08T08:38:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.927+0000 7f412ad9d640 1 -- 172.21.15.16:0/3799783273 learned_addr learned my addr 172.21.15.16:0/3799783273 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:46.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f412a59c640 1 --2- 172.21.15.16:0/3799783273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f412c0ce3e0 0x7f412c0b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 -- 172.21.15.16:0/3799783273 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414136456 0 0) 0x7f412c070db0 con 0x7f412c140a10 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 -- 172.21.15.16:0/3799783273 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4114003520 con 0x7f412c140a10 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 -- 172.21.15.16:0/3799783273 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2819729100 0 0) 0x7f4114003520 con 0x7f412c140a10 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 -- 172.21.15.16:0/3799783273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f412c0ce3e0 msgr2=0x7f412c0b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 --2- 172.21.15.16:0/3799783273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f412c0ce3e0 0x7f412c0b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 -- 172.21.15.16:0/3799783273 >> v1:172.21.15.112:6789/0 conn(0x7f412c071e50 legacy=0x7f412c06c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 -- 172.21.15.16:0/3799783273 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c071370 con 0x7f412c140a10 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 -- 172.21.15.16:0/3799783273 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f4124001670 con 0x7f412c140a10 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 -- 172.21.15.16:0/3799783273 >> v1:172.21.15.16:6789/0 conn(0x7f412c140a10 legacy=0x7f412c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 --2- 172.21.15.16:0/3799783273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41140039a0 0x7f4114003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 --2- 172.21.15.16:0/3799783273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f412c0ce3e0 0x7f41140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 --2- 172.21.15.16:0/3799783273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4114004d00 0x7f4114005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 -- 172.21.15.16:0/3799783273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c070ba0 con 0x7f4114004d00 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 -- 172.21.15.16:0/3799783273 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411400bac0 con 0x7f41140039a0 2023-12-08T08:38:46.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f4129d9b640 1 -- 172.21.15.16:0/3799783273 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411400bc80 con 0x7f412c0ce3e0 2023-12-08T08:38:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f412b59e640 1 --2- 172.21.15.16:0/3799783273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4114004d00 0x7f4114005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f412ad9d640 1 --2- 172.21.15.16:0/3799783273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41140039a0 0x7f4114003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.928+0000 7f412a59c640 1 --2- 172.21.15.16:0/3799783273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f412c0ce3e0 0x7f41140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.929+0000 7f412b59e640 1 -- 172.21.15.16:0/3799783273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f412c0ce3e0 msgr2=0x7f41140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.929+0000 7f412b59e640 1 --2- 172.21.15.16:0/3799783273 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f412c0ce3e0 0x7f41140047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.929+0000 7f412b59e640 1 -- 172.21.15.16:0/3799783273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41140039a0 msgr2=0x7f4114003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.929+0000 7f412b59e640 1 --2- 172.21.15.16:0/3799783273 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41140039a0 0x7f4114003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.929+0000 7f412b59e640 1 -- 172.21.15.16:0/3799783273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4114003520 con 0x7f4114004d00 2023-12-08T08:38:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.929+0000 7f412b59e640 1 --2- 172.21.15.16:0/3799783273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4114004d00 0x7f4114005180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f411c001e20 tx=0x7f41140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67790b89d9e664a0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.929+0000 7f4129d9b640 1 -- 172.21.15.16:0/3799783273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c003b70 con 0x7f4114004d00 2023-12-08T08:38:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.929+0000 7f4129d9b640 1 -- 172.21.15.16:0/3799783273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c003cd0 con 0x7f4114004d00 2023-12-08T08:38:46.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.929+0000 7f4129d9b640 1 -- 172.21.15.16:0/3799783273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c005310 con 0x7f4114004d00 2023-12-08T08:38:46.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.930+0000 7f4131089640 1 -- 172.21.15.16:0/3799783273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4114004d00 msgr2=0x7f4114005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.930+0000 7f4131089640 1 --2- 172.21.15.16:0/3799783273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4114004d00 0x7f4114005180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f411c001e20 tx=0x7f41140057f0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.930+0000 7f412b59e640 1 -- 172.21.15.16:0/3799783273 reap_dead start 2023-12-08T08:38:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.930+0000 7f4131089640 1 -- 172.21.15.16:0/3799783273 shutdown_connections 2023-12-08T08:38:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.930+0000 7f4131089640 1 -- 172.21.15.16:0/3799783273 >> 172.21.15.16:0/3799783273 conn(0x7f412c0b09f0 msgr2=0x7f412c0b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.930+0000 7f4131089640 1 -- 172.21.15.16:0/3799783273 shutdown_connections 2023-12-08T08:38:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.930+0000 7f4131089640 1 -- 172.21.15.16:0/3799783273 wait complete. 2023-12-08T08:38:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.931+0000 7f4131089640 1 Processor -- start 2023-12-08T08:38:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.932+0000 7f4131089640 1 -- start start 2023-12-08T08:38:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.932+0000 7f4131089640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f412c09be90 0x7f412c0af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.932+0000 7f4131089640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c09c270 0x7f412c15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.932+0000 7f4131089640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f412c158250 0x7f412c1586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.932+0000 7f4131089640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c15b530 con 0x7f412c09c270 2023-12-08T08:38:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.932+0000 7f4131089640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c15c7c0 con 0x7f412c158250 2023-12-08T08:38:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.932+0000 7f4131089640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412c158c20 con 0x7f412c09be90 2023-12-08T08:38:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.932+0000 7f412a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c09c270 0x7f412c15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.932+0000 7f412a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c09c270 0x7f412c15aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53474/0 (socket says 172.21.15.16:53474) 2023-12-08T08:38:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.932+0000 7f412a59c640 1 -- 172.21.15.16:0/3992263408 learned_addr learned my addr 172.21.15.16:0/3992263408 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.932+0000 7f412ad9d640 1 --2- 172.21.15.16:0/3992263408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f412c09be90 0x7f412c0af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.932+0000 7f412b59e640 1 --2- 172.21.15.16:0/3992263408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f412c158250 0x7f412c1586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.933+0000 7f412a59c640 1 -- 172.21.15.16:0/3992263408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f412c09be90 msgr2=0x7f412c0af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.933+0000 7f412a59c640 1 --2- 172.21.15.16:0/3992263408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f412c09be90 0x7f412c0af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.933+0000 7f412a59c640 1 -- 172.21.15.16:0/3992263408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f412c158250 msgr2=0x7f412c1586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.933+0000 7f412a59c640 1 --2- 172.21.15.16:0/3992263408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f412c158250 0x7f412c1586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.933+0000 7f412a59c640 1 -- 172.21.15.16:0/3992263408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c15b720 con 0x7f412c09c270 2023-12-08T08:38:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.933+0000 7f412a59c640 1 --2- 172.21.15.16:0/3992263408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c09c270 0x7f412c15aff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4120011940 tx=0x7f4120011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.933+0000 7f41137fe640 1 -- 172.21.15.16:0/3992263408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120004240 con 0x7f412c09c270 2023-12-08T08:38:46.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.933+0000 7f41137fe640 1 -- 172.21.15.16:0/3992263408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41200043a0 con 0x7f412c09c270 2023-12-08T08:38:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.933+0000 7f4131089640 1 -- 172.21.15.16:0/3992263408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412c15ba00 con 0x7f412c09c270 2023-12-08T08:38:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.933+0000 7f41137fe640 1 -- 172.21.15.16:0/3992263408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412000faa0 con 0x7f412c09c270 2023-12-08T08:38:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.933+0000 7f4131089640 1 -- 172.21.15.16:0/3992263408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412c15bfc0 con 0x7f412c09c270 2023-12-08T08:38:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.934+0000 7f4131089640 1 -- 172.21.15.16:0/3992263408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412c067430 con 0x7f412c09c270 2023-12-08T08:38:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.935+0000 7f41137fe640 1 -- 172.21.15.16:0/3992263408 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f412000fc00 con 0x7f412c09c270 2023-12-08T08:38:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.935+0000 7f41137fe640 1 --2- 172.21.15.16:0/3992263408 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f411806ae40 0x7f411806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.935+0000 7f41137fe640 1 -- 172.21.15.16:0/3992263408 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4120089e50 con 0x7f412c09c270 2023-12-08T08:38:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.935+0000 7f412ad9d640 1 --2- 172.21.15.16:0/3992263408 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f411806ae40 0x7f411806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.936+0000 7f412ad9d640 1 --2- 172.21.15.16:0/3992263408 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f411806ae40 0x7f411806d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f412c0b3ec0 tx=0x7f4124000f80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:46.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:46.939+0000 7f41137fe640 1 -- 172.21.15.16:0/3992263408 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4120054cc0 con 0x7f412c09c270 2023-12-08T08:38:47.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.110+0000 7f4131089640 1 -- 172.21.15.16:0/3992263408 --> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f412c071930 con 0x7f411806ae40 2023-12-08T08:38:47.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.119+0000 7f41137fe640 1 -- 172.21.15.16:0/3992263408 <== mgr.4111 v2:172.21.15.112:6832/804067392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345244 (secure 0 0 0) 0x7f4124004d30 con 0x7f411806ae40 2023-12-08T08:38:47.125 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:47.127 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-12-08T08:38:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.123+0000 7f4131089640 1 -- 172.21.15.16:0/3992263408 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f411806ae40 msgr2=0x7f411806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.123+0000 7f4131089640 1 --2- 172.21.15.16:0/3992263408 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f411806ae40 0x7f411806d300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f412c0b3ec0 tx=0x7f4124000f80 comp rx=0 tx=0).stop 2023-12-08T08:38:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.123+0000 7f4131089640 1 -- 172.21.15.16:0/3992263408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c09c270 msgr2=0x7f412c15aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.123+0000 7f4131089640 1 --2- 172.21.15.16:0/3992263408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c09c270 0x7f412c15aff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4120011940 tx=0x7f4120011e00 comp rx=0 tx=0).stop 2023-12-08T08:38:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.123+0000 7f4131089640 1 -- 172.21.15.16:0/3992263408 shutdown_connections 2023-12-08T08:38:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.123+0000 7f4131089640 1 --2- 172.21.15.16:0/3992263408 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f411806ae40 0x7f411806d300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.123+0000 7f4131089640 1 --2- 172.21.15.16:0/3992263408 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f412c158250 0x7f412c1586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.123+0000 7f4131089640 1 --2- 172.21.15.16:0/3992263408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f412c09c270 0x7f412c15aff0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.123+0000 7f4131089640 1 --2- 172.21.15.16:0/3992263408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f412c09be90 0x7f412c0af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.123+0000 7f4131089640 1 -- 172.21.15.16:0/3992263408 >> 172.21.15.16:0/3992263408 conn(0x7f412c0b09f0 msgr2=0x7f412c0add50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.123+0000 7f4131089640 1 -- 172.21.15.16:0/3992263408 shutdown_connections 2023-12-08T08:38:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.123+0000 7f4131089640 1 -- 172.21.15.16:0/3992263408 wait complete. 2023-12-08T08:38:47.136 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-12-08T08:38:46.981252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231864,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742856,"statfs":{"total":773094113280,"available":772856684544,"internally_reserved":0,"allocated":3284992,"data_stored":1988346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003518"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659961+0000","last_change":"2023-12-08T08:38:18.634619+0000","last_active":"2023-12-08T08:38:23.659961+0000","last_peered":"2023-12-08T08:38:23.659961+0000","last_clean":"2023-12-08T08:38:23.659961+0000","last_became_active":"2023-12-08T08:38:18.634328+0000","last_became_peered":"2023-12-08T08:38:18.634328+0000","last_unstale":"2023-12-08T08:38:23.659961+0000","last_undegraded":"2023-12-08T08:38:23.659961+0000","last_fullsized":"2023-12-08T08:38:23.659961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:11.067589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656122+0000","last_change":"2023-12-08T08:38:19.627512+0000","last_active":"2023-12-08T08:38:23.656122+0000","last_peered":"2023-12-08T08:38:23.656122+0000","last_clean":"2023-12-08T08:38:23.656122+0000","last_became_active":"2023-12-08T08:38:19.625456+0000","last_became_peered":"2023-12-08T08:38:19.625456+0000","last_unstale":"2023-12-08T08:38:23.656122+0000","last_undegraded":"2023-12-08T08:38:23.656122+0000","last_fullsized":"2023-12-08T08:38:23.656122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576137+0000","last_change":"2023-12-08T08:38:23.856216+0000","last_active":"2023-12-08T08:38:25.576137+0000","last_peered":"2023-12-08T08:38:25.576137+0000","last_clean":"2023-12-08T08:38:25.576137+0000","last_became_active":"2023-12-08T08:38:23.855623+0000","last_became_peered":"2023-12-08T08:38:23.855623+0000","last_unstale":"2023-12-08T08:38:25.576137+0000","last_undegraded":"2023-12-08T08:38:25.576137+0000","last_fullsized":"2023-12-08T08:38:25.576137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:26:16.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656314+0000","last_change":"2023-12-08T08:38:18.631598+0000","last_active":"2023-12-08T08:38:23.656314+0000","last_peered":"2023-12-08T08:38:23.656314+0000","last_clean":"2023-12-08T08:38:23.656314+0000","last_became_active":"2023-12-08T08:38:18.631295+0000","last_became_peered":"2023-12-08T08:38:18.631295+0000","last_unstale":"2023-12-08T08:38:23.656314+0000","last_undegraded":"2023-12-08T08:38:23.656314+0000","last_fullsized":"2023-12-08T08:38:23.656314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659744+0000","last_change":"2023-12-08T08:38:19.633463+0000","last_active":"2023-12-08T08:38:23.659744+0000","last_peered":"2023-12-08T08:38:23.659744+0000","last_clean":"2023-12-08T08:38:23.659744+0000","last_became_active":"2023-12-08T08:38:19.633094+0000","last_became_peered":"2023-12-08T08:38:19.633094+0000","last_unstale":"2023-12-08T08:38:23.659744+0000","last_undegraded":"2023-12-08T08:38:23.659744+0000","last_fullsized":"2023-12-08T08:38:23.659744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608661+0000","last_change":"2023-12-08T08:38:23.674366+0000","last_active":"2023-12-08T08:38:24.608661+0000","last_peered":"2023-12-08T08:38:24.608661+0000","last_clean":"2023-12-08T08:38:24.608661+0000","last_became_active":"2023-12-08T08:38:23.673810+0000","last_became_peered":"2023-12-08T08:38:23.673810+0000","last_unstale":"2023-12-08T08:38:24.608661+0000","last_undegraded":"2023-12-08T08:38:24.608661+0000","last_fullsized":"2023-12-08T08:38:24.608661+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:47.374495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852405+0000","last_change":"2023-12-08T08:38:18.633940+0000","last_active":"2023-12-08T08:38:23.852405+0000","last_peered":"2023-12-08T08:38:23.852405+0000","last_clean":"2023-12-08T08:38:23.852405+0000","last_became_active":"2023-12-08T08:38:18.631964+0000","last_became_peered":"2023-12-08T08:38:18.631964+0000","last_unstale":"2023-12-08T08:38:23.852405+0000","last_undegraded":"2023-12-08T08:38:23.852405+0000","last_fullsized":"2023-12-08T08:38:23.852405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:37.317712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661342+0000","last_change":"2023-12-08T08:38:19.627445+0000","last_active":"2023-12-08T08:38:23.661342+0000","last_peered":"2023-12-08T08:38:23.661342+0000","last_clean":"2023-12-08T08:38:23.661342+0000","last_became_active":"2023-12-08T08:38:19.627174+0000","last_became_peered":"2023-12-08T08:38:19.627174+0000","last_unstale":"2023-12-08T08:38:23.661342+0000","last_undegraded":"2023-12-08T08:38:23.661342+0000","last_fullsized":"2023-12-08T08:38:23.661342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:32.116348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609171+0000","last_change":"2023-12-08T08:38:23.674338+0000","last_active":"2023-12-08T08:38:24.609171+0000","last_peered":"2023-12-08T08:38:24.609171+0000","last_clean":"2023-12-08T08:38:24.609171+0000","last_became_active":"2023-12-08T08:38:23.673614+0000","last_became_peered":"2023-12-08T08:38:23.673614+0000","last_unstale":"2023-12-08T08:38:24.609171+0000","last_undegraded":"2023-12-08T08:38:24.609171+0000","last_fullsized":"2023-12-08T08:38:24.609171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:59.405196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852122+0000","last_change":"2023-12-08T08:38:18.633561+0000","last_active":"2023-12-08T08:38:23.852122+0000","last_peered":"2023-12-08T08:38:23.852122+0000","last_clean":"2023-12-08T08:38:23.852122+0000","last_became_active":"2023-12-08T08:38:18.632646+0000","last_became_peered":"2023-12-08T08:38:18.632646+0000","last_unstale":"2023-12-08T08:38:23.852122+0000","last_undegraded":"2023-12-08T08:38:23.852122+0000","last_fullsized":"2023-12-08T08:38:23.852122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:36:34.339195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662141+0000","last_change":"2023-12-08T08:38:19.632216+0000","last_active":"2023-12-08T08:38:23.662141+0000","last_peered":"2023-12-08T08:38:23.662141+0000","last_clean":"2023-12-08T08:38:23.662141+0000","last_became_active":"2023-12-08T08:38:19.631599+0000","last_became_peered":"2023-12-08T08:38:19.631599+0000","last_unstale":"2023-12-08T08:38:23.662141+0000","last_undegraded":"2023-12-08T08:38:23.662141+0000","last_fullsized":"2023-12-08T08:38:23.662141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:11.797914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608922+0000","last_change":"2023-12-08T08:38:23.669631+0000","last_active":"2023-12-08T08:38:24.608922+0000","last_peered":"2023-12-08T08:38:24.608922+0000","last_clean":"2023-12-08T08:38:24.608922+0000","last_became_active":"2023-12-08T08:38:23.669222+0000","last_became_peered":"2023-12-08T08:38:23.669222+0000","last_unstale":"2023-12-08T08:38:24.608922+0000","last_undegraded":"2023-12-08T08:38:24.608922+0000","last_fullsized":"2023-12-08T08:38:24.608922+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:49:33.936211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664041+0000","last_change":"2023-12-08T08:38:19.620724+0000","last_active":"2023-12-08T08:38:23.664041+0000","last_peered":"2023-12-08T08:38:23.664041+0000","last_clean":"2023-12-08T08:38:23.664041+0000","last_became_active":"2023-12-08T08:38:19.620366+0000","last_became_peered":"2023-12-08T08:38:19.620366+0000","last_unstale":"2023-12-08T08:38:23.664041+0000","last_undegraded":"2023-12-08T08:38:23.664041+0000","last_fullsized":"2023-12-08T08:38:23.664041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:00:01.146409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852136+0000","last_change":"2023-12-08T08:38:18.632086+0000","last_active":"2023-12-08T08:38:23.852136+0000","last_peered":"2023-12-08T08:38:23.852136+0000","last_clean":"2023-12-08T08:38:23.852136+0000","last_became_active":"2023-12-08T08:38:18.631756+0000","last_became_peered":"2023-12-08T08:38:18.631756+0000","last_unstale":"2023-12-08T08:38:23.852136+0000","last_undegraded":"2023-12-08T08:38:23.852136+0000","last_fullsized":"2023-12-08T08:38:23.852136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:14.342540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609352+0000","last_change":"2023-12-08T08:38:23.673898+0000","last_active":"2023-12-08T08:38:24.609352+0000","last_peered":"2023-12-08T08:38:24.609352+0000","last_clean":"2023-12-08T08:38:24.609352+0000","last_became_active":"2023-12-08T08:38:23.673389+0000","last_became_peered":"2023-12-08T08:38:23.673389+0000","last_unstale":"2023-12-08T08:38:24.609352+0000","last_undegraded":"2023-12-08T08:38:24.609352+0000","last_fullsized":"2023-12-08T08:38:24.609352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:15.835775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655652+0000","last_change":"2023-12-08T08:38:19.627148+0000","last_active":"2023-12-08T08:38:23.655652+0000","last_peered":"2023-12-08T08:38:23.655652+0000","last_clean":"2023-12-08T08:38:23.655652+0000","last_became_active":"2023-12-08T08:38:19.626846+0000","last_became_peered":"2023-12-08T08:38:19.626846+0000","last_unstale":"2023-12-08T08:38:23.655652+0000","last_undegraded":"2023-12-08T08:38:23.655652+0000","last_fullsized":"2023-12-08T08:38:23.655652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.666654+0000","last_change":"2023-12-08T08:38:18.632191+0000","last_active":"2023-12-08T08:38:23.666654+0000","last_peered":"2023-12-08T08:38:23.666654+0000","last_clean":"2023-12-08T08:38:23.666654+0000","last_became_active":"2023-12-08T08:38:18.631927+0000","last_became_peered":"2023-12-08T08:38:18.631927+0000","last_unstale":"2023-12-08T08:38:23.666654+0000","last_undegraded":"2023-12-08T08:38:23.666654+0000","last_fullsized":"2023-12-08T08:38:23.666654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:10.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608968+0000","last_change":"2023-12-08T08:38:23.678875+0000","last_active":"2023-12-08T08:38:24.608968+0000","last_peered":"2023-12-08T08:38:24.608968+0000","last_clean":"2023-12-08T08:38:24.608968+0000","last_became_active":"2023-12-08T08:38:23.678332+0000","last_became_peered":"2023-12-08T08:38:23.678332+0000","last_unstale":"2023-12-08T08:38:24.608968+0000","last_undegraded":"2023-12-08T08:38:24.608968+0000","last_fullsized":"2023-12-08T08:38:24.608968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:36.154208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656075+0000","last_change":"2023-12-08T08:38:19.625657+0000","last_active":"2023-12-08T08:38:23.656075+0000","last_peered":"2023-12-08T08:38:23.656075+0000","last_clean":"2023-12-08T08:38:23.656075+0000","last_became_active":"2023-12-08T08:38:19.624790+0000","last_became_peered":"2023-12-08T08:38:19.624790+0000","last_unstale":"2023-12-08T08:38:23.656075+0000","last_undegraded":"2023-12-08T08:38:23.656075+0000","last_fullsized":"2023-12-08T08:38:23.656075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:32.116348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664683+0000","last_change":"2023-12-08T08:38:18.637627+0000","last_active":"2023-12-08T08:38:23.664683+0000","last_peered":"2023-12-08T08:38:23.664683+0000","last_clean":"2023-12-08T08:38:23.664683+0000","last_became_active":"2023-12-08T08:38:18.637335+0000","last_became_peered":"2023-12-08T08:38:18.637335+0000","last_unstale":"2023-12-08T08:38:23.664683+0000","last_undegraded":"2023-12-08T08:38:23.664683+0000","last_fullsized":"2023-12-08T08:38:23.664683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.828080+0000","last_change":"2023-12-08T08:38:23.852913+0000","last_active":"2023-12-08T08:38:25.828080+0000","last_peered":"2023-12-08T08:38:25.828080+0000","last_clean":"2023-12-08T08:38:25.828080+0000","last_became_active":"2023-12-08T08:38:23.852508+0000","last_became_peered":"2023-12-08T08:38:23.852508+0000","last_unstale":"2023-12-08T08:38:25.828080+0000","last_undegraded":"2023-12-08T08:38:25.828080+0000","last_fullsized":"2023-12-08T08:38:25.828080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:48.357070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658598+0000","last_change":"2023-12-08T08:38:19.623518+0000","last_active":"2023-12-08T08:38:23.658598+0000","last_peered":"2023-12-08T08:38:23.658598+0000","last_clean":"2023-12-08T08:38:23.658598+0000","last_became_active":"2023-12-08T08:38:19.623240+0000","last_became_peered":"2023-12-08T08:38:19.623240+0000","last_unstale":"2023-12-08T08:38:23.658598+0000","last_undegraded":"2023-12-08T08:38:23.658598+0000","last_fullsized":"2023-12-08T08:38:23.658598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:43.773215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851377+0000","last_change":"2023-12-08T08:38:18.633585+0000","last_active":"2023-12-08T08:38:23.851377+0000","last_peered":"2023-12-08T08:38:23.851377+0000","last_clean":"2023-12-08T08:38:23.851377+0000","last_became_active":"2023-12-08T08:38:18.633112+0000","last_became_peered":"2023-12-08T08:38:18.633112+0000","last_unstale":"2023-12-08T08:38:23.851377+0000","last_undegraded":"2023-12-08T08:38:23.851377+0000","last_fullsized":"2023-12-08T08:38:23.851377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:42.768579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609486+0000","last_change":"2023-12-08T08:38:23.678810+0000","last_active":"2023-12-08T08:38:24.609486+0000","last_peered":"2023-12-08T08:38:24.609486+0000","last_clean":"2023-12-08T08:38:24.609486+0000","last_became_active":"2023-12-08T08:38:23.678331+0000","last_became_peered":"2023-12-08T08:38:23.678331+0000","last_unstale":"2023-12-08T08:38:24.609486+0000","last_undegraded":"2023-12-08T08:38:24.609486+0000","last_fullsized":"2023-12-08T08:38:24.609486+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:17.394710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.666222+0000","last_change":"2023-12-08T08:38:19.619485+0000","last_active":"2023-12-08T08:38:23.666222+0000","last_peered":"2023-12-08T08:38:23.666222+0000","last_clean":"2023-12-08T08:38:23.666222+0000","last_became_active":"2023-12-08T08:38:19.618880+0000","last_became_peered":"2023-12-08T08:38:19.618880+0000","last_unstale":"2023-12-08T08:38:23.666222+0000","last_undegraded":"2023-12-08T08:38:23.666222+0000","last_fullsized":"2023-12-08T08:38:23.666222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:43.336635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665379+0000","last_change":"2023-12-08T08:38:18.633093+0000","last_active":"2023-12-08T08:38:23.665379+0000","last_peered":"2023-12-08T08:38:23.665379+0000","last_clean":"2023-12-08T08:38:23.665379+0000","last_became_active":"2023-12-08T08:38:18.632832+0000","last_became_peered":"2023-12-08T08:38:18.632832+0000","last_unstale":"2023-12-08T08:38:23.665379+0000","last_undegraded":"2023-12-08T08:38:23.665379+0000","last_fullsized":"2023-12-08T08:38:23.665379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:37.317712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608658+0000","last_change":"2023-12-08T08:38:23.853769+0000","last_active":"2023-12-08T08:38:24.608658+0000","last_peered":"2023-12-08T08:38:24.608658+0000","last_clean":"2023-12-08T08:38:24.608658+0000","last_became_active":"2023-12-08T08:38:23.853211+0000","last_became_peered":"2023-12-08T08:38:23.853211+0000","last_unstale":"2023-12-08T08:38:24.608658+0000","last_undegraded":"2023-12-08T08:38:24.608658+0000","last_fullsized":"2023-12-08T08:38:24.608658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:59:15.111403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656171+0000","last_change":"2023-12-08T08:38:19.626321+0000","last_active":"2023-12-08T08:38:23.656171+0000","last_peered":"2023-12-08T08:38:23.656171+0000","last_clean":"2023-12-08T08:38:23.656171+0000","last_became_active":"2023-12-08T08:38:19.624741+0000","last_became_peered":"2023-12-08T08:38:19.624741+0000","last_unstale":"2023-12-08T08:38:23.656171+0000","last_undegraded":"2023-12-08T08:38:23.656171+0000","last_fullsized":"2023-12-08T08:38:23.656171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:49:29.898351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664728+0000","last_change":"2023-12-08T08:38:18.636600+0000","last_active":"2023-12-08T08:38:23.664728+0000","last_peered":"2023-12-08T08:38:23.664728+0000","last_clean":"2023-12-08T08:38:23.664728+0000","last_became_active":"2023-12-08T08:38:18.633400+0000","last_became_peered":"2023-12-08T08:38:18.633400+0000","last_unstale":"2023-12-08T08:38:23.664728+0000","last_undegraded":"2023-12-08T08:38:23.664728+0000","last_fullsized":"2023-12-08T08:38:23.664728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:37.317712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576086+0000","last_change":"2023-12-08T08:38:23.856023+0000","last_active":"2023-12-08T08:38:25.576086+0000","last_peered":"2023-12-08T08:38:25.576086+0000","last_clean":"2023-12-08T08:38:25.576086+0000","last_became_active":"2023-12-08T08:38:23.855599+0000","last_became_peered":"2023-12-08T08:38:23.855599+0000","last_unstale":"2023-12-08T08:38:25.576086+0000","last_undegraded":"2023-12-08T08:38:25.576086+0000","last_fullsized":"2023-12-08T08:38:25.576086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T20:21:44.003574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662450+0000","last_change":"2023-12-08T08:38:19.622140+0000","last_active":"2023-12-08T08:38:23.662450+0000","last_peered":"2023-12-08T08:38:23.662450+0000","last_clean":"2023-12-08T08:38:23.662450+0000","last_became_active":"2023-12-08T08:38:19.621784+0000","last_became_peered":"2023-12-08T08:38:19.621784+0000","last_unstale":"2023-12-08T08:38:23.662450+0000","last_undegraded":"2023-12-08T08:38:23.662450+0000","last_fullsized":"2023-12-08T08:38:23.662450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852845+0000","last_change":"2023-12-08T08:38:18.634123+0000","last_active":"2023-12-08T08:38:23.852845+0000","last_peered":"2023-12-08T08:38:23.852845+0000","last_clean":"2023-12-08T08:38:23.852845+0000","last_became_active":"2023-12-08T08:38:18.632874+0000","last_became_peered":"2023-12-08T08:38:18.632874+0000","last_unstale":"2023-12-08T08:38:23.852845+0000","last_undegraded":"2023-12-08T08:38:23.852845+0000","last_fullsized":"2023-12-08T08:38:23.852845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:07.497079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.326786+0000","last_change":"2023-12-08T08:38:23.852386+0000","last_active":"2023-12-08T08:38:26.326786+0000","last_peered":"2023-12-08T08:38:26.326786+0000","last_clean":"2023-12-08T08:38:26.326786+0000","last_became_active":"2023-12-08T08:38:23.851973+0000","last_became_peered":"2023-12-08T08:38:23.851973+0000","last_unstale":"2023-12-08T08:38:26.326786+0000","last_undegraded":"2023-12-08T08:38:26.326786+0000","last_fullsized":"2023-12-08T08:38:26.326786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:22.372635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662330+0000","last_change":"2023-12-08T08:38:19.631928+0000","last_active":"2023-12-08T08:38:23.662330+0000","last_peered":"2023-12-08T08:38:23.662330+0000","last_clean":"2023-12-08T08:38:23.662330+0000","last_became_active":"2023-12-08T08:38:19.631447+0000","last_became_peered":"2023-12-08T08:38:19.631447+0000","last_unstale":"2023-12-08T08:38:23.662330+0000","last_undegraded":"2023-12-08T08:38:23.662330+0000","last_fullsized":"2023-12-08T08:38:23.662330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:13.356849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662548+0000","last_change":"2023-12-08T08:38:18.634530+0000","last_active":"2023-12-08T08:38:23.662548+0000","last_peered":"2023-12-08T08:38:23.662548+0000","last_clean":"2023-12-08T08:38:23.662548+0000","last_became_active":"2023-12-08T08:38:18.634239+0000","last_became_peered":"2023-12-08T08:38:18.634239+0000","last_unstale":"2023-12-08T08:38:23.662548+0000","last_undegraded":"2023-12-08T08:38:23.662548+0000","last_fullsized":"2023-12-08T08:38:23.662548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:42.768579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.827628+0000","last_change":"2023-12-08T08:38:23.676640+0000","last_active":"2023-12-08T08:38:25.827628+0000","last_peered":"2023-12-08T08:38:25.827628+0000","last_clean":"2023-12-08T08:38:25.827628+0000","last_became_active":"2023-12-08T08:38:23.675999+0000","last_became_peered":"2023-12-08T08:38:23.675999+0000","last_unstale":"2023-12-08T08:38:25.827628+0000","last_undegraded":"2023-12-08T08:38:25.827628+0000","last_fullsized":"2023-12-08T08:38:25.827628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:15.835775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658939+0000","last_change":"2023-12-08T08:38:19.615261+0000","last_active":"2023-12-08T08:38:23.658939+0000","last_peered":"2023-12-08T08:38:23.658939+0000","last_clean":"2023-12-08T08:38:23.658939+0000","last_became_active":"2023-12-08T08:38:19.614990+0000","last_became_peered":"2023-12-08T08:38:19.614990+0000","last_unstale":"2023-12-08T08:38:23.658939+0000","last_undegraded":"2023-12-08T08:38:23.658939+0000","last_fullsized":"2023-12-08T08:38:23.658939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:05:08.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662040+0000","last_change":"2023-12-08T08:38:18.630062+0000","last_active":"2023-12-08T08:38:23.662040+0000","last_peered":"2023-12-08T08:38:23.662040+0000","last_clean":"2023-12-08T08:38:23.662040+0000","last_became_active":"2023-12-08T08:38:18.629798+0000","last_became_peered":"2023-12-08T08:38:18.629798+0000","last_unstale":"2023-12-08T08:38:23.662040+0000","last_undegraded":"2023-12-08T08:38:23.662040+0000","last_fullsized":"2023-12-08T08:38:23.662040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:13.807850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.827791+0000","last_change":"2023-12-08T08:38:23.674624+0000","last_active":"2023-12-08T08:38:25.827791+0000","last_peered":"2023-12-08T08:38:25.827791+0000","last_clean":"2023-12-08T08:38:25.827791+0000","last_became_active":"2023-12-08T08:38:23.674121+0000","last_became_peered":"2023-12-08T08:38:23.674121+0000","last_unstale":"2023-12-08T08:38:25.827791+0000","last_undegraded":"2023-12-08T08:38:25.827791+0000","last_fullsized":"2023-12-08T08:38:25.827791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:17.394710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665266+0000","last_change":"2023-12-08T08:38:19.621679+0000","last_active":"2023-12-08T08:38:23.665266+0000","last_peered":"2023-12-08T08:38:23.665266+0000","last_clean":"2023-12-08T08:38:23.665266+0000","last_became_active":"2023-12-08T08:38:19.621388+0000","last_became_peered":"2023-12-08T08:38:19.621388+0000","last_unstale":"2023-12-08T08:38:23.665266+0000","last_undegraded":"2023-12-08T08:38:23.665266+0000","last_fullsized":"2023-12-08T08:38:23.665266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662810+0000","last_change":"2023-12-08T08:38:18.632407+0000","last_active":"2023-12-08T08:38:23.662810+0000","last_peered":"2023-12-08T08:38:23.662810+0000","last_clean":"2023-12-08T08:38:23.662810+0000","last_became_active":"2023-12-08T08:38:18.632115+0000","last_became_peered":"2023-12-08T08:38:18.632115+0000","last_unstale":"2023-12-08T08:38:23.662810+0000","last_undegraded":"2023-12-08T08:38:23.662810+0000","last_fullsized":"2023-12-08T08:38:23.662810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:05:14.591351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.828126+0000","last_change":"2023-12-08T08:38:23.675339+0000","last_active":"2023-12-08T08:38:25.828126+0000","last_peered":"2023-12-08T08:38:25.828126+0000","last_clean":"2023-12-08T08:38:25.828126+0000","last_became_active":"2023-12-08T08:38:23.674249+0000","last_became_peered":"2023-12-08T08:38:23.674249+0000","last_unstale":"2023-12-08T08:38:25.828126+0000","last_undegraded":"2023-12-08T08:38:25.828126+0000","last_fullsized":"2023-12-08T08:38:25.828126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:34:29.925547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.660253+0000","last_change":"2023-12-08T08:38:19.611646+0000","last_active":"2023-12-08T08:38:23.660253+0000","last_peered":"2023-12-08T08:38:23.660253+0000","last_clean":"2023-12-08T08:38:23.660253+0000","last_became_active":"2023-12-08T08:38:19.611315+0000","last_became_peered":"2023-12-08T08:38:19.611315+0000","last_unstale":"2023-12-08T08:38:23.660253+0000","last_undegraded":"2023-12-08T08:38:23.660253+0000","last_fullsized":"2023-12-08T08:38:23.660253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:11.612354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851317+0000","last_change":"2023-12-08T08:38:18.634473+0000","last_active":"2023-12-08T08:38:23.851317+0000","last_peered":"2023-12-08T08:38:23.851317+0000","last_clean":"2023-12-08T08:38:23.851317+0000","last_became_active":"2023-12-08T08:38:18.633755+0000","last_became_peered":"2023-12-08T08:38:18.633755+0000","last_unstale":"2023-12-08T08:38:23.851317+0000","last_undegraded":"2023-12-08T08:38:23.851317+0000","last_fullsized":"2023-12-08T08:38:23.851317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:11.067589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.327085+0000","last_change":"2023-12-08T08:38:23.852846+0000","last_active":"2023-12-08T08:38:26.327085+0000","last_peered":"2023-12-08T08:38:26.327085+0000","last_clean":"2023-12-08T08:38:26.327085+0000","last_became_active":"2023-12-08T08:38:23.852393+0000","last_became_peered":"2023-12-08T08:38:23.852393+0000","last_unstale":"2023-12-08T08:38:26.327085+0000","last_undegraded":"2023-12-08T08:38:26.327085+0000","last_fullsized":"2023-12-08T08:38:26.327085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:32:58.184058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655788+0000","last_change":"2023-12-08T08:38:19.634986+0000","last_active":"2023-12-08T08:38:23.655788+0000","last_peered":"2023-12-08T08:38:23.655788+0000","last_clean":"2023-12-08T08:38:23.655788+0000","last_became_active":"2023-12-08T08:38:19.634682+0000","last_became_peered":"2023-12-08T08:38:19.634682+0000","last_unstale":"2023-12-08T08:38:23.655788+0000","last_undegraded":"2023-12-08T08:38:23.655788+0000","last_fullsized":"2023-12-08T08:38:23.655788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661184+0000","last_change":"2023-12-08T08:38:18.631253+0000","last_active":"2023-12-08T08:38:23.661184+0000","last_peered":"2023-12-08T08:38:23.661184+0000","last_clean":"2023-12-08T08:38:23.661184+0000","last_became_active":"2023-12-08T08:38:18.630979+0000","last_became_peered":"2023-12-08T08:38:18.630979+0000","last_unstale":"2023-12-08T08:38:23.661184+0000","last_undegraded":"2023-12-08T08:38:23.661184+0000","last_fullsized":"2023-12-08T08:38:23.661184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:00:00.141773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851733+0000","last_change":"2023-12-08T08:38:08.276815+0000","last_active":"2023-12-08T08:38:23.851733+0000","last_peered":"2023-12-08T08:38:23.851733+0000","last_clean":"2023-12-08T08:38:23.851733+0000","last_became_active":"2023-12-08T08:38:08.275828+0000","last_became_peered":"2023-12-08T08:38:08.275828+0000","last_unstale":"2023-12-08T08:38:23.851733+0000","last_undegraded":"2023-12-08T08:38:23.851733+0000","last_fullsized":"2023-12-08T08:38:23.851733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_clean_scrub_stamp":"2023-12-08T08:38:07.104197+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:35:34.851149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.328098+0000","last_change":"2023-12-08T08:38:23.855031+0000","last_active":"2023-12-08T08:38:26.328098+0000","last_peered":"2023-12-08T08:38:26.328098+0000","last_clean":"2023-12-08T08:38:26.328098+0000","last_became_active":"2023-12-08T08:38:23.854583+0000","last_became_peered":"2023-12-08T08:38:23.854583+0000","last_unstale":"2023-12-08T08:38:26.328098+0000","last_undegraded":"2023-12-08T08:38:26.328098+0000","last_fullsized":"2023-12-08T08:38:26.328098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:01.683162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852252+0000","last_change":"2023-12-08T08:38:19.633600+0000","last_active":"2023-12-08T08:38:23.852252+0000","last_peered":"2023-12-08T08:38:23.852252+0000","last_clean":"2023-12-08T08:38:23.852252+0000","last_became_active":"2023-12-08T08:38:19.633358+0000","last_became_peered":"2023-12-08T08:38:19.633358+0000","last_unstale":"2023-12-08T08:38:23.852252+0000","last_undegraded":"2023-12-08T08:38:23.852252+0000","last_fullsized":"2023-12-08T08:38:23.852252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:13.356849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662546+0000","last_change":"2023-12-08T08:38:18.631378+0000","last_active":"2023-12-08T08:38:23.662546+0000","last_peered":"2023-12-08T08:38:23.662546+0000","last_clean":"2023-12-08T08:38:23.662546+0000","last_became_active":"2023-12-08T08:38:18.631072+0000","last_became_peered":"2023-12-08T08:38:18.631072+0000","last_unstale":"2023-12-08T08:38:23.662546+0000","last_undegraded":"2023-12-08T08:38:23.662546+0000","last_fullsized":"2023-12-08T08:38:23.662546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:27:03.426918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.327456+0000","last_change":"2023-12-08T08:38:23.853285+0000","last_active":"2023-12-08T08:38:26.327456+0000","last_peered":"2023-12-08T08:38:26.327456+0000","last_clean":"2023-12-08T08:38:26.327456+0000","last_became_active":"2023-12-08T08:38:23.852823+0000","last_became_peered":"2023-12-08T08:38:23.852823+0000","last_unstale":"2023-12-08T08:38:26.327456+0000","last_undegraded":"2023-12-08T08:38:26.327456+0000","last_fullsized":"2023-12-08T08:38:26.327456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:59.405196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661432+0000","last_change":"2023-12-08T08:38:19.632038+0000","last_active":"2023-12-08T08:38:23.661432+0000","last_peered":"2023-12-08T08:38:23.661432+0000","last_clean":"2023-12-08T08:38:23.661432+0000","last_became_active":"2023-12-08T08:38:19.630797+0000","last_became_peered":"2023-12-08T08:38:19.630797+0000","last_unstale":"2023-12-08T08:38:23.661432+0000","last_undegraded":"2023-12-08T08:38:23.661432+0000","last_fullsized":"2023-12-08T08:38:23.661432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:46:57.645302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662789+0000","last_change":"2023-12-08T08:38:18.633629+0000","last_active":"2023-12-08T08:38:23.662789+0000","last_peered":"2023-12-08T08:38:23.662789+0000","last_clean":"2023-12-08T08:38:23.662789+0000","last_became_active":"2023-12-08T08:38:18.633224+0000","last_became_peered":"2023-12-08T08:38:18.633224+0000","last_unstale":"2023-12-08T08:38:23.662789+0000","last_undegraded":"2023-12-08T08:38:23.662789+0000","last_fullsized":"2023-12-08T08:38:23.662789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:13:39.901023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.327158+0000","last_change":"2023-12-08T08:38:23.680301+0000","last_active":"2023-12-08T08:38:26.327158+0000","last_peered":"2023-12-08T08:38:26.327158+0000","last_clean":"2023-12-08T08:38:26.327158+0000","last_became_active":"2023-12-08T08:38:23.679652+0000","last_became_peered":"2023-12-08T08:38:23.679652+0000","last_unstale":"2023-12-08T08:38:26.327158+0000","last_undegraded":"2023-12-08T08:38:26.327158+0000","last_fullsized":"2023-12-08T08:38:26.327158+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:18:13.891999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.853813+0000","last_change":"2023-12-08T08:38:19.622603+0000","last_active":"2023-12-08T08:38:23.853813+0000","last_peered":"2023-12-08T08:38:23.853813+0000","last_clean":"2023-12-08T08:38:23.853813+0000","last_became_active":"2023-12-08T08:38:19.622325+0000","last_became_peered":"2023-12-08T08:38:19.622325+0000","last_unstale":"2023-12-08T08:38:23.853813+0000","last_undegraded":"2023-12-08T08:38:23.853813+0000","last_fullsized":"2023-12-08T08:38:23.853813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:01:22.871596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659628+0000","last_change":"2023-12-08T08:38:18.635791+0000","last_active":"2023-12-08T08:38:23.659628+0000","last_peered":"2023-12-08T08:38:23.659628+0000","last_clean":"2023-12-08T08:38:23.659628+0000","last_became_active":"2023-12-08T08:38:18.632773+0000","last_became_peered":"2023-12-08T08:38:18.632773+0000","last_unstale":"2023-12-08T08:38:23.659628+0000","last_undegraded":"2023-12-08T08:38:23.659628+0000","last_fullsized":"2023-12-08T08:38:23.659628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:46:19.770776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.626169+0000","last_change":"2023-12-08T08:38:23.853534+0000","last_active":"2023-12-08T08:38:24.626169+0000","last_peered":"2023-12-08T08:38:24.626169+0000","last_clean":"2023-12-08T08:38:24.626169+0000","last_became_active":"2023-12-08T08:38:23.852476+0000","last_became_peered":"2023-12-08T08:38:23.852476+0000","last_unstale":"2023-12-08T08:38:24.626169+0000","last_undegraded":"2023-12-08T08:38:24.626169+0000","last_fullsized":"2023-12-08T08:38:24.626169+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:01:26.909456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852501+0000","last_change":"2023-12-08T08:38:19.621763+0000","last_active":"2023-12-08T08:38:23.852501+0000","last_peered":"2023-12-08T08:38:23.852501+0000","last_clean":"2023-12-08T08:38:23.852501+0000","last_became_active":"2023-12-08T08:38:19.621504+0000","last_became_peered":"2023-12-08T08:38:19.621504+0000","last_unstale":"2023-12-08T08:38:23.852501+0000","last_undegraded":"2023-12-08T08:38:23.852501+0000","last_fullsized":"2023-12-08T08:38:23.852501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:34:25.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658538+0000","last_change":"2023-12-08T08:38:18.632647+0000","last_active":"2023-12-08T08:38:23.658538+0000","last_peered":"2023-12-08T08:38:23.658538+0000","last_clean":"2023-12-08T08:38:23.658538+0000","last_became_active":"2023-12-08T08:38:18.632398+0000","last_became_peered":"2023-12-08T08:38:18.632398+0000","last_unstale":"2023-12-08T08:38:23.658538+0000","last_undegraded":"2023-12-08T08:38:23.658538+0000","last_fullsized":"2023-12-08T08:38:23.658538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:42:30.138259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608618+0000","last_change":"2023-12-08T08:38:23.674111+0000","last_active":"2023-12-08T08:38:24.608618+0000","last_peered":"2023-12-08T08:38:24.608618+0000","last_clean":"2023-12-08T08:38:24.608618+0000","last_became_active":"2023-12-08T08:38:23.673287+0000","last_became_peered":"2023-12-08T08:38:23.673287+0000","last_unstale":"2023-12-08T08:38:24.608618+0000","last_undegraded":"2023-12-08T08:38:24.608618+0000","last_fullsized":"2023-12-08T08:38:24.608618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:22:01.560111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851975+0000","last_change":"2023-12-08T08:38:19.620974+0000","last_active":"2023-12-08T08:38:23.851975+0000","last_peered":"2023-12-08T08:38:23.851975+0000","last_clean":"2023-12-08T08:38:23.851975+0000","last_became_active":"2023-12-08T08:38:19.619478+0000","last_became_peered":"2023-12-08T08:38:19.619478+0000","last_unstale":"2023-12-08T08:38:23.851975+0000","last_undegraded":"2023-12-08T08:38:23.851975+0000","last_fullsized":"2023-12-08T08:38:23.851975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:55.367335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659793+0000","last_change":"2023-12-08T08:38:18.629485+0000","last_active":"2023-12-08T08:38:23.659793+0000","last_peered":"2023-12-08T08:38:23.659793+0000","last_clean":"2023-12-08T08:38:23.659793+0000","last_became_active":"2023-12-08T08:38:18.629182+0000","last_became_peered":"2023-12-08T08:38:18.629182+0000","last_unstale":"2023-12-08T08:38:23.659793+0000","last_undegraded":"2023-12-08T08:38:23.659793+0000","last_fullsized":"2023-12-08T08:38:23.659793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:47:43.142370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.326907+0000","last_change":"2023-12-08T08:38:23.673762+0000","last_active":"2023-12-08T08:38:26.326907+0000","last_peered":"2023-12-08T08:38:26.326907+0000","last_clean":"2023-12-08T08:38:26.326907+0000","last_became_active":"2023-12-08T08:38:23.673236+0000","last_became_peered":"2023-12-08T08:38:23.673236+0000","last_unstale":"2023-12-08T08:38:26.326907+0000","last_undegraded":"2023-12-08T08:38:26.326907+0000","last_fullsized":"2023-12-08T08:38:26.326907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:34:29.925547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664201+0000","last_change":"2023-12-08T08:38:19.624828+0000","last_active":"2023-12-08T08:38:23.664201+0000","last_peered":"2023-12-08T08:38:23.664201+0000","last_clean":"2023-12-08T08:38:23.664201+0000","last_became_active":"2023-12-08T08:38:19.624249+0000","last_became_peered":"2023-12-08T08:38:19.624249+0000","last_unstale":"2023-12-08T08:38:23.664201+0000","last_undegraded":"2023-12-08T08:38:23.664201+0000","last_fullsized":"2023-12-08T08:38:23.664201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.663711+0000","last_change":"2023-12-08T08:38:18.624832+0000","last_active":"2023-12-08T08:38:23.663711+0000","last_peered":"2023-12-08T08:38:23.663711+0000","last_clean":"2023-12-08T08:38:23.663711+0000","last_became_active":"2023-12-08T08:38:18.624580+0000","last_became_peered":"2023-12-08T08:38:18.624580+0000","last_unstale":"2023-12-08T08:38:23.663711+0000","last_undegraded":"2023-12-08T08:38:23.663711+0000","last_fullsized":"2023-12-08T08:38:23.663711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:13:39.901023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576884+0000","last_change":"2023-12-08T08:38:23.856627+0000","last_active":"2023-12-08T08:38:25.576884+0000","last_peered":"2023-12-08T08:38:25.576884+0000","last_clean":"2023-12-08T08:38:25.576884+0000","last_became_active":"2023-12-08T08:38:23.855845+0000","last_became_peered":"2023-12-08T08:38:23.855845+0000","last_unstale":"2023-12-08T08:38:25.576884+0000","last_undegraded":"2023-12-08T08:38:25.576884+0000","last_fullsized":"2023-12-08T08:38:25.576884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:20:54.195474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852053+0000","last_change":"2023-12-08T08:38:19.621686+0000","last_active":"2023-12-08T08:38:23.852053+0000","last_peered":"2023-12-08T08:38:23.852053+0000","last_clean":"2023-12-08T08:38:23.852053+0000","last_became_active":"2023-12-08T08:38:19.621366+0000","last_became_peered":"2023-12-08T08:38:19.621366+0000","last_unstale":"2023-12-08T08:38:23.852053+0000","last_undegraded":"2023-12-08T08:38:23.852053+0000","last_fullsized":"2023-12-08T08:38:23.852053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:18.334775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661960+0000","last_change":"2023-12-08T08:38:18.637940+0000","last_active":"2023-12-08T08:38:23.661960+0000","last_peered":"2023-12-08T08:38:23.661960+0000","last_clean":"2023-12-08T08:38:23.661960+0000","last_became_active":"2023-12-08T08:38:18.637552+0000","last_became_peered":"2023-12-08T08:38:18.637552+0000","last_unstale":"2023-12-08T08:38:23.661960+0000","last_undegraded":"2023-12-08T08:38:23.661960+0000","last_fullsized":"2023-12-08T08:38:23.661960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:14.342540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576721+0000","last_change":"2023-12-08T08:38:23.852937+0000","last_active":"2023-12-08T08:38:25.576721+0000","last_peered":"2023-12-08T08:38:25.576721+0000","last_clean":"2023-12-08T08:38:25.576721+0000","last_became_active":"2023-12-08T08:38:23.852228+0000","last_became_peered":"2023-12-08T08:38:23.852228+0000","last_unstale":"2023-12-08T08:38:25.576721+0000","last_undegraded":"2023-12-08T08:38:25.576721+0000","last_fullsized":"2023-12-08T08:38:25.576721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:46:14.364194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664273+0000","last_change":"2023-12-08T08:38:19.618681+0000","last_active":"2023-12-08T08:38:23.664273+0000","last_peered":"2023-12-08T08:38:23.664273+0000","last_clean":"2023-12-08T08:38:23.664273+0000","last_became_active":"2023-12-08T08:38:19.618102+0000","last_became_peered":"2023-12-08T08:38:19.618102+0000","last_unstale":"2023-12-08T08:38:23.664273+0000","last_undegraded":"2023-12-08T08:38:23.664273+0000","last_fullsized":"2023-12-08T08:38:23.664273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:00:01.146409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661116+0000","last_change":"2023-12-08T08:38:18.636521+0000","last_active":"2023-12-08T08:38:23.661116+0000","last_peered":"2023-12-08T08:38:23.661116+0000","last_clean":"2023-12-08T08:38:23.661116+0000","last_became_active":"2023-12-08T08:38:18.636242+0000","last_became_peered":"2023-12-08T08:38:18.636242+0000","last_unstale":"2023-12-08T08:38:23.661116+0000","last_undegraded":"2023-12-08T08:38:23.661116+0000","last_fullsized":"2023-12-08T08:38:23.661116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.626765+0000","last_change":"2023-12-08T08:38:23.855694+0000","last_active":"2023-12-08T08:38:24.626765+0000","last_peered":"2023-12-08T08:38:24.626765+0000","last_clean":"2023-12-08T08:38:24.626765+0000","last_became_active":"2023-12-08T08:38:23.855342+0000","last_became_peered":"2023-12-08T08:38:23.855342+0000","last_unstale":"2023-12-08T08:38:24.626765+0000","last_undegraded":"2023-12-08T08:38:24.626765+0000","last_fullsized":"2023-12-08T08:38:24.626765+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T20:21:44.003574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852120+0000","last_change":"2023-12-08T08:38:19.627826+0000","last_active":"2023-12-08T08:38:23.852120+0000","last_peered":"2023-12-08T08:38:23.852120+0000","last_clean":"2023-12-08T08:38:23.852120+0000","last_became_active":"2023-12-08T08:38:19.627154+0000","last_became_peered":"2023-12-08T08:38:19.627154+0000","last_unstale":"2023-12-08T08:38:23.852120+0000","last_undegraded":"2023-12-08T08:38:23.852120+0000","last_fullsized":"2023-12-08T08:38:23.852120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:11.797914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:29.035583+0000","last_change":"2023-12-08T08:38:18.630599+0000","last_active":"2023-12-08T08:38:29.035583+0000","last_peered":"2023-12-08T08:38:29.035583+0000","last_clean":"2023-12-08T08:38:29.035583+0000","last_became_active":"2023-12-08T08:38:18.630349+0000","last_became_peered":"2023-12-08T08:38:18.630349+0000","last_unstale":"2023-12-08T08:38:29.035583+0000","last_undegraded":"2023-12-08T08:38:29.035583+0000","last_fullsized":"2023-12-08T08:38:29.035583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:36:34.339195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3839,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.827898+0000","last_change":"2023-12-08T08:38:23.676381+0000","last_active":"2023-12-08T08:38:25.827898+0000","last_peered":"2023-12-08T08:38:25.827898+0000","last_clean":"2023-12-08T08:38:25.827898+0000","last_became_active":"2023-12-08T08:38:23.675561+0000","last_became_peered":"2023-12-08T08:38:23.675561+0000","last_unstale":"2023-12-08T08:38:25.827898+0000","last_undegraded":"2023-12-08T08:38:25.827898+0000","last_fullsized":"2023-12-08T08:38:25.827898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:22.372635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659110+0000","last_change":"2023-12-08T08:38:19.615536+0000","last_active":"2023-12-08T08:38:23.659110+0000","last_peered":"2023-12-08T08:38:23.659110+0000","last_clean":"2023-12-08T08:38:23.659110+0000","last_became_active":"2023-12-08T08:38:19.615243+0000","last_became_peered":"2023-12-08T08:38:19.615243+0000","last_unstale":"2023-12-08T08:38:23.659110+0000","last_undegraded":"2023-12-08T08:38:23.659110+0000","last_fullsized":"2023-12-08T08:38:23.659110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:13:40.905659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662690+0000","last_change":"2023-12-08T08:38:18.632950+0000","last_active":"2023-12-08T08:38:23.662690+0000","last_peered":"2023-12-08T08:38:23.662690+0000","last_clean":"2023-12-08T08:38:23.662690+0000","last_became_active":"2023-12-08T08:38:18.632652+0000","last_became_peered":"2023-12-08T08:38:18.632652+0000","last_unstale":"2023-12-08T08:38:23.662690+0000","last_undegraded":"2023-12-08T08:38:23.662690+0000","last_fullsized":"2023-12-08T08:38:23.662690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:36:34.339195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609288+0000","last_change":"2023-12-08T08:38:23.674379+0000","last_active":"2023-12-08T08:38:24.609288+0000","last_peered":"2023-12-08T08:38:24.609288+0000","last_clean":"2023-12-08T08:38:24.609288+0000","last_became_active":"2023-12-08T08:38:23.673581+0000","last_became_peered":"2023-12-08T08:38:23.673581+0000","last_unstale":"2023-12-08T08:38:24.609288+0000","last_undegraded":"2023-12-08T08:38:24.609288+0000","last_fullsized":"2023-12-08T08:38:24.609288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:01.683162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659853+0000","last_change":"2023-12-08T08:38:19.633344+0000","last_active":"2023-12-08T08:38:23.659853+0000","last_peered":"2023-12-08T08:38:23.659853+0000","last_clean":"2023-12-08T08:38:23.659853+0000","last_became_active":"2023-12-08T08:38:19.633035+0000","last_became_peered":"2023-12-08T08:38:19.633035+0000","last_unstale":"2023-12-08T08:38:23.659853+0000","last_undegraded":"2023-12-08T08:38:23.659853+0000","last_fullsized":"2023-12-08T08:38:23.659853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661808+0000","last_change":"2023-12-08T08:38:18.631126+0000","last_active":"2023-12-08T08:38:23.661808+0000","last_peered":"2023-12-08T08:38:23.661808+0000","last_clean":"2023-12-08T08:38:23.661808+0000","last_became_active":"2023-12-08T08:38:18.630841+0000","last_became_peered":"2023-12-08T08:38:18.630841+0000","last_unstale":"2023-12-08T08:38:23.661808+0000","last_undegraded":"2023-12-08T08:38:23.661808+0000","last_fullsized":"2023-12-08T08:38:23.661808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:07.497079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576255+0000","last_change":"2023-12-08T08:38:23.678178+0000","last_active":"2023-12-08T08:38:25.576255+0000","last_peered":"2023-12-08T08:38:25.576255+0000","last_clean":"2023-12-08T08:38:25.576255+0000","last_became_active":"2023-12-08T08:38:23.677800+0000","last_became_peered":"2023-12-08T08:38:23.677800+0000","last_unstale":"2023-12-08T08:38:25.576255+0000","last_undegraded":"2023-12-08T08:38:25.576255+0000","last_fullsized":"2023-12-08T08:38:25.576255+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:22:37.463722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661701+0000","last_change":"2023-12-08T08:38:19.631623+0000","last_active":"2023-12-08T08:38:23.661701+0000","last_peered":"2023-12-08T08:38:23.661701+0000","last_clean":"2023-12-08T08:38:23.661701+0000","last_became_active":"2023-12-08T08:38:19.630737+0000","last_became_peered":"2023-12-08T08:38:19.630737+0000","last_unstale":"2023-12-08T08:38:23.661701+0000","last_undegraded":"2023-12-08T08:38:23.661701+0000","last_fullsized":"2023-12-08T08:38:23.661701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:18.334775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665261+0000","last_change":"2023-12-08T08:38:18.630876+0000","last_active":"2023-12-08T08:38:23.665261+0000","last_peered":"2023-12-08T08:38:23.665261+0000","last_clean":"2023-12-08T08:38:23.665261+0000","last_became_active":"2023-12-08T08:38:18.630581+0000","last_became_peered":"2023-12-08T08:38:18.630581+0000","last_unstale":"2023-12-08T08:38:23.665261+0000","last_undegraded":"2023-12-08T08:38:23.665261+0000","last_fullsized":"2023-12-08T08:38:23.665261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576234+0000","last_change":"2023-12-08T08:38:23.855920+0000","last_active":"2023-12-08T08:38:25.576234+0000","last_peered":"2023-12-08T08:38:25.576234+0000","last_clean":"2023-12-08T08:38:25.576234+0000","last_became_active":"2023-12-08T08:38:23.855491+0000","last_became_peered":"2023-12-08T08:38:23.855491+0000","last_unstale":"2023-12-08T08:38:25.576234+0000","last_undegraded":"2023-12-08T08:38:25.576234+0000","last_fullsized":"2023-12-08T08:38:25.576234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:10:52.070614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.326821+0000","last_change":"2023-12-08T08:38:23.855660+0000","last_active":"2023-12-08T08:38:26.326821+0000","last_peered":"2023-12-08T08:38:26.326821+0000","last_clean":"2023-12-08T08:38:26.326821+0000","last_became_active":"2023-12-08T08:38:23.855190+0000","last_became_peered":"2023-12-08T08:38:23.855190+0000","last_unstale":"2023-12-08T08:38:26.326821+0000","last_undegraded":"2023-12-08T08:38:26.326821+0000","last_fullsized":"2023-12-08T08:38:26.326821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:48.357070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664507+0000","last_change":"2023-12-08T08:38:19.620116+0000","last_active":"2023-12-08T08:38:23.664507+0000","last_peered":"2023-12-08T08:38:23.664507+0000","last_clean":"2023-12-08T08:38:23.664507+0000","last_became_active":"2023-12-08T08:38:19.619412+0000","last_became_peered":"2023-12-08T08:38:19.619412+0000","last_unstale":"2023-12-08T08:38:23.664507+0000","last_undegraded":"2023-12-08T08:38:23.664507+0000","last_fullsized":"2023-12-08T08:38:23.664507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661919+0000","last_change":"2023-12-08T08:38:18.630228+0000","last_active":"2023-12-08T08:38:23.661919+0000","last_peered":"2023-12-08T08:38:23.661919+0000","last_clean":"2023-12-08T08:38:23.661919+0000","last_became_active":"2023-12-08T08:38:18.629445+0000","last_became_peered":"2023-12-08T08:38:18.629445+0000","last_unstale":"2023-12-08T08:38:23.661919+0000","last_undegraded":"2023-12-08T08:38:23.661919+0000","last_fullsized":"2023-12-08T08:38:23.661919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:57.526979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.328319+0000","last_change":"2023-12-08T08:38:23.678105+0000","last_active":"2023-12-08T08:38:26.328319+0000","last_peered":"2023-12-08T08:38:26.328319+0000","last_clean":"2023-12-08T08:38:26.328319+0000","last_became_active":"2023-12-08T08:38:23.677716+0000","last_became_peered":"2023-12-08T08:38:23.677716+0000","last_unstale":"2023-12-08T08:38:26.328319+0000","last_undegraded":"2023-12-08T08:38:26.328319+0000","last_fullsized":"2023-12-08T08:38:26.328319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:15.835775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655896+0000","last_change":"2023-12-08T08:38:19.628959+0000","last_active":"2023-12-08T08:38:23.655896+0000","last_peered":"2023-12-08T08:38:23.655896+0000","last_clean":"2023-12-08T08:38:23.655896+0000","last_became_active":"2023-12-08T08:38:19.628662+0000","last_became_peered":"2023-12-08T08:38:19.628662+0000","last_unstale":"2023-12-08T08:38:23.655896+0000","last_undegraded":"2023-12-08T08:38:23.655896+0000","last_fullsized":"2023-12-08T08:38:23.655896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658005+0000","last_change":"2023-12-08T08:38:18.637772+0000","last_active":"2023-12-08T08:38:23.658005+0000","last_peered":"2023-12-08T08:38:23.658005+0000","last_clean":"2023-12-08T08:38:23.658005+0000","last_became_active":"2023-12-08T08:38:18.637318+0000","last_became_peered":"2023-12-08T08:38:18.637318+0000","last_unstale":"2023-12-08T08:38:23.658005+0000","last_undegraded":"2023-12-08T08:38:23.658005+0000","last_fullsized":"2023-12-08T08:38:23.658005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:11.067589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664203+0000","last_change":"2023-12-08T08:38:18.630848+0000","last_active":"2023-12-08T08:38:23.664203+0000","last_peered":"2023-12-08T08:38:23.664203+0000","last_clean":"2023-12-08T08:38:23.664203+0000","last_became_active":"2023-12-08T08:38:18.629736+0000","last_became_peered":"2023-12-08T08:38:18.629736+0000","last_unstale":"2023-12-08T08:38:23.664203+0000","last_undegraded":"2023-12-08T08:38:23.664203+0000","last_fullsized":"2023-12-08T08:38:23.664203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:10.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661026+0000","last_change":"2023-12-08T08:38:19.612508+0000","last_active":"2023-12-08T08:38:23.661026+0000","last_peered":"2023-12-08T08:38:23.661026+0000","last_clean":"2023-12-08T08:38:23.661026+0000","last_became_active":"2023-12-08T08:38:19.612212+0000","last_became_peered":"2023-12-08T08:38:19.612212+0000","last_unstale":"2023-12-08T08:38:23.661026+0000","last_undegraded":"2023-12-08T08:38:23.661026+0000","last_fullsized":"2023-12-08T08:38:23.661026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608202+0000","last_change":"2023-12-08T08:38:23.678309+0000","last_active":"2023-12-08T08:38:24.608202+0000","last_peered":"2023-12-08T08:38:24.608202+0000","last_clean":"2023-12-08T08:38:24.608202+0000","last_became_active":"2023-12-08T08:38:23.677882+0000","last_became_peered":"2023-12-08T08:38:23.677882+0000","last_unstale":"2023-12-08T08:38:24.608202+0000","last_undegraded":"2023-12-08T08:38:24.608202+0000","last_fullsized":"2023-12-08T08:38:24.608202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:15.118488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655793+0000","last_change":"2023-12-08T08:38:18.638936+0000","last_active":"2023-12-08T08:38:23.655793+0000","last_peered":"2023-12-08T08:38:23.655793+0000","last_clean":"2023-12-08T08:38:23.655793+0000","last_became_active":"2023-12-08T08:38:18.638058+0000","last_became_peered":"2023-12-08T08:38:18.638058+0000","last_unstale":"2023-12-08T08:38:23.655793+0000","last_undegraded":"2023-12-08T08:38:23.655793+0000","last_fullsized":"2023-12-08T08:38:23.655793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:10.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658137+0000","last_change":"2023-12-08T08:38:19.619468+0000","last_active":"2023-12-08T08:38:23.658137+0000","last_peered":"2023-12-08T08:38:23.658137+0000","last_clean":"2023-12-08T08:38:23.658137+0000","last_became_active":"2023-12-08T08:38:19.619177+0000","last_became_peered":"2023-12-08T08:38:19.619177+0000","last_unstale":"2023-12-08T08:38:23.658137+0000","last_undegraded":"2023-12-08T08:38:23.658137+0000","last_fullsized":"2023-12-08T08:38:23.658137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:46:20.775412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576589+0000","last_change":"2023-12-08T08:38:23.679648+0000","last_active":"2023-12-08T08:38:25.576589+0000","last_peered":"2023-12-08T08:38:25.576589+0000","last_clean":"2023-12-08T08:38:25.576589+0000","last_became_active":"2023-12-08T08:38:23.679229+0000","last_became_peered":"2023-12-08T08:38:23.679229+0000","last_unstale":"2023-12-08T08:38:25.576589+0000","last_undegraded":"2023-12-08T08:38:25.576589+0000","last_fullsized":"2023-12-08T08:38:25.576589+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:53:05.969247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.663580+0000","last_change":"2023-12-08T08:38:19.624993+0000","last_active":"2023-12-08T08:38:23.663580+0000","last_peered":"2023-12-08T08:38:23.663580+0000","last_clean":"2023-12-08T08:38:23.663580+0000","last_became_active":"2023-12-08T08:38:19.624569+0000","last_became_peered":"2023-12-08T08:38:19.624569+0000","last_unstale":"2023-12-08T08:38:23.663580+0000","last_undegraded":"2023-12-08T08:38:23.663580+0000","last_fullsized":"2023-12-08T08:38:23.663580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662648+0000","last_change":"2023-12-08T08:38:19.631559+0000","last_active":"2023-12-08T08:38:23.662648+0000","last_peered":"2023-12-08T08:38:23.662648+0000","last_clean":"2023-12-08T08:38:23.662648+0000","last_became_active":"2023-12-08T08:38:19.631225+0000","last_became_peered":"2023-12-08T08:38:19.631225+0000","last_unstale":"2023-12-08T08:38:23.662648+0000","last_undegraded":"2023-12-08T08:38:23.662648+0000","last_fullsized":"2023-12-08T08:38:23.662648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:12.072225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661866+0000","last_change":"2023-12-08T08:38:19.631476+0000","last_active":"2023-12-08T08:38:23.661866+0000","last_peered":"2023-12-08T08:38:23.661866+0000","last_clean":"2023-12-08T08:38:23.661866+0000","last_became_active":"2023-12-08T08:38:19.630628+0000","last_became_peered":"2023-12-08T08:38:19.630628+0000","last_unstale":"2023-12-08T08:38:23.661866+0000","last_undegraded":"2023-12-08T08:38:23.661866+0000","last_fullsized":"2023-12-08T08:38:23.661866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:55.367335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.660943+0000","last_change":"2023-12-08T08:38:19.623252+0000","last_active":"2023-12-08T08:38:23.660943+0000","last_peered":"2023-12-08T08:38:23.660943+0000","last_clean":"2023-12-08T08:38:23.660943+0000","last_became_active":"2023-12-08T08:38:19.622970+0000","last_became_peered":"2023-12-08T08:38:19.622970+0000","last_unstale":"2023-12-08T08:38:23.660943+0000","last_undegraded":"2023-12-08T08:38:23.660943+0000","last_fullsized":"2023-12-08T08:38:23.660943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.657026+0000","last_change":"2023-12-08T08:38:19.621907+0000","last_active":"2023-12-08T08:38:23.657026+0000","last_peered":"2023-12-08T08:38:23.657026+0000","last_clean":"2023-12-08T08:38:23.657026+0000","last_became_active":"2023-12-08T08:38:19.621597+0000","last_became_peered":"2023-12-08T08:38:19.621597+0000","last_unstale":"2023-12-08T08:38:23.657026+0000","last_undegraded":"2023-12-08T08:38:23.657026+0000","last_fullsized":"2023-12-08T08:38:23.657026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:55.367335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662589+0000","last_change":"2023-12-08T08:38:19.621460+0000","last_active":"2023-12-08T08:38:23.662589+0000","last_peered":"2023-12-08T08:38:23.662589+0000","last_clean":"2023-12-08T08:38:23.662589+0000","last_became_active":"2023-12-08T08:38:19.621090+0000","last_became_peered":"2023-12-08T08:38:19.621090+0000","last_unstale":"2023-12-08T08:38:23.662589+0000","last_undegraded":"2023-12-08T08:38:23.662589+0000","last_fullsized":"2023-12-08T08:38:23.662589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665456+0000","last_change":"2023-12-08T08:38:19.620846+0000","last_active":"2023-12-08T08:38:23.665456+0000","last_peered":"2023-12-08T08:38:23.665456+0000","last_clean":"2023-12-08T08:38:23.665456+0000","last_became_active":"2023-12-08T08:38:19.620575+0000","last_became_peered":"2023-12-08T08:38:19.620575+0000","last_unstale":"2023-12-08T08:38:23.665456+0000","last_undegraded":"2023-12-08T08:38:23.665456+0000","last_fullsized":"2023-12-08T08:38:23.665456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656076+0000","last_change":"2023-12-08T08:38:19.623987+0000","last_active":"2023-12-08T08:38:23.656076+0000","last_peered":"2023-12-08T08:38:23.656076+0000","last_clean":"2023-12-08T08:38:23.656076+0000","last_became_active":"2023-12-08T08:38:19.623637+0000","last_became_peered":"2023-12-08T08:38:19.623637+0000","last_unstale":"2023-12-08T08:38:23.656076+0000","last_undegraded":"2023-12-08T08:38:23.656076+0000","last_fullsized":"2023-12-08T08:38:23.656076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:43.336635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661985+0000","last_change":"2023-12-08T08:38:19.627606+0000","last_active":"2023-12-08T08:38:23.661985+0000","last_peered":"2023-12-08T08:38:23.661985+0000","last_clean":"2023-12-08T08:38:23.661985+0000","last_became_active":"2023-12-08T08:38:19.627345+0000","last_became_peered":"2023-12-08T08:38:19.627345+0000","last_unstale":"2023-12-08T08:38:23.661985+0000","last_undegraded":"2023-12-08T08:38:23.661985+0000","last_fullsized":"2023-12-08T08:38:23.661985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:43.336635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.657865+0000","last_change":"2023-12-08T08:38:19.623195+0000","last_active":"2023-12-08T08:38:23.657865+0000","last_peered":"2023-12-08T08:38:23.657865+0000","last_clean":"2023-12-08T08:38:23.657865+0000","last_became_active":"2023-12-08T08:38:19.622896+0000","last_became_peered":"2023-12-08T08:38:19.622896+0000","last_unstale":"2023-12-08T08:38:23.657865+0000","last_undegraded":"2023-12-08T08:38:23.657865+0000","last_fullsized":"2023-12-08T08:38:23.657865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:59:11.073543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661771+0000","last_change":"2023-12-08T08:38:19.632017+0000","last_active":"2023-12-08T08:38:23.661771+0000","last_peered":"2023-12-08T08:38:23.661771+0000","last_clean":"2023-12-08T08:38:23.661771+0000","last_became_active":"2023-12-08T08:38:19.631771+0000","last_became_peered":"2023-12-08T08:38:19.631771+0000","last_unstale":"2023-12-08T08:38:23.661771+0000","last_undegraded":"2023-12-08T08:38:23.661771+0000","last_fullsized":"2023-12-08T08:38:23.661771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:59:11.073543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.660849+0000","last_change":"2023-12-08T08:38:19.618872+0000","last_active":"2023-12-08T08:38:23.660849+0000","last_peered":"2023-12-08T08:38:23.660849+0000","last_clean":"2023-12-08T08:38:23.660849+0000","last_became_active":"2023-12-08T08:38:19.617964+0000","last_became_peered":"2023-12-08T08:38:19.617964+0000","last_unstale":"2023-12-08T08:38:23.660849+0000","last_undegraded":"2023-12-08T08:38:23.660849+0000","last_fullsized":"2023-12-08T08:38:23.660849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665351+0000","last_change":"2023-12-08T08:38:19.622440+0000","last_active":"2023-12-08T08:38:23.665351+0000","last_peered":"2023-12-08T08:38:23.665351+0000","last_clean":"2023-12-08T08:38:23.665351+0000","last_became_active":"2023-12-08T08:38:19.621997+0000","last_became_peered":"2023-12-08T08:38:19.621997+0000","last_unstale":"2023-12-08T08:38:23.665351+0000","last_undegraded":"2023-12-08T08:38:23.665351+0000","last_fullsized":"2023-12-08T08:38:23.665351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852435+0000","last_change":"2023-12-08T08:38:19.621395+0000","last_active":"2023-12-08T08:38:23.852435+0000","last_peered":"2023-12-08T08:38:23.852435+0000","last_clean":"2023-12-08T08:38:23.852435+0000","last_became_active":"2023-12-08T08:38:19.621148+0000","last_became_peered":"2023-12-08T08:38:19.621148+0000","last_unstale":"2023-12-08T08:38:23.852435+0000","last_undegraded":"2023-12-08T08:38:23.852435+0000","last_fullsized":"2023-12-08T08:38:23.852435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:44.319210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659499+0000","last_change":"2023-12-08T08:38:19.626108+0000","last_active":"2023-12-08T08:38:23.659499+0000","last_peered":"2023-12-08T08:38:23.659499+0000","last_clean":"2023-12-08T08:38:23.659499+0000","last_became_active":"2023-12-08T08:38:19.625773+0000","last_became_peered":"2023-12-08T08:38:19.625773+0000","last_unstale":"2023-12-08T08:38:23.659499+0000","last_undegraded":"2023-12-08T08:38:23.659499+0000","last_fullsized":"2023-12-08T08:38:23.659499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:05:08.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662077+0000","last_change":"2023-12-08T08:38:19.631410+0000","last_active":"2023-12-08T08:38:23.662077+0000","last_peered":"2023-12-08T08:38:23.662077+0000","last_clean":"2023-12-08T08:38:23.662077+0000","last_became_active":"2023-12-08T08:38:19.630653+0000","last_became_peered":"2023-12-08T08:38:19.630653+0000","last_unstale":"2023-12-08T08:38:23.662077+0000","last_undegraded":"2023-12-08T08:38:23.662077+0000","last_fullsized":"2023-12-08T08:38:23.662077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:44.319210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661818+0000","last_change":"2023-12-08T08:38:19.630190+0000","last_active":"2023-12-08T08:38:23.661818+0000","last_peered":"2023-12-08T08:38:23.661818+0000","last_clean":"2023-12-08T08:38:23.661818+0000","last_became_active":"2023-12-08T08:38:19.629923+0000","last_became_peered":"2023-12-08T08:38:19.629923+0000","last_unstale":"2023-12-08T08:38:23.661818+0000","last_undegraded":"2023-12-08T08:38:23.661818+0000","last_fullsized":"2023-12-08T08:38:23.661818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664602+0000","last_change":"2023-12-08T08:38:19.620155+0000","last_active":"2023-12-08T08:38:23.664602+0000","last_peered":"2023-12-08T08:38:23.664602+0000","last_clean":"2023-12-08T08:38:23.664602+0000","last_became_active":"2023-12-08T08:38:19.619814+0000","last_became_peered":"2023-12-08T08:38:19.619814+0000","last_unstale":"2023-12-08T08:38:23.664602+0000","last_undegraded":"2023-12-08T08:38:23.664602+0000","last_fullsized":"2023-12-08T08:38:23.664602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664183+0000","last_change":"2023-12-08T08:38:19.616693+0000","last_active":"2023-12-08T08:38:23.664183+0000","last_peered":"2023-12-08T08:38:23.664183+0000","last_clean":"2023-12-08T08:38:23.664183+0000","last_became_active":"2023-12-08T08:38:19.616369+0000","last_became_peered":"2023-12-08T08:38:19.616369+0000","last_unstale":"2023-12-08T08:38:23.664183+0000","last_undegraded":"2023-12-08T08:38:23.664183+0000","last_fullsized":"2023-12-08T08:38:23.664183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:38.322348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655532+0000","last_change":"2023-12-08T08:38:19.625929+0000","last_active":"2023-12-08T08:38:23.655532+0000","last_peered":"2023-12-08T08:38:23.655532+0000","last_clean":"2023-12-08T08:38:23.655532+0000","last_became_active":"2023-12-08T08:38:19.624245+0000","last_became_peered":"2023-12-08T08:38:19.624245+0000","last_unstale":"2023-12-08T08:38:23.655532+0000","last_undegraded":"2023-12-08T08:38:23.655532+0000","last_fullsized":"2023-12-08T08:38:23.655532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659198+0000","last_change":"2023-12-08T08:38:19.625517+0000","last_active":"2023-12-08T08:38:23.659198+0000","last_peered":"2023-12-08T08:38:23.659198+0000","last_clean":"2023-12-08T08:38:23.659198+0000","last_became_active":"2023-12-08T08:38:19.625079+0000","last_became_peered":"2023-12-08T08:38:19.625079+0000","last_unstale":"2023-12-08T08:38:23.659198+0000","last_undegraded":"2023-12-08T08:38:23.659198+0000","last_fullsized":"2023-12-08T08:38:23.659198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:12.072225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665048+0000","last_change":"2023-12-08T08:38:19.632480+0000","last_active":"2023-12-08T08:38:23.665048+0000","last_peered":"2023-12-08T08:38:23.665048+0000","last_clean":"2023-12-08T08:38:23.665048+0000","last_became_active":"2023-12-08T08:38:19.632215+0000","last_became_peered":"2023-12-08T08:38:19.632215+0000","last_unstale":"2023-12-08T08:38:23.665048+0000","last_undegraded":"2023-12-08T08:38:23.665048+0000","last_fullsized":"2023-12-08T08:38:23.665048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665415+0000","last_change":"2023-12-08T08:38:19.635234+0000","last_active":"2023-12-08T08:38:23.665415+0000","last_peered":"2023-12-08T08:38:23.665415+0000","last_clean":"2023-12-08T08:38:23.665415+0000","last_became_active":"2023-12-08T08:38:19.634638+0000","last_became_peered":"2023-12-08T08:38:19.634638+0000","last_unstale":"2023-12-08T08:38:23.665415+0000","last_undegraded":"2023-12-08T08:38:23.665415+0000","last_fullsized":"2023-12-08T08:38:23.665415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.853718+0000","last_change":"2023-12-08T08:38:19.633557+0000","last_active":"2023-12-08T08:38:23.853718+0000","last_peered":"2023-12-08T08:38:23.853718+0000","last_clean":"2023-12-08T08:38:23.853718+0000","last_became_active":"2023-12-08T08:38:19.633168+0000","last_became_peered":"2023-12-08T08:38:19.633168+0000","last_unstale":"2023-12-08T08:38:23.853718+0000","last_undegraded":"2023-12-08T08:38:23.853718+0000","last_fullsized":"2023-12-08T08:38:23.853718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:18:09.854138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655258+0000","last_change":"2023-12-08T08:38:19.635104+0000","last_active":"2023-12-08T08:38:23.655258+0000","last_peered":"2023-12-08T08:38:23.655258+0000","last_clean":"2023-12-08T08:38:23.655258+0000","last_became_active":"2023-12-08T08:38:19.634711+0000","last_became_peered":"2023-12-08T08:38:19.634711+0000","last_unstale":"2023-12-08T08:38:23.655258+0000","last_undegraded":"2023-12-08T08:38:23.655258+0000","last_fullsized":"2023-12-08T08:38:23.655258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851929+0000","last_change":"2023-12-08T08:38:19.618128+0000","last_active":"2023-12-08T08:38:23.851929+0000","last_peered":"2023-12-08T08:38:23.851929+0000","last_clean":"2023-12-08T08:38:23.851929+0000","last_became_active":"2023-12-08T08:38:19.617870+0000","last_became_peered":"2023-12-08T08:38:19.617870+0000","last_unstale":"2023-12-08T08:38:23.851929+0000","last_undegraded":"2023-12-08T08:38:23.851929+0000","last_fullsized":"2023-12-08T08:38:23.851929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:32.116348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661665+0000","last_change":"2023-12-08T08:38:19.631953+0000","last_active":"2023-12-08T08:38:23.661665+0000","last_peered":"2023-12-08T08:38:23.661665+0000","last_clean":"2023-12-08T08:38:23.661665+0000","last_became_active":"2023-12-08T08:38:19.631638+0000","last_became_peered":"2023-12-08T08:38:19.631638+0000","last_unstale":"2023-12-08T08:38:23.661665+0000","last_undegraded":"2023-12-08T08:38:23.661665+0000","last_fullsized":"2023-12-08T08:38:23.661665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851995+0000","last_change":"2023-12-08T08:38:19.621369+0000","last_active":"2023-12-08T08:38:23.851995+0000","last_peered":"2023-12-08T08:38:23.851995+0000","last_clean":"2023-12-08T08:38:23.851995+0000","last_became_active":"2023-12-08T08:38:19.621103+0000","last_became_peered":"2023-12-08T08:38:19.621103+0000","last_unstale":"2023-12-08T08:38:23.851995+0000","last_undegraded":"2023-12-08T08:38:23.851995+0000","last_fullsized":"2023-12-08T08:38:23.851995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:46:57.645302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661240+0000","last_change":"2023-12-08T08:38:19.630541+0000","last_active":"2023-12-08T08:38:23.661240+0000","last_peered":"2023-12-08T08:38:23.661240+0000","last_clean":"2023-12-08T08:38:23.661240+0000","last_became_active":"2023-12-08T08:38:19.630175+0000","last_became_peered":"2023-12-08T08:38:19.630175+0000","last_unstale":"2023-12-08T08:38:23.661240+0000","last_undegraded":"2023-12-08T08:38:23.661240+0000","last_fullsized":"2023-12-08T08:38:23.661240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:49:29.898351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664269+0000","last_change":"2023-12-08T08:38:19.626884+0000","last_active":"2023-12-08T08:38:23.664269+0000","last_peered":"2023-12-08T08:38:23.664269+0000","last_clean":"2023-12-08T08:38:23.664269+0000","last_became_active":"2023-12-08T08:38:19.626525+0000","last_became_peered":"2023-12-08T08:38:19.626525+0000","last_unstale":"2023-12-08T08:38:23.664269+0000","last_undegraded":"2023-12-08T08:38:23.664269+0000","last_fullsized":"2023-12-08T08:38:23.664269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664251+0000","last_change":"2023-12-08T08:38:19.621125+0000","last_active":"2023-12-08T08:38:23.664251+0000","last_peered":"2023-12-08T08:38:23.664251+0000","last_clean":"2023-12-08T08:38:23.664251+0000","last_became_active":"2023-12-08T08:38:19.620831+0000","last_became_peered":"2023-12-08T08:38:19.620831+0000","last_unstale":"2023-12-08T08:38:23.664251+0000","last_undegraded":"2023-12-08T08:38:23.664251+0000","last_fullsized":"2023-12-08T08:38:23.664251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852760+0000","last_change":"2023-12-08T08:38:19.624484+0000","last_active":"2023-12-08T08:38:23.852760+0000","last_peered":"2023-12-08T08:38:23.852760+0000","last_clean":"2023-12-08T08:38:23.852760+0000","last_became_active":"2023-12-08T08:38:19.623921+0000","last_became_peered":"2023-12-08T08:38:19.623921+0000","last_unstale":"2023-12-08T08:38:23.852760+0000","last_undegraded":"2023-12-08T08:38:23.852760+0000","last_fullsized":"2023-12-08T08:38:23.852760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:32:54.146197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4637,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":99918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":99952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4730000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":690798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5920000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-08T08:38:47.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-08T08:38:47.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.281+0000 7f3876951640 1 Processor -- start 2023-12-08T08:38:47.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.281+0000 7f3876951640 1 -- start start 2023-12-08T08:38:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f3876951640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38700d1d60 0x7f38700d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f3876951640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f387015ee90 con 0x7f387015d580 2023-12-08T08:38:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f3876951640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38700d0fe0 con 0x7f387015f890 2023-12-08T08:38:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f3876951640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38700d1280 con 0x7f38700d1d60 2023-12-08T08:38:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f386ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f387015d580 0x7f387015f180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37498/0 (socket says 172.21.15.16:37498) 2023-12-08T08:38:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f386ffff640 1 -- 172.21.15.16:0/2692725082 learned_addr learned my addr 172.21.15.16:0/2692725082 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f386effd640 1 -- 172.21.15.16:0/2692725082 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007491250 0 0) 0x7f387015ee90 con 0x7f387015d580 2023-12-08T08:38:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f386f7fe640 1 --2- 172.21.15.16:0/2692725082 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38700d1d60 0x7f38700d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f386effd640 1 -- 172.21.15.16:0/2692725082 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3858003520 con 0x7f387015d580 2023-12-08T08:38:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f386effd640 1 -- 172.21.15.16:0/2692725082 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2882269807 0 0) 0x7f3858003520 con 0x7f387015d580 2023-12-08T08:38:47.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f386effd640 1 -- 172.21.15.16:0/2692725082 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38700d1d60 msgr2=0x7f38700d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f386effd640 1 --2- 172.21.15.16:0/2692725082 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38700d1d60 0x7f38700d3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f386effd640 1 -- 172.21.15.16:0/2692725082 >> v1:172.21.15.112:6789/0 conn(0x7f387015f890 legacy=0x7f387015d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f386effd640 1 -- 172.21.15.16:0/2692725082 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38700d1560 con 0x7f387015d580 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.282+0000 7f386effd640 1 -- 172.21.15.16:0/2692725082 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f3864001670 con 0x7f387015d580 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f386effd640 1 -- 172.21.15.16:0/2692725082 >> v1:172.21.15.16:6789/0 conn(0x7f387015d580 legacy=0x7f387015f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f386effd640 1 --2- 172.21.15.16:0/2692725082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38580039a0 0x7f3858003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f386effd640 1 --2- 172.21.15.16:0/2692725082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38580047c0 0x7f3858004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f386effd640 1 --2- 172.21.15.16:0/2692725082 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38700d1d60 0x7f38580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f386effd640 1 -- 172.21.15.16:0/2692725082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38700d1280 con 0x7f38580039a0 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f386effd640 1 -- 172.21.15.16:0/2692725082 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385800bae0 con 0x7f38580047c0 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f386effd640 1 -- 172.21.15.16:0/2692725082 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385800bca0 con 0x7f38700d1d60 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f386ffff640 1 --2- 172.21.15.16:0/2692725082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38580039a0 0x7f3858003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f3874eee640 1 --2- 172.21.15.16:0/2692725082 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38700d1d60 0x7f38580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f386f7fe640 1 --2- 172.21.15.16:0/2692725082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38580047c0 0x7f3858004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f386ffff640 1 -- 172.21.15.16:0/2692725082 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38700d1d60 msgr2=0x7f38580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f386ffff640 1 --2- 172.21.15.16:0/2692725082 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38700d1d60 0x7f38580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f386ffff640 1 -- 172.21.15.16:0/2692725082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38580047c0 msgr2=0x7f3858004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f386ffff640 1 --2- 172.21.15.16:0/2692725082 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38580047c0 0x7f3858004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.283+0000 7f386ffff640 1 -- 172.21.15.16:0/2692725082 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3858003520 con 0x7f38580039a0 2023-12-08T08:38:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.284+0000 7f386ffff640 1 --2- 172.21.15.16:0/2692725082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38580039a0 0x7f3858003dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f386400c240 tx=0x7f38580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8a5eb9647a04ad server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.284+0000 7f386effd640 1 -- 172.21.15.16:0/2692725082 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864005ad0 con 0x7f38580039a0 2023-12-08T08:38:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.284+0000 7f386effd640 1 -- 172.21.15.16:0/2692725082 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3864005c30 con 0x7f38580039a0 2023-12-08T08:38:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.284+0000 7f386effd640 1 -- 172.21.15.16:0/2692725082 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386402c8b0 con 0x7f38580039a0 2023-12-08T08:38:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.284+0000 7f3876951640 1 -- 172.21.15.16:0/2692725082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38580039a0 msgr2=0x7f3858003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.284+0000 7f3876951640 1 --2- 172.21.15.16:0/2692725082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38580039a0 0x7f3858003dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f386400c240 tx=0x7f38580057f0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.284+0000 7f3874eee640 1 -- 172.21.15.16:0/2692725082 reap_dead start 2023-12-08T08:38:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.284+0000 7f3876951640 1 -- 172.21.15.16:0/2692725082 shutdown_connections 2023-12-08T08:38:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.284+0000 7f3876951640 1 -- 172.21.15.16:0/2692725082 >> 172.21.15.16:0/2692725082 conn(0x7f38700d2fb0 msgr2=0x7f38700d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.285+0000 7f3876951640 1 -- 172.21.15.16:0/2692725082 shutdown_connections 2023-12-08T08:38:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.285+0000 7f3876951640 1 -- 172.21.15.16:0/2692725082 wait complete. 2023-12-08T08:38:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.286+0000 7f3876951640 1 Processor -- start 2023-12-08T08:38:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.286+0000 7f3876951640 1 -- start start 2023-12-08T08:38:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f3876951640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38580039a0 0x7f38700ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f3876951640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38700c8100 0x7f387014dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f3876951640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38700c8580 0x7f387014e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f3876951640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387009dac0 con 0x7f38580039a0 2023-12-08T08:38:47.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f3876951640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38700ca4a0 con 0x7f38700c8580 2023-12-08T08:38:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f3876951640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38700ca780 con 0x7f38700c8100 2023-12-08T08:38:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f386ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38580039a0 0x7f38700ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f386ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38580039a0 0x7f38700ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53480/0 (socket says 172.21.15.16:53480) 2023-12-08T08:38:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f386ffff640 1 -- 172.21.15.16:0/214734205 learned_addr learned my addr 172.21.15.16:0/214734205 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f3874eee640 1 --2- 172.21.15.16:0/214734205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38700c8580 0x7f387014e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f386f7fe640 1 --2- 172.21.15.16:0/214734205 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38700c8100 0x7f387014dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f386ffff640 1 -- 172.21.15.16:0/214734205 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38700c8100 msgr2=0x7f387014dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f386ffff640 1 --2- 172.21.15.16:0/214734205 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38700c8100 0x7f387014dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f386ffff640 1 -- 172.21.15.16:0/214734205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38700c8580 msgr2=0x7f387014e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f386ffff640 1 --2- 172.21.15.16:0/214734205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38700c8580 0x7f387014e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f386ffff640 1 -- 172.21.15.16:0/214734205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38700caa60 con 0x7f38580039a0 2023-12-08T08:38:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f386ffff640 1 --2- 172.21.15.16:0/214734205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38580039a0 0x7f38700ccd50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f386402c450 tx=0x7f3864001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f386cff9640 1 -- 172.21.15.16:0/214734205 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3864039070 con 0x7f38580039a0 2023-12-08T08:38:47.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f386cff9640 1 -- 172.21.15.16:0/214734205 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386402ccc0 con 0x7f38580039a0 2023-12-08T08:38:47.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f3876951640 1 -- 172.21.15.16:0/214734205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38700cad40 con 0x7f38580039a0 2023-12-08T08:38:47.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f386cff9640 1 -- 172.21.15.16:0/214734205 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386403c040 con 0x7f38580039a0 2023-12-08T08:38:47.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.287+0000 7f3876951640 1 -- 172.21.15.16:0/214734205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38700ca0c0 con 0x7f38580039a0 2023-12-08T08:38:47.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.288+0000 7f3876951640 1 -- 172.21.15.16:0/214734205 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38700ca250 con 0x7f38580039a0 2023-12-08T08:38:47.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.289+0000 7f386cff9640 1 -- 172.21.15.16:0/214734205 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3864032070 con 0x7f38580039a0 2023-12-08T08:38:47.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.289+0000 7f386cff9640 1 --2- 172.21.15.16:0/214734205 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f385c06ae70 0x7f385c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.289+0000 7f386cff9640 1 -- 172.21.15.16:0/214734205 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f38640aaa70 con 0x7f38580039a0 2023-12-08T08:38:47.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.289+0000 7f386f7fe640 1 --2- 172.21.15.16:0/214734205 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f385c06ae70 0x7f385c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.290+0000 7f386f7fe640 1 --2- 172.21.15.16:0/214734205 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f385c06ae70 0x7f385c06d330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f38700ccab0 tx=0x7f3860009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:47.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.294+0000 7f386cff9640 1 -- 172.21.15.16:0/214734205 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38640758f0 con 0x7f38580039a0 2023-12-08T08:38:47.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.468+0000 7f3876951640 1 -- 172.21.15.16:0/214734205 --> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38700b33c0 con 0x7f385c06ae70 2023-12-08T08:38:47.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.477+0000 7f386cff9640 1 -- 172.21.15.16:0/214734205 <== mgr.4111 v2:172.21.15.112:6832/804067392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345244 (secure 0 0 0) 0x7f3860019070 con 0x7f385c06ae70 2023-12-08T08:38:47.483 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:47.484 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-12-08T08:38:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.480+0000 7f3876951640 1 -- 172.21.15.16:0/214734205 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f385c06ae70 msgr2=0x7f385c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.480+0000 7f3876951640 1 --2- 172.21.15.16:0/214734205 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f385c06ae70 0x7f385c06d330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f38700ccab0 tx=0x7f3860009190 comp rx=0 tx=0).stop 2023-12-08T08:38:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.480+0000 7f3876951640 1 -- 172.21.15.16:0/214734205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38580039a0 msgr2=0x7f38700ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.480+0000 7f3876951640 1 --2- 172.21.15.16:0/214734205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38580039a0 0x7f38700ccd50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f386402c450 tx=0x7f3864001a00 comp rx=0 tx=0).stop 2023-12-08T08:38:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.480+0000 7f3876951640 1 -- 172.21.15.16:0/214734205 shutdown_connections 2023-12-08T08:38:47.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.480+0000 7f3876951640 1 --2- 172.21.15.16:0/214734205 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f385c06ae70 0x7f385c06d330 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.480+0000 7f3876951640 1 --2- 172.21.15.16:0/214734205 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f38700c8580 0x7f387014e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.480+0000 7f3876951640 1 --2- 172.21.15.16:0/214734205 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f38700c8100 0x7f387014dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.480+0000 7f3876951640 1 --2- 172.21.15.16:0/214734205 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38580039a0 0x7f38700ccd50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.480+0000 7f3876951640 1 -- 172.21.15.16:0/214734205 >> 172.21.15.16:0/214734205 conn(0x7f38700d2fb0 msgr2=0x7f387015f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.480+0000 7f3876951640 1 -- 172.21.15.16:0/214734205 shutdown_connections 2023-12-08T08:38:47.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.481+0000 7f3876951640 1 -- 172.21.15.16:0/214734205 wait complete. 2023-12-08T08:38:47.494 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-12-08T08:38:46.981252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595005,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231864,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754742856,"statfs":{"total":773094113280,"available":772856684544,"internally_reserved":0,"allocated":3284992,"data_stored":1988346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003518"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659961+0000","last_change":"2023-12-08T08:38:18.634619+0000","last_active":"2023-12-08T08:38:23.659961+0000","last_peered":"2023-12-08T08:38:23.659961+0000","last_clean":"2023-12-08T08:38:23.659961+0000","last_became_active":"2023-12-08T08:38:18.634328+0000","last_became_peered":"2023-12-08T08:38:18.634328+0000","last_unstale":"2023-12-08T08:38:23.659961+0000","last_undegraded":"2023-12-08T08:38:23.659961+0000","last_fullsized":"2023-12-08T08:38:23.659961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:11.067589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656122+0000","last_change":"2023-12-08T08:38:19.627512+0000","last_active":"2023-12-08T08:38:23.656122+0000","last_peered":"2023-12-08T08:38:23.656122+0000","last_clean":"2023-12-08T08:38:23.656122+0000","last_became_active":"2023-12-08T08:38:19.625456+0000","last_became_peered":"2023-12-08T08:38:19.625456+0000","last_unstale":"2023-12-08T08:38:23.656122+0000","last_undegraded":"2023-12-08T08:38:23.656122+0000","last_fullsized":"2023-12-08T08:38:23.656122+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576137+0000","last_change":"2023-12-08T08:38:23.856216+0000","last_active":"2023-12-08T08:38:25.576137+0000","last_peered":"2023-12-08T08:38:25.576137+0000","last_clean":"2023-12-08T08:38:25.576137+0000","last_became_active":"2023-12-08T08:38:23.855623+0000","last_became_peered":"2023-12-08T08:38:23.855623+0000","last_unstale":"2023-12-08T08:38:25.576137+0000","last_undegraded":"2023-12-08T08:38:25.576137+0000","last_fullsized":"2023-12-08T08:38:25.576137+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:26:16.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656314+0000","last_change":"2023-12-08T08:38:18.631598+0000","last_active":"2023-12-08T08:38:23.656314+0000","last_peered":"2023-12-08T08:38:23.656314+0000","last_clean":"2023-12-08T08:38:23.656314+0000","last_became_active":"2023-12-08T08:38:18.631295+0000","last_became_peered":"2023-12-08T08:38:18.631295+0000","last_unstale":"2023-12-08T08:38:23.656314+0000","last_undegraded":"2023-12-08T08:38:23.656314+0000","last_fullsized":"2023-12-08T08:38:23.656314+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659744+0000","last_change":"2023-12-08T08:38:19.633463+0000","last_active":"2023-12-08T08:38:23.659744+0000","last_peered":"2023-12-08T08:38:23.659744+0000","last_clean":"2023-12-08T08:38:23.659744+0000","last_became_active":"2023-12-08T08:38:19.633094+0000","last_became_peered":"2023-12-08T08:38:19.633094+0000","last_unstale":"2023-12-08T08:38:23.659744+0000","last_undegraded":"2023-12-08T08:38:23.659744+0000","last_fullsized":"2023-12-08T08:38:23.659744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608661+0000","last_change":"2023-12-08T08:38:23.674366+0000","last_active":"2023-12-08T08:38:24.608661+0000","last_peered":"2023-12-08T08:38:24.608661+0000","last_clean":"2023-12-08T08:38:24.608661+0000","last_became_active":"2023-12-08T08:38:23.673810+0000","last_became_peered":"2023-12-08T08:38:23.673810+0000","last_unstale":"2023-12-08T08:38:24.608661+0000","last_undegraded":"2023-12-08T08:38:24.608661+0000","last_fullsized":"2023-12-08T08:38:24.608661+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:47.374495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852405+0000","last_change":"2023-12-08T08:38:18.633940+0000","last_active":"2023-12-08T08:38:23.852405+0000","last_peered":"2023-12-08T08:38:23.852405+0000","last_clean":"2023-12-08T08:38:23.852405+0000","last_became_active":"2023-12-08T08:38:18.631964+0000","last_became_peered":"2023-12-08T08:38:18.631964+0000","last_unstale":"2023-12-08T08:38:23.852405+0000","last_undegraded":"2023-12-08T08:38:23.852405+0000","last_fullsized":"2023-12-08T08:38:23.852405+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:37.317712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661342+0000","last_change":"2023-12-08T08:38:19.627445+0000","last_active":"2023-12-08T08:38:23.661342+0000","last_peered":"2023-12-08T08:38:23.661342+0000","last_clean":"2023-12-08T08:38:23.661342+0000","last_became_active":"2023-12-08T08:38:19.627174+0000","last_became_peered":"2023-12-08T08:38:19.627174+0000","last_unstale":"2023-12-08T08:38:23.661342+0000","last_undegraded":"2023-12-08T08:38:23.661342+0000","last_fullsized":"2023-12-08T08:38:23.661342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:32.116348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609171+0000","last_change":"2023-12-08T08:38:23.674338+0000","last_active":"2023-12-08T08:38:24.609171+0000","last_peered":"2023-12-08T08:38:24.609171+0000","last_clean":"2023-12-08T08:38:24.609171+0000","last_became_active":"2023-12-08T08:38:23.673614+0000","last_became_peered":"2023-12-08T08:38:23.673614+0000","last_unstale":"2023-12-08T08:38:24.609171+0000","last_undegraded":"2023-12-08T08:38:24.609171+0000","last_fullsized":"2023-12-08T08:38:24.609171+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:59.405196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852122+0000","last_change":"2023-12-08T08:38:18.633561+0000","last_active":"2023-12-08T08:38:23.852122+0000","last_peered":"2023-12-08T08:38:23.852122+0000","last_clean":"2023-12-08T08:38:23.852122+0000","last_became_active":"2023-12-08T08:38:18.632646+0000","last_became_peered":"2023-12-08T08:38:18.632646+0000","last_unstale":"2023-12-08T08:38:23.852122+0000","last_undegraded":"2023-12-08T08:38:23.852122+0000","last_fullsized":"2023-12-08T08:38:23.852122+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:36:34.339195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662141+0000","last_change":"2023-12-08T08:38:19.632216+0000","last_active":"2023-12-08T08:38:23.662141+0000","last_peered":"2023-12-08T08:38:23.662141+0000","last_clean":"2023-12-08T08:38:23.662141+0000","last_became_active":"2023-12-08T08:38:19.631599+0000","last_became_peered":"2023-12-08T08:38:19.631599+0000","last_unstale":"2023-12-08T08:38:23.662141+0000","last_undegraded":"2023-12-08T08:38:23.662141+0000","last_fullsized":"2023-12-08T08:38:23.662141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:11.797914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608922+0000","last_change":"2023-12-08T08:38:23.669631+0000","last_active":"2023-12-08T08:38:24.608922+0000","last_peered":"2023-12-08T08:38:24.608922+0000","last_clean":"2023-12-08T08:38:24.608922+0000","last_became_active":"2023-12-08T08:38:23.669222+0000","last_became_peered":"2023-12-08T08:38:23.669222+0000","last_unstale":"2023-12-08T08:38:24.608922+0000","last_undegraded":"2023-12-08T08:38:24.608922+0000","last_fullsized":"2023-12-08T08:38:24.608922+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:49:33.936211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664041+0000","last_change":"2023-12-08T08:38:19.620724+0000","last_active":"2023-12-08T08:38:23.664041+0000","last_peered":"2023-12-08T08:38:23.664041+0000","last_clean":"2023-12-08T08:38:23.664041+0000","last_became_active":"2023-12-08T08:38:19.620366+0000","last_became_peered":"2023-12-08T08:38:19.620366+0000","last_unstale":"2023-12-08T08:38:23.664041+0000","last_undegraded":"2023-12-08T08:38:23.664041+0000","last_fullsized":"2023-12-08T08:38:23.664041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:00:01.146409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852136+0000","last_change":"2023-12-08T08:38:18.632086+0000","last_active":"2023-12-08T08:38:23.852136+0000","last_peered":"2023-12-08T08:38:23.852136+0000","last_clean":"2023-12-08T08:38:23.852136+0000","last_became_active":"2023-12-08T08:38:18.631756+0000","last_became_peered":"2023-12-08T08:38:18.631756+0000","last_unstale":"2023-12-08T08:38:23.852136+0000","last_undegraded":"2023-12-08T08:38:23.852136+0000","last_fullsized":"2023-12-08T08:38:23.852136+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:14.342540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609352+0000","last_change":"2023-12-08T08:38:23.673898+0000","last_active":"2023-12-08T08:38:24.609352+0000","last_peered":"2023-12-08T08:38:24.609352+0000","last_clean":"2023-12-08T08:38:24.609352+0000","last_became_active":"2023-12-08T08:38:23.673389+0000","last_became_peered":"2023-12-08T08:38:23.673389+0000","last_unstale":"2023-12-08T08:38:24.609352+0000","last_undegraded":"2023-12-08T08:38:24.609352+0000","last_fullsized":"2023-12-08T08:38:24.609352+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:15.835775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655652+0000","last_change":"2023-12-08T08:38:19.627148+0000","last_active":"2023-12-08T08:38:23.655652+0000","last_peered":"2023-12-08T08:38:23.655652+0000","last_clean":"2023-12-08T08:38:23.655652+0000","last_became_active":"2023-12-08T08:38:19.626846+0000","last_became_peered":"2023-12-08T08:38:19.626846+0000","last_unstale":"2023-12-08T08:38:23.655652+0000","last_undegraded":"2023-12-08T08:38:23.655652+0000","last_fullsized":"2023-12-08T08:38:23.655652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.666654+0000","last_change":"2023-12-08T08:38:18.632191+0000","last_active":"2023-12-08T08:38:23.666654+0000","last_peered":"2023-12-08T08:38:23.666654+0000","last_clean":"2023-12-08T08:38:23.666654+0000","last_became_active":"2023-12-08T08:38:18.631927+0000","last_became_peered":"2023-12-08T08:38:18.631927+0000","last_unstale":"2023-12-08T08:38:23.666654+0000","last_undegraded":"2023-12-08T08:38:23.666654+0000","last_fullsized":"2023-12-08T08:38:23.666654+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:10.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608968+0000","last_change":"2023-12-08T08:38:23.678875+0000","last_active":"2023-12-08T08:38:24.608968+0000","last_peered":"2023-12-08T08:38:24.608968+0000","last_clean":"2023-12-08T08:38:24.608968+0000","last_became_active":"2023-12-08T08:38:23.678332+0000","last_became_peered":"2023-12-08T08:38:23.678332+0000","last_unstale":"2023-12-08T08:38:24.608968+0000","last_undegraded":"2023-12-08T08:38:24.608968+0000","last_fullsized":"2023-12-08T08:38:24.608968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:36.154208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656075+0000","last_change":"2023-12-08T08:38:19.625657+0000","last_active":"2023-12-08T08:38:23.656075+0000","last_peered":"2023-12-08T08:38:23.656075+0000","last_clean":"2023-12-08T08:38:23.656075+0000","last_became_active":"2023-12-08T08:38:19.624790+0000","last_became_peered":"2023-12-08T08:38:19.624790+0000","last_unstale":"2023-12-08T08:38:23.656075+0000","last_undegraded":"2023-12-08T08:38:23.656075+0000","last_fullsized":"2023-12-08T08:38:23.656075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:32.116348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664683+0000","last_change":"2023-12-08T08:38:18.637627+0000","last_active":"2023-12-08T08:38:23.664683+0000","last_peered":"2023-12-08T08:38:23.664683+0000","last_clean":"2023-12-08T08:38:23.664683+0000","last_became_active":"2023-12-08T08:38:18.637335+0000","last_became_peered":"2023-12-08T08:38:18.637335+0000","last_unstale":"2023-12-08T08:38:23.664683+0000","last_undegraded":"2023-12-08T08:38:23.664683+0000","last_fullsized":"2023-12-08T08:38:23.664683+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.828080+0000","last_change":"2023-12-08T08:38:23.852913+0000","last_active":"2023-12-08T08:38:25.828080+0000","last_peered":"2023-12-08T08:38:25.828080+0000","last_clean":"2023-12-08T08:38:25.828080+0000","last_became_active":"2023-12-08T08:38:23.852508+0000","last_became_peered":"2023-12-08T08:38:23.852508+0000","last_unstale":"2023-12-08T08:38:25.828080+0000","last_undegraded":"2023-12-08T08:38:25.828080+0000","last_fullsized":"2023-12-08T08:38:25.828080+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:48.357070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658598+0000","last_change":"2023-12-08T08:38:19.623518+0000","last_active":"2023-12-08T08:38:23.658598+0000","last_peered":"2023-12-08T08:38:23.658598+0000","last_clean":"2023-12-08T08:38:23.658598+0000","last_became_active":"2023-12-08T08:38:19.623240+0000","last_became_peered":"2023-12-08T08:38:19.623240+0000","last_unstale":"2023-12-08T08:38:23.658598+0000","last_undegraded":"2023-12-08T08:38:23.658598+0000","last_fullsized":"2023-12-08T08:38:23.658598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:43.773215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851377+0000","last_change":"2023-12-08T08:38:18.633585+0000","last_active":"2023-12-08T08:38:23.851377+0000","last_peered":"2023-12-08T08:38:23.851377+0000","last_clean":"2023-12-08T08:38:23.851377+0000","last_became_active":"2023-12-08T08:38:18.633112+0000","last_became_peered":"2023-12-08T08:38:18.633112+0000","last_unstale":"2023-12-08T08:38:23.851377+0000","last_undegraded":"2023-12-08T08:38:23.851377+0000","last_fullsized":"2023-12-08T08:38:23.851377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:42.768579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609486+0000","last_change":"2023-12-08T08:38:23.678810+0000","last_active":"2023-12-08T08:38:24.609486+0000","last_peered":"2023-12-08T08:38:24.609486+0000","last_clean":"2023-12-08T08:38:24.609486+0000","last_became_active":"2023-12-08T08:38:23.678331+0000","last_became_peered":"2023-12-08T08:38:23.678331+0000","last_unstale":"2023-12-08T08:38:24.609486+0000","last_undegraded":"2023-12-08T08:38:24.609486+0000","last_fullsized":"2023-12-08T08:38:24.609486+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:17.394710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.666222+0000","last_change":"2023-12-08T08:38:19.619485+0000","last_active":"2023-12-08T08:38:23.666222+0000","last_peered":"2023-12-08T08:38:23.666222+0000","last_clean":"2023-12-08T08:38:23.666222+0000","last_became_active":"2023-12-08T08:38:19.618880+0000","last_became_peered":"2023-12-08T08:38:19.618880+0000","last_unstale":"2023-12-08T08:38:23.666222+0000","last_undegraded":"2023-12-08T08:38:23.666222+0000","last_fullsized":"2023-12-08T08:38:23.666222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:43.336635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665379+0000","last_change":"2023-12-08T08:38:18.633093+0000","last_active":"2023-12-08T08:38:23.665379+0000","last_peered":"2023-12-08T08:38:23.665379+0000","last_clean":"2023-12-08T08:38:23.665379+0000","last_became_active":"2023-12-08T08:38:18.632832+0000","last_became_peered":"2023-12-08T08:38:18.632832+0000","last_unstale":"2023-12-08T08:38:23.665379+0000","last_undegraded":"2023-12-08T08:38:23.665379+0000","last_fullsized":"2023-12-08T08:38:23.665379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:37.317712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608658+0000","last_change":"2023-12-08T08:38:23.853769+0000","last_active":"2023-12-08T08:38:24.608658+0000","last_peered":"2023-12-08T08:38:24.608658+0000","last_clean":"2023-12-08T08:38:24.608658+0000","last_became_active":"2023-12-08T08:38:23.853211+0000","last_became_peered":"2023-12-08T08:38:23.853211+0000","last_unstale":"2023-12-08T08:38:24.608658+0000","last_undegraded":"2023-12-08T08:38:24.608658+0000","last_fullsized":"2023-12-08T08:38:24.608658+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:59:15.111403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656171+0000","last_change":"2023-12-08T08:38:19.626321+0000","last_active":"2023-12-08T08:38:23.656171+0000","last_peered":"2023-12-08T08:38:23.656171+0000","last_clean":"2023-12-08T08:38:23.656171+0000","last_became_active":"2023-12-08T08:38:19.624741+0000","last_became_peered":"2023-12-08T08:38:19.624741+0000","last_unstale":"2023-12-08T08:38:23.656171+0000","last_undegraded":"2023-12-08T08:38:23.656171+0000","last_fullsized":"2023-12-08T08:38:23.656171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:49:29.898351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664728+0000","last_change":"2023-12-08T08:38:18.636600+0000","last_active":"2023-12-08T08:38:23.664728+0000","last_peered":"2023-12-08T08:38:23.664728+0000","last_clean":"2023-12-08T08:38:23.664728+0000","last_became_active":"2023-12-08T08:38:18.633400+0000","last_became_peered":"2023-12-08T08:38:18.633400+0000","last_unstale":"2023-12-08T08:38:23.664728+0000","last_undegraded":"2023-12-08T08:38:23.664728+0000","last_fullsized":"2023-12-08T08:38:23.664728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:37.317712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576086+0000","last_change":"2023-12-08T08:38:23.856023+0000","last_active":"2023-12-08T08:38:25.576086+0000","last_peered":"2023-12-08T08:38:25.576086+0000","last_clean":"2023-12-08T08:38:25.576086+0000","last_became_active":"2023-12-08T08:38:23.855599+0000","last_became_peered":"2023-12-08T08:38:23.855599+0000","last_unstale":"2023-12-08T08:38:25.576086+0000","last_undegraded":"2023-12-08T08:38:25.576086+0000","last_fullsized":"2023-12-08T08:38:25.576086+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T20:21:44.003574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662450+0000","last_change":"2023-12-08T08:38:19.622140+0000","last_active":"2023-12-08T08:38:23.662450+0000","last_peered":"2023-12-08T08:38:23.662450+0000","last_clean":"2023-12-08T08:38:23.662450+0000","last_became_active":"2023-12-08T08:38:19.621784+0000","last_became_peered":"2023-12-08T08:38:19.621784+0000","last_unstale":"2023-12-08T08:38:23.662450+0000","last_undegraded":"2023-12-08T08:38:23.662450+0000","last_fullsized":"2023-12-08T08:38:23.662450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852845+0000","last_change":"2023-12-08T08:38:18.634123+0000","last_active":"2023-12-08T08:38:23.852845+0000","last_peered":"2023-12-08T08:38:23.852845+0000","last_clean":"2023-12-08T08:38:23.852845+0000","last_became_active":"2023-12-08T08:38:18.632874+0000","last_became_peered":"2023-12-08T08:38:18.632874+0000","last_unstale":"2023-12-08T08:38:23.852845+0000","last_undegraded":"2023-12-08T08:38:23.852845+0000","last_fullsized":"2023-12-08T08:38:23.852845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:07.497079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.326786+0000","last_change":"2023-12-08T08:38:23.852386+0000","last_active":"2023-12-08T08:38:26.326786+0000","last_peered":"2023-12-08T08:38:26.326786+0000","last_clean":"2023-12-08T08:38:26.326786+0000","last_became_active":"2023-12-08T08:38:23.851973+0000","last_became_peered":"2023-12-08T08:38:23.851973+0000","last_unstale":"2023-12-08T08:38:26.326786+0000","last_undegraded":"2023-12-08T08:38:26.326786+0000","last_fullsized":"2023-12-08T08:38:26.326786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:22.372635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662330+0000","last_change":"2023-12-08T08:38:19.631928+0000","last_active":"2023-12-08T08:38:23.662330+0000","last_peered":"2023-12-08T08:38:23.662330+0000","last_clean":"2023-12-08T08:38:23.662330+0000","last_became_active":"2023-12-08T08:38:19.631447+0000","last_became_peered":"2023-12-08T08:38:19.631447+0000","last_unstale":"2023-12-08T08:38:23.662330+0000","last_undegraded":"2023-12-08T08:38:23.662330+0000","last_fullsized":"2023-12-08T08:38:23.662330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:13.356849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662548+0000","last_change":"2023-12-08T08:38:18.634530+0000","last_active":"2023-12-08T08:38:23.662548+0000","last_peered":"2023-12-08T08:38:23.662548+0000","last_clean":"2023-12-08T08:38:23.662548+0000","last_became_active":"2023-12-08T08:38:18.634239+0000","last_became_peered":"2023-12-08T08:38:18.634239+0000","last_unstale":"2023-12-08T08:38:23.662548+0000","last_undegraded":"2023-12-08T08:38:23.662548+0000","last_fullsized":"2023-12-08T08:38:23.662548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:42.768579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.827628+0000","last_change":"2023-12-08T08:38:23.676640+0000","last_active":"2023-12-08T08:38:25.827628+0000","last_peered":"2023-12-08T08:38:25.827628+0000","last_clean":"2023-12-08T08:38:25.827628+0000","last_became_active":"2023-12-08T08:38:23.675999+0000","last_became_peered":"2023-12-08T08:38:23.675999+0000","last_unstale":"2023-12-08T08:38:25.827628+0000","last_undegraded":"2023-12-08T08:38:25.827628+0000","last_fullsized":"2023-12-08T08:38:25.827628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:15.835775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658939+0000","last_change":"2023-12-08T08:38:19.615261+0000","last_active":"2023-12-08T08:38:23.658939+0000","last_peered":"2023-12-08T08:38:23.658939+0000","last_clean":"2023-12-08T08:38:23.658939+0000","last_became_active":"2023-12-08T08:38:19.614990+0000","last_became_peered":"2023-12-08T08:38:19.614990+0000","last_unstale":"2023-12-08T08:38:23.658939+0000","last_undegraded":"2023-12-08T08:38:23.658939+0000","last_fullsized":"2023-12-08T08:38:23.658939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:05:08.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662040+0000","last_change":"2023-12-08T08:38:18.630062+0000","last_active":"2023-12-08T08:38:23.662040+0000","last_peered":"2023-12-08T08:38:23.662040+0000","last_clean":"2023-12-08T08:38:23.662040+0000","last_became_active":"2023-12-08T08:38:18.629798+0000","last_became_peered":"2023-12-08T08:38:18.629798+0000","last_unstale":"2023-12-08T08:38:23.662040+0000","last_undegraded":"2023-12-08T08:38:23.662040+0000","last_fullsized":"2023-12-08T08:38:23.662040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:13.807850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.827791+0000","last_change":"2023-12-08T08:38:23.674624+0000","last_active":"2023-12-08T08:38:25.827791+0000","last_peered":"2023-12-08T08:38:25.827791+0000","last_clean":"2023-12-08T08:38:25.827791+0000","last_became_active":"2023-12-08T08:38:23.674121+0000","last_became_peered":"2023-12-08T08:38:23.674121+0000","last_unstale":"2023-12-08T08:38:25.827791+0000","last_undegraded":"2023-12-08T08:38:25.827791+0000","last_fullsized":"2023-12-08T08:38:25.827791+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:17.394710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665266+0000","last_change":"2023-12-08T08:38:19.621679+0000","last_active":"2023-12-08T08:38:23.665266+0000","last_peered":"2023-12-08T08:38:23.665266+0000","last_clean":"2023-12-08T08:38:23.665266+0000","last_became_active":"2023-12-08T08:38:19.621388+0000","last_became_peered":"2023-12-08T08:38:19.621388+0000","last_unstale":"2023-12-08T08:38:23.665266+0000","last_undegraded":"2023-12-08T08:38:23.665266+0000","last_fullsized":"2023-12-08T08:38:23.665266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662810+0000","last_change":"2023-12-08T08:38:18.632407+0000","last_active":"2023-12-08T08:38:23.662810+0000","last_peered":"2023-12-08T08:38:23.662810+0000","last_clean":"2023-12-08T08:38:23.662810+0000","last_became_active":"2023-12-08T08:38:18.632115+0000","last_became_peered":"2023-12-08T08:38:18.632115+0000","last_unstale":"2023-12-08T08:38:23.662810+0000","last_undegraded":"2023-12-08T08:38:23.662810+0000","last_fullsized":"2023-12-08T08:38:23.662810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:05:14.591351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.828126+0000","last_change":"2023-12-08T08:38:23.675339+0000","last_active":"2023-12-08T08:38:25.828126+0000","last_peered":"2023-12-08T08:38:25.828126+0000","last_clean":"2023-12-08T08:38:25.828126+0000","last_became_active":"2023-12-08T08:38:23.674249+0000","last_became_peered":"2023-12-08T08:38:23.674249+0000","last_unstale":"2023-12-08T08:38:25.828126+0000","last_undegraded":"2023-12-08T08:38:25.828126+0000","last_fullsized":"2023-12-08T08:38:25.828126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:34:29.925547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.660253+0000","last_change":"2023-12-08T08:38:19.611646+0000","last_active":"2023-12-08T08:38:23.660253+0000","last_peered":"2023-12-08T08:38:23.660253+0000","last_clean":"2023-12-08T08:38:23.660253+0000","last_became_active":"2023-12-08T08:38:19.611315+0000","last_became_peered":"2023-12-08T08:38:19.611315+0000","last_unstale":"2023-12-08T08:38:23.660253+0000","last_undegraded":"2023-12-08T08:38:23.660253+0000","last_fullsized":"2023-12-08T08:38:23.660253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:11.612354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851317+0000","last_change":"2023-12-08T08:38:18.634473+0000","last_active":"2023-12-08T08:38:23.851317+0000","last_peered":"2023-12-08T08:38:23.851317+0000","last_clean":"2023-12-08T08:38:23.851317+0000","last_became_active":"2023-12-08T08:38:18.633755+0000","last_became_peered":"2023-12-08T08:38:18.633755+0000","last_unstale":"2023-12-08T08:38:23.851317+0000","last_undegraded":"2023-12-08T08:38:23.851317+0000","last_fullsized":"2023-12-08T08:38:23.851317+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:11.067589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.327085+0000","last_change":"2023-12-08T08:38:23.852846+0000","last_active":"2023-12-08T08:38:26.327085+0000","last_peered":"2023-12-08T08:38:26.327085+0000","last_clean":"2023-12-08T08:38:26.327085+0000","last_became_active":"2023-12-08T08:38:23.852393+0000","last_became_peered":"2023-12-08T08:38:23.852393+0000","last_unstale":"2023-12-08T08:38:26.327085+0000","last_undegraded":"2023-12-08T08:38:26.327085+0000","last_fullsized":"2023-12-08T08:38:26.327085+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:32:58.184058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655788+0000","last_change":"2023-12-08T08:38:19.634986+0000","last_active":"2023-12-08T08:38:23.655788+0000","last_peered":"2023-12-08T08:38:23.655788+0000","last_clean":"2023-12-08T08:38:23.655788+0000","last_became_active":"2023-12-08T08:38:19.634682+0000","last_became_peered":"2023-12-08T08:38:19.634682+0000","last_unstale":"2023-12-08T08:38:23.655788+0000","last_undegraded":"2023-12-08T08:38:23.655788+0000","last_fullsized":"2023-12-08T08:38:23.655788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661184+0000","last_change":"2023-12-08T08:38:18.631253+0000","last_active":"2023-12-08T08:38:23.661184+0000","last_peered":"2023-12-08T08:38:23.661184+0000","last_clean":"2023-12-08T08:38:23.661184+0000","last_became_active":"2023-12-08T08:38:18.630979+0000","last_became_peered":"2023-12-08T08:38:18.630979+0000","last_unstale":"2023-12-08T08:38:23.661184+0000","last_undegraded":"2023-12-08T08:38:23.661184+0000","last_fullsized":"2023-12-08T08:38:23.661184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:00:00.141773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":169,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851733+0000","last_change":"2023-12-08T08:38:08.276815+0000","last_active":"2023-12-08T08:38:23.851733+0000","last_peered":"2023-12-08T08:38:23.851733+0000","last_clean":"2023-12-08T08:38:23.851733+0000","last_became_active":"2023-12-08T08:38:08.275828+0000","last_became_peered":"2023-12-08T08:38:08.275828+0000","last_unstale":"2023-12-08T08:38:23.851733+0000","last_undegraded":"2023-12-08T08:38:23.851733+0000","last_fullsized":"2023-12-08T08:38:23.851733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_clean_scrub_stamp":"2023-12-08T08:38:07.104197+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:35:34.851149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.328098+0000","last_change":"2023-12-08T08:38:23.855031+0000","last_active":"2023-12-08T08:38:26.328098+0000","last_peered":"2023-12-08T08:38:26.328098+0000","last_clean":"2023-12-08T08:38:26.328098+0000","last_became_active":"2023-12-08T08:38:23.854583+0000","last_became_peered":"2023-12-08T08:38:23.854583+0000","last_unstale":"2023-12-08T08:38:26.328098+0000","last_undegraded":"2023-12-08T08:38:26.328098+0000","last_fullsized":"2023-12-08T08:38:26.328098+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:01.683162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852252+0000","last_change":"2023-12-08T08:38:19.633600+0000","last_active":"2023-12-08T08:38:23.852252+0000","last_peered":"2023-12-08T08:38:23.852252+0000","last_clean":"2023-12-08T08:38:23.852252+0000","last_became_active":"2023-12-08T08:38:19.633358+0000","last_became_peered":"2023-12-08T08:38:19.633358+0000","last_unstale":"2023-12-08T08:38:23.852252+0000","last_undegraded":"2023-12-08T08:38:23.852252+0000","last_fullsized":"2023-12-08T08:38:23.852252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:13.356849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662546+0000","last_change":"2023-12-08T08:38:18.631378+0000","last_active":"2023-12-08T08:38:23.662546+0000","last_peered":"2023-12-08T08:38:23.662546+0000","last_clean":"2023-12-08T08:38:23.662546+0000","last_became_active":"2023-12-08T08:38:18.631072+0000","last_became_peered":"2023-12-08T08:38:18.631072+0000","last_unstale":"2023-12-08T08:38:23.662546+0000","last_undegraded":"2023-12-08T08:38:23.662546+0000","last_fullsized":"2023-12-08T08:38:23.662546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:27:03.426918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.327456+0000","last_change":"2023-12-08T08:38:23.853285+0000","last_active":"2023-12-08T08:38:26.327456+0000","last_peered":"2023-12-08T08:38:26.327456+0000","last_clean":"2023-12-08T08:38:26.327456+0000","last_became_active":"2023-12-08T08:38:23.852823+0000","last_became_peered":"2023-12-08T08:38:23.852823+0000","last_unstale":"2023-12-08T08:38:26.327456+0000","last_undegraded":"2023-12-08T08:38:26.327456+0000","last_fullsized":"2023-12-08T08:38:26.327456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:59.405196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661432+0000","last_change":"2023-12-08T08:38:19.632038+0000","last_active":"2023-12-08T08:38:23.661432+0000","last_peered":"2023-12-08T08:38:23.661432+0000","last_clean":"2023-12-08T08:38:23.661432+0000","last_became_active":"2023-12-08T08:38:19.630797+0000","last_became_peered":"2023-12-08T08:38:19.630797+0000","last_unstale":"2023-12-08T08:38:23.661432+0000","last_undegraded":"2023-12-08T08:38:23.661432+0000","last_fullsized":"2023-12-08T08:38:23.661432+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:46:57.645302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662789+0000","last_change":"2023-12-08T08:38:18.633629+0000","last_active":"2023-12-08T08:38:23.662789+0000","last_peered":"2023-12-08T08:38:23.662789+0000","last_clean":"2023-12-08T08:38:23.662789+0000","last_became_active":"2023-12-08T08:38:18.633224+0000","last_became_peered":"2023-12-08T08:38:18.633224+0000","last_unstale":"2023-12-08T08:38:23.662789+0000","last_undegraded":"2023-12-08T08:38:23.662789+0000","last_fullsized":"2023-12-08T08:38:23.662789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:13:39.901023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.327158+0000","last_change":"2023-12-08T08:38:23.680301+0000","last_active":"2023-12-08T08:38:26.327158+0000","last_peered":"2023-12-08T08:38:26.327158+0000","last_clean":"2023-12-08T08:38:26.327158+0000","last_became_active":"2023-12-08T08:38:23.679652+0000","last_became_peered":"2023-12-08T08:38:23.679652+0000","last_unstale":"2023-12-08T08:38:26.327158+0000","last_undegraded":"2023-12-08T08:38:26.327158+0000","last_fullsized":"2023-12-08T08:38:26.327158+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:18:13.891999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.853813+0000","last_change":"2023-12-08T08:38:19.622603+0000","last_active":"2023-12-08T08:38:23.853813+0000","last_peered":"2023-12-08T08:38:23.853813+0000","last_clean":"2023-12-08T08:38:23.853813+0000","last_became_active":"2023-12-08T08:38:19.622325+0000","last_became_peered":"2023-12-08T08:38:19.622325+0000","last_unstale":"2023-12-08T08:38:23.853813+0000","last_undegraded":"2023-12-08T08:38:23.853813+0000","last_fullsized":"2023-12-08T08:38:23.853813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:01:22.871596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659628+0000","last_change":"2023-12-08T08:38:18.635791+0000","last_active":"2023-12-08T08:38:23.659628+0000","last_peered":"2023-12-08T08:38:23.659628+0000","last_clean":"2023-12-08T08:38:23.659628+0000","last_became_active":"2023-12-08T08:38:18.632773+0000","last_became_peered":"2023-12-08T08:38:18.632773+0000","last_unstale":"2023-12-08T08:38:23.659628+0000","last_undegraded":"2023-12-08T08:38:23.659628+0000","last_fullsized":"2023-12-08T08:38:23.659628+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:46:19.770776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.626169+0000","last_change":"2023-12-08T08:38:23.853534+0000","last_active":"2023-12-08T08:38:24.626169+0000","last_peered":"2023-12-08T08:38:24.626169+0000","last_clean":"2023-12-08T08:38:24.626169+0000","last_became_active":"2023-12-08T08:38:23.852476+0000","last_became_peered":"2023-12-08T08:38:23.852476+0000","last_unstale":"2023-12-08T08:38:24.626169+0000","last_undegraded":"2023-12-08T08:38:24.626169+0000","last_fullsized":"2023-12-08T08:38:24.626169+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:01:26.909456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852501+0000","last_change":"2023-12-08T08:38:19.621763+0000","last_active":"2023-12-08T08:38:23.852501+0000","last_peered":"2023-12-08T08:38:23.852501+0000","last_clean":"2023-12-08T08:38:23.852501+0000","last_became_active":"2023-12-08T08:38:19.621504+0000","last_became_peered":"2023-12-08T08:38:19.621504+0000","last_unstale":"2023-12-08T08:38:23.852501+0000","last_undegraded":"2023-12-08T08:38:23.852501+0000","last_fullsized":"2023-12-08T08:38:23.852501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:34:25.887687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658538+0000","last_change":"2023-12-08T08:38:18.632647+0000","last_active":"2023-12-08T08:38:23.658538+0000","last_peered":"2023-12-08T08:38:23.658538+0000","last_clean":"2023-12-08T08:38:23.658538+0000","last_became_active":"2023-12-08T08:38:18.632398+0000","last_became_peered":"2023-12-08T08:38:18.632398+0000","last_unstale":"2023-12-08T08:38:23.658538+0000","last_undegraded":"2023-12-08T08:38:23.658538+0000","last_fullsized":"2023-12-08T08:38:23.658538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:42:30.138259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608618+0000","last_change":"2023-12-08T08:38:23.674111+0000","last_active":"2023-12-08T08:38:24.608618+0000","last_peered":"2023-12-08T08:38:24.608618+0000","last_clean":"2023-12-08T08:38:24.608618+0000","last_became_active":"2023-12-08T08:38:23.673287+0000","last_became_peered":"2023-12-08T08:38:23.673287+0000","last_unstale":"2023-12-08T08:38:24.608618+0000","last_undegraded":"2023-12-08T08:38:24.608618+0000","last_fullsized":"2023-12-08T08:38:24.608618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:22:01.560111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851975+0000","last_change":"2023-12-08T08:38:19.620974+0000","last_active":"2023-12-08T08:38:23.851975+0000","last_peered":"2023-12-08T08:38:23.851975+0000","last_clean":"2023-12-08T08:38:23.851975+0000","last_became_active":"2023-12-08T08:38:19.619478+0000","last_became_peered":"2023-12-08T08:38:19.619478+0000","last_unstale":"2023-12-08T08:38:23.851975+0000","last_undegraded":"2023-12-08T08:38:23.851975+0000","last_fullsized":"2023-12-08T08:38:23.851975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:55.367335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659793+0000","last_change":"2023-12-08T08:38:18.629485+0000","last_active":"2023-12-08T08:38:23.659793+0000","last_peered":"2023-12-08T08:38:23.659793+0000","last_clean":"2023-12-08T08:38:23.659793+0000","last_became_active":"2023-12-08T08:38:18.629182+0000","last_became_peered":"2023-12-08T08:38:18.629182+0000","last_unstale":"2023-12-08T08:38:23.659793+0000","last_undegraded":"2023-12-08T08:38:23.659793+0000","last_fullsized":"2023-12-08T08:38:23.659793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:47:43.142370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.326907+0000","last_change":"2023-12-08T08:38:23.673762+0000","last_active":"2023-12-08T08:38:26.326907+0000","last_peered":"2023-12-08T08:38:26.326907+0000","last_clean":"2023-12-08T08:38:26.326907+0000","last_became_active":"2023-12-08T08:38:23.673236+0000","last_became_peered":"2023-12-08T08:38:23.673236+0000","last_unstale":"2023-12-08T08:38:26.326907+0000","last_undegraded":"2023-12-08T08:38:26.326907+0000","last_fullsized":"2023-12-08T08:38:26.326907+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:34:29.925547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664201+0000","last_change":"2023-12-08T08:38:19.624828+0000","last_active":"2023-12-08T08:38:23.664201+0000","last_peered":"2023-12-08T08:38:23.664201+0000","last_clean":"2023-12-08T08:38:23.664201+0000","last_became_active":"2023-12-08T08:38:19.624249+0000","last_became_peered":"2023-12-08T08:38:19.624249+0000","last_unstale":"2023-12-08T08:38:23.664201+0000","last_undegraded":"2023-12-08T08:38:23.664201+0000","last_fullsized":"2023-12-08T08:38:23.664201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.663711+0000","last_change":"2023-12-08T08:38:18.624832+0000","last_active":"2023-12-08T08:38:23.663711+0000","last_peered":"2023-12-08T08:38:23.663711+0000","last_clean":"2023-12-08T08:38:23.663711+0000","last_became_active":"2023-12-08T08:38:18.624580+0000","last_became_peered":"2023-12-08T08:38:18.624580+0000","last_unstale":"2023-12-08T08:38:23.663711+0000","last_undegraded":"2023-12-08T08:38:23.663711+0000","last_fullsized":"2023-12-08T08:38:23.663711+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:13:39.901023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576884+0000","last_change":"2023-12-08T08:38:23.856627+0000","last_active":"2023-12-08T08:38:25.576884+0000","last_peered":"2023-12-08T08:38:25.576884+0000","last_clean":"2023-12-08T08:38:25.576884+0000","last_became_active":"2023-12-08T08:38:23.855845+0000","last_became_peered":"2023-12-08T08:38:23.855845+0000","last_unstale":"2023-12-08T08:38:25.576884+0000","last_undegraded":"2023-12-08T08:38:25.576884+0000","last_fullsized":"2023-12-08T08:38:25.576884+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:20:54.195474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852053+0000","last_change":"2023-12-08T08:38:19.621686+0000","last_active":"2023-12-08T08:38:23.852053+0000","last_peered":"2023-12-08T08:38:23.852053+0000","last_clean":"2023-12-08T08:38:23.852053+0000","last_became_active":"2023-12-08T08:38:19.621366+0000","last_became_peered":"2023-12-08T08:38:19.621366+0000","last_unstale":"2023-12-08T08:38:23.852053+0000","last_undegraded":"2023-12-08T08:38:23.852053+0000","last_fullsized":"2023-12-08T08:38:23.852053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:18.334775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661960+0000","last_change":"2023-12-08T08:38:18.637940+0000","last_active":"2023-12-08T08:38:23.661960+0000","last_peered":"2023-12-08T08:38:23.661960+0000","last_clean":"2023-12-08T08:38:23.661960+0000","last_became_active":"2023-12-08T08:38:18.637552+0000","last_became_peered":"2023-12-08T08:38:18.637552+0000","last_unstale":"2023-12-08T08:38:23.661960+0000","last_undegraded":"2023-12-08T08:38:23.661960+0000","last_fullsized":"2023-12-08T08:38:23.661960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:14.342540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576721+0000","last_change":"2023-12-08T08:38:23.852937+0000","last_active":"2023-12-08T08:38:25.576721+0000","last_peered":"2023-12-08T08:38:25.576721+0000","last_clean":"2023-12-08T08:38:25.576721+0000","last_became_active":"2023-12-08T08:38:23.852228+0000","last_became_peered":"2023-12-08T08:38:23.852228+0000","last_unstale":"2023-12-08T08:38:25.576721+0000","last_undegraded":"2023-12-08T08:38:25.576721+0000","last_fullsized":"2023-12-08T08:38:25.576721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:46:14.364194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664273+0000","last_change":"2023-12-08T08:38:19.618681+0000","last_active":"2023-12-08T08:38:23.664273+0000","last_peered":"2023-12-08T08:38:23.664273+0000","last_clean":"2023-12-08T08:38:23.664273+0000","last_became_active":"2023-12-08T08:38:19.618102+0000","last_became_peered":"2023-12-08T08:38:19.618102+0000","last_unstale":"2023-12-08T08:38:23.664273+0000","last_undegraded":"2023-12-08T08:38:23.664273+0000","last_fullsized":"2023-12-08T08:38:23.664273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:00:01.146409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661116+0000","last_change":"2023-12-08T08:38:18.636521+0000","last_active":"2023-12-08T08:38:23.661116+0000","last_peered":"2023-12-08T08:38:23.661116+0000","last_clean":"2023-12-08T08:38:23.661116+0000","last_became_active":"2023-12-08T08:38:18.636242+0000","last_became_peered":"2023-12-08T08:38:18.636242+0000","last_unstale":"2023-12-08T08:38:23.661116+0000","last_undegraded":"2023-12-08T08:38:23.661116+0000","last_fullsized":"2023-12-08T08:38:23.661116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.626765+0000","last_change":"2023-12-08T08:38:23.855694+0000","last_active":"2023-12-08T08:38:24.626765+0000","last_peered":"2023-12-08T08:38:24.626765+0000","last_clean":"2023-12-08T08:38:24.626765+0000","last_became_active":"2023-12-08T08:38:23.855342+0000","last_became_peered":"2023-12-08T08:38:23.855342+0000","last_unstale":"2023-12-08T08:38:24.626765+0000","last_undegraded":"2023-12-08T08:38:24.626765+0000","last_fullsized":"2023-12-08T08:38:24.626765+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T20:21:44.003574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852120+0000","last_change":"2023-12-08T08:38:19.627826+0000","last_active":"2023-12-08T08:38:23.852120+0000","last_peered":"2023-12-08T08:38:23.852120+0000","last_clean":"2023-12-08T08:38:23.852120+0000","last_became_active":"2023-12-08T08:38:19.627154+0000","last_became_peered":"2023-12-08T08:38:19.627154+0000","last_unstale":"2023-12-08T08:38:23.852120+0000","last_undegraded":"2023-12-08T08:38:23.852120+0000","last_fullsized":"2023-12-08T08:38:23.852120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:11.797914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:29.035583+0000","last_change":"2023-12-08T08:38:18.630599+0000","last_active":"2023-12-08T08:38:29.035583+0000","last_peered":"2023-12-08T08:38:29.035583+0000","last_clean":"2023-12-08T08:38:29.035583+0000","last_became_active":"2023-12-08T08:38:18.630349+0000","last_became_peered":"2023-12-08T08:38:18.630349+0000","last_unstale":"2023-12-08T08:38:29.035583+0000","last_undegraded":"2023-12-08T08:38:29.035583+0000","last_fullsized":"2023-12-08T08:38:29.035583+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:36:34.339195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3839,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.827898+0000","last_change":"2023-12-08T08:38:23.676381+0000","last_active":"2023-12-08T08:38:25.827898+0000","last_peered":"2023-12-08T08:38:25.827898+0000","last_clean":"2023-12-08T08:38:25.827898+0000","last_became_active":"2023-12-08T08:38:23.675561+0000","last_became_peered":"2023-12-08T08:38:23.675561+0000","last_unstale":"2023-12-08T08:38:25.827898+0000","last_undegraded":"2023-12-08T08:38:25.827898+0000","last_fullsized":"2023-12-08T08:38:25.827898+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:22.372635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659110+0000","last_change":"2023-12-08T08:38:19.615536+0000","last_active":"2023-12-08T08:38:23.659110+0000","last_peered":"2023-12-08T08:38:23.659110+0000","last_clean":"2023-12-08T08:38:23.659110+0000","last_became_active":"2023-12-08T08:38:19.615243+0000","last_became_peered":"2023-12-08T08:38:19.615243+0000","last_unstale":"2023-12-08T08:38:23.659110+0000","last_undegraded":"2023-12-08T08:38:23.659110+0000","last_fullsized":"2023-12-08T08:38:23.659110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:13:40.905659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662690+0000","last_change":"2023-12-08T08:38:18.632950+0000","last_active":"2023-12-08T08:38:23.662690+0000","last_peered":"2023-12-08T08:38:23.662690+0000","last_clean":"2023-12-08T08:38:23.662690+0000","last_became_active":"2023-12-08T08:38:18.632652+0000","last_became_peered":"2023-12-08T08:38:18.632652+0000","last_unstale":"2023-12-08T08:38:23.662690+0000","last_undegraded":"2023-12-08T08:38:23.662690+0000","last_fullsized":"2023-12-08T08:38:23.662690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:36:34.339195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.609288+0000","last_change":"2023-12-08T08:38:23.674379+0000","last_active":"2023-12-08T08:38:24.609288+0000","last_peered":"2023-12-08T08:38:24.609288+0000","last_clean":"2023-12-08T08:38:24.609288+0000","last_became_active":"2023-12-08T08:38:23.673581+0000","last_became_peered":"2023-12-08T08:38:23.673581+0000","last_unstale":"2023-12-08T08:38:24.609288+0000","last_undegraded":"2023-12-08T08:38:24.609288+0000","last_fullsized":"2023-12-08T08:38:24.609288+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:47:01.683162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659853+0000","last_change":"2023-12-08T08:38:19.633344+0000","last_active":"2023-12-08T08:38:23.659853+0000","last_peered":"2023-12-08T08:38:23.659853+0000","last_clean":"2023-12-08T08:38:23.659853+0000","last_became_active":"2023-12-08T08:38:19.633035+0000","last_became_peered":"2023-12-08T08:38:19.633035+0000","last_unstale":"2023-12-08T08:38:23.659853+0000","last_undegraded":"2023-12-08T08:38:23.659853+0000","last_fullsized":"2023-12-08T08:38:23.659853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661808+0000","last_change":"2023-12-08T08:38:18.631126+0000","last_active":"2023-12-08T08:38:23.661808+0000","last_peered":"2023-12-08T08:38:23.661808+0000","last_clean":"2023-12-08T08:38:23.661808+0000","last_became_active":"2023-12-08T08:38:18.630841+0000","last_became_peered":"2023-12-08T08:38:18.630841+0000","last_unstale":"2023-12-08T08:38:23.661808+0000","last_undegraded":"2023-12-08T08:38:23.661808+0000","last_fullsized":"2023-12-08T08:38:23.661808+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:07.497079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576255+0000","last_change":"2023-12-08T08:38:23.678178+0000","last_active":"2023-12-08T08:38:25.576255+0000","last_peered":"2023-12-08T08:38:25.576255+0000","last_clean":"2023-12-08T08:38:25.576255+0000","last_became_active":"2023-12-08T08:38:23.677800+0000","last_became_peered":"2023-12-08T08:38:23.677800+0000","last_unstale":"2023-12-08T08:38:25.576255+0000","last_undegraded":"2023-12-08T08:38:25.576255+0000","last_fullsized":"2023-12-08T08:38:25.576255+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:22:37.463722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661701+0000","last_change":"2023-12-08T08:38:19.631623+0000","last_active":"2023-12-08T08:38:23.661701+0000","last_peered":"2023-12-08T08:38:23.661701+0000","last_clean":"2023-12-08T08:38:23.661701+0000","last_became_active":"2023-12-08T08:38:19.630737+0000","last_became_peered":"2023-12-08T08:38:19.630737+0000","last_unstale":"2023-12-08T08:38:23.661701+0000","last_undegraded":"2023-12-08T08:38:23.661701+0000","last_fullsized":"2023-12-08T08:38:23.661701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:12:18.334775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665261+0000","last_change":"2023-12-08T08:38:18.630876+0000","last_active":"2023-12-08T08:38:23.665261+0000","last_peered":"2023-12-08T08:38:23.665261+0000","last_clean":"2023-12-08T08:38:23.665261+0000","last_became_active":"2023-12-08T08:38:18.630581+0000","last_became_peered":"2023-12-08T08:38:18.630581+0000","last_unstale":"2023-12-08T08:38:23.665261+0000","last_undegraded":"2023-12-08T08:38:23.665261+0000","last_fullsized":"2023-12-08T08:38:23.665261+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:10.075992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576234+0000","last_change":"2023-12-08T08:38:23.855920+0000","last_active":"2023-12-08T08:38:25.576234+0000","last_peered":"2023-12-08T08:38:25.576234+0000","last_clean":"2023-12-08T08:38:25.576234+0000","last_became_active":"2023-12-08T08:38:23.855491+0000","last_became_peered":"2023-12-08T08:38:23.855491+0000","last_unstale":"2023-12-08T08:38:25.576234+0000","last_undegraded":"2023-12-08T08:38:25.576234+0000","last_fullsized":"2023-12-08T08:38:25.576234+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:10:52.070614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.326821+0000","last_change":"2023-12-08T08:38:23.855660+0000","last_active":"2023-12-08T08:38:26.326821+0000","last_peered":"2023-12-08T08:38:26.326821+0000","last_clean":"2023-12-08T08:38:26.326821+0000","last_became_active":"2023-12-08T08:38:23.855190+0000","last_became_peered":"2023-12-08T08:38:23.855190+0000","last_unstale":"2023-12-08T08:38:26.326821+0000","last_undegraded":"2023-12-08T08:38:26.326821+0000","last_fullsized":"2023-12-08T08:38:26.326821+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:48.357070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664507+0000","last_change":"2023-12-08T08:38:19.620116+0000","last_active":"2023-12-08T08:38:23.664507+0000","last_peered":"2023-12-08T08:38:23.664507+0000","last_clean":"2023-12-08T08:38:23.664507+0000","last_became_active":"2023-12-08T08:38:19.619412+0000","last_became_peered":"2023-12-08T08:38:19.619412+0000","last_unstale":"2023-12-08T08:38:23.664507+0000","last_undegraded":"2023-12-08T08:38:23.664507+0000","last_fullsized":"2023-12-08T08:38:23.664507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661919+0000","last_change":"2023-12-08T08:38:18.630228+0000","last_active":"2023-12-08T08:38:23.661919+0000","last_peered":"2023-12-08T08:38:23.661919+0000","last_clean":"2023-12-08T08:38:23.661919+0000","last_became_active":"2023-12-08T08:38:18.629445+0000","last_became_peered":"2023-12-08T08:38:18.629445+0000","last_unstale":"2023-12-08T08:38:23.661919+0000","last_undegraded":"2023-12-08T08:38:23.661919+0000","last_fullsized":"2023-12-08T08:38:23.661919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:57.526979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:26.328319+0000","last_change":"2023-12-08T08:38:23.678105+0000","last_active":"2023-12-08T08:38:26.328319+0000","last_peered":"2023-12-08T08:38:26.328319+0000","last_clean":"2023-12-08T08:38:26.328319+0000","last_became_active":"2023-12-08T08:38:23.677716+0000","last_became_peered":"2023-12-08T08:38:23.677716+0000","last_unstale":"2023-12-08T08:38:26.328319+0000","last_undegraded":"2023-12-08T08:38:26.328319+0000","last_fullsized":"2023-12-08T08:38:26.328319+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:04:15.835775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655896+0000","last_change":"2023-12-08T08:38:19.628959+0000","last_active":"2023-12-08T08:38:23.655896+0000","last_peered":"2023-12-08T08:38:23.655896+0000","last_clean":"2023-12-08T08:38:23.655896+0000","last_became_active":"2023-12-08T08:38:19.628662+0000","last_became_peered":"2023-12-08T08:38:19.628662+0000","last_unstale":"2023-12-08T08:38:23.655896+0000","last_undegraded":"2023-12-08T08:38:23.655896+0000","last_fullsized":"2023-12-08T08:38:23.655896+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658005+0000","last_change":"2023-12-08T08:38:18.637772+0000","last_active":"2023-12-08T08:38:23.658005+0000","last_peered":"2023-12-08T08:38:23.658005+0000","last_clean":"2023-12-08T08:38:23.658005+0000","last_became_active":"2023-12-08T08:38:18.637318+0000","last_became_peered":"2023-12-08T08:38:18.637318+0000","last_unstale":"2023-12-08T08:38:23.658005+0000","last_undegraded":"2023-12-08T08:38:23.658005+0000","last_fullsized":"2023-12-08T08:38:23.658005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:11.067589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664203+0000","last_change":"2023-12-08T08:38:18.630848+0000","last_active":"2023-12-08T08:38:23.664203+0000","last_peered":"2023-12-08T08:38:23.664203+0000","last_clean":"2023-12-08T08:38:23.664203+0000","last_became_active":"2023-12-08T08:38:18.629736+0000","last_became_peered":"2023-12-08T08:38:18.629736+0000","last_unstale":"2023-12-08T08:38:23.664203+0000","last_undegraded":"2023-12-08T08:38:23.664203+0000","last_fullsized":"2023-12-08T08:38:23.664203+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:10.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661026+0000","last_change":"2023-12-08T08:38:19.612508+0000","last_active":"2023-12-08T08:38:23.661026+0000","last_peered":"2023-12-08T08:38:23.661026+0000","last_clean":"2023-12-08T08:38:23.661026+0000","last_became_active":"2023-12-08T08:38:19.612212+0000","last_became_peered":"2023-12-08T08:38:19.612212+0000","last_unstale":"2023-12-08T08:38:23.661026+0000","last_undegraded":"2023-12-08T08:38:23.661026+0000","last_fullsized":"2023-12-08T08:38:23.661026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:24.608202+0000","last_change":"2023-12-08T08:38:23.678309+0000","last_active":"2023-12-08T08:38:24.608202+0000","last_peered":"2023-12-08T08:38:24.608202+0000","last_clean":"2023-12-08T08:38:24.608202+0000","last_became_active":"2023-12-08T08:38:23.677882+0000","last_became_peered":"2023-12-08T08:38:23.677882+0000","last_unstale":"2023-12-08T08:38:24.608202+0000","last_undegraded":"2023-12-08T08:38:24.608202+0000","last_fullsized":"2023-12-08T08:38:24.608202+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:20:15.118488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655793+0000","last_change":"2023-12-08T08:38:18.638936+0000","last_active":"2023-12-08T08:38:23.655793+0000","last_peered":"2023-12-08T08:38:23.655793+0000","last_clean":"2023-12-08T08:38:23.655793+0000","last_became_active":"2023-12-08T08:38:18.638058+0000","last_became_peered":"2023-12-08T08:38:18.638058+0000","last_unstale":"2023-12-08T08:38:23.655793+0000","last_undegraded":"2023-12-08T08:38:23.655793+0000","last_fullsized":"2023-12-08T08:38:23.655793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:17.607422+0000","last_clean_scrub_stamp":"2023-12-08T08:38:17.607422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:50:10.607717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.658137+0000","last_change":"2023-12-08T08:38:19.619468+0000","last_active":"2023-12-08T08:38:23.658137+0000","last_peered":"2023-12-08T08:38:23.658137+0000","last_clean":"2023-12-08T08:38:23.658137+0000","last_became_active":"2023-12-08T08:38:19.619177+0000","last_became_peered":"2023-12-08T08:38:19.619177+0000","last_unstale":"2023-12-08T08:38:23.658137+0000","last_undegraded":"2023-12-08T08:38:23.658137+0000","last_fullsized":"2023-12-08T08:38:23.658137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:46:20.775412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-12-08T08:38:25.576589+0000","last_change":"2023-12-08T08:38:23.679648+0000","last_active":"2023-12-08T08:38:25.576589+0000","last_peered":"2023-12-08T08:38:25.576589+0000","last_clean":"2023-12-08T08:38:25.576589+0000","last_became_active":"2023-12-08T08:38:23.679229+0000","last_became_peered":"2023-12-08T08:38:23.679229+0000","last_unstale":"2023-12-08T08:38:25.576589+0000","last_undegraded":"2023-12-08T08:38:25.576589+0000","last_fullsized":"2023-12-08T08:38:25.576589+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:22.649918+0000","last_clean_scrub_stamp":"2023-12-08T08:38:22.649918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:53:05.969247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.663580+0000","last_change":"2023-12-08T08:38:19.624993+0000","last_active":"2023-12-08T08:38:23.663580+0000","last_peered":"2023-12-08T08:38:23.663580+0000","last_clean":"2023-12-08T08:38:23.663580+0000","last_became_active":"2023-12-08T08:38:19.624569+0000","last_became_peered":"2023-12-08T08:38:19.624569+0000","last_unstale":"2023-12-08T08:38:23.663580+0000","last_undegraded":"2023-12-08T08:38:23.663580+0000","last_fullsized":"2023-12-08T08:38:23.663580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662648+0000","last_change":"2023-12-08T08:38:19.631559+0000","last_active":"2023-12-08T08:38:23.662648+0000","last_peered":"2023-12-08T08:38:23.662648+0000","last_clean":"2023-12-08T08:38:23.662648+0000","last_became_active":"2023-12-08T08:38:19.631225+0000","last_became_peered":"2023-12-08T08:38:19.631225+0000","last_unstale":"2023-12-08T08:38:23.662648+0000","last_undegraded":"2023-12-08T08:38:23.662648+0000","last_fullsized":"2023-12-08T08:38:23.662648+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:12.072225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661866+0000","last_change":"2023-12-08T08:38:19.631476+0000","last_active":"2023-12-08T08:38:23.661866+0000","last_peered":"2023-12-08T08:38:23.661866+0000","last_clean":"2023-12-08T08:38:23.661866+0000","last_became_active":"2023-12-08T08:38:19.630628+0000","last_became_peered":"2023-12-08T08:38:19.630628+0000","last_unstale":"2023-12-08T08:38:23.661866+0000","last_undegraded":"2023-12-08T08:38:23.661866+0000","last_fullsized":"2023-12-08T08:38:23.661866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:55.367335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.660943+0000","last_change":"2023-12-08T08:38:19.623252+0000","last_active":"2023-12-08T08:38:23.660943+0000","last_peered":"2023-12-08T08:38:23.660943+0000","last_clean":"2023-12-08T08:38:23.660943+0000","last_became_active":"2023-12-08T08:38:19.622970+0000","last_became_peered":"2023-12-08T08:38:19.622970+0000","last_unstale":"2023-12-08T08:38:23.660943+0000","last_undegraded":"2023-12-08T08:38:23.660943+0000","last_fullsized":"2023-12-08T08:38:23.660943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.657026+0000","last_change":"2023-12-08T08:38:19.621907+0000","last_active":"2023-12-08T08:38:23.657026+0000","last_peered":"2023-12-08T08:38:23.657026+0000","last_clean":"2023-12-08T08:38:23.657026+0000","last_became_active":"2023-12-08T08:38:19.621597+0000","last_became_peered":"2023-12-08T08:38:19.621597+0000","last_unstale":"2023-12-08T08:38:23.657026+0000","last_undegraded":"2023-12-08T08:38:23.657026+0000","last_fullsized":"2023-12-08T08:38:23.657026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:30:55.367335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662589+0000","last_change":"2023-12-08T08:38:19.621460+0000","last_active":"2023-12-08T08:38:23.662589+0000","last_peered":"2023-12-08T08:38:23.662589+0000","last_clean":"2023-12-08T08:38:23.662589+0000","last_became_active":"2023-12-08T08:38:19.621090+0000","last_became_peered":"2023-12-08T08:38:19.621090+0000","last_unstale":"2023-12-08T08:38:23.662589+0000","last_undegraded":"2023-12-08T08:38:23.662589+0000","last_fullsized":"2023-12-08T08:38:23.662589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665456+0000","last_change":"2023-12-08T08:38:19.620846+0000","last_active":"2023-12-08T08:38:23.665456+0000","last_peered":"2023-12-08T08:38:23.665456+0000","last_clean":"2023-12-08T08:38:23.665456+0000","last_became_active":"2023-12-08T08:38:19.620575+0000","last_became_peered":"2023-12-08T08:38:19.620575+0000","last_unstale":"2023-12-08T08:38:23.665456+0000","last_undegraded":"2023-12-08T08:38:23.665456+0000","last_fullsized":"2023-12-08T08:38:23.665456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.656076+0000","last_change":"2023-12-08T08:38:19.623987+0000","last_active":"2023-12-08T08:38:23.656076+0000","last_peered":"2023-12-08T08:38:23.656076+0000","last_clean":"2023-12-08T08:38:23.656076+0000","last_became_active":"2023-12-08T08:38:19.623637+0000","last_became_peered":"2023-12-08T08:38:19.623637+0000","last_unstale":"2023-12-08T08:38:23.656076+0000","last_undegraded":"2023-12-08T08:38:23.656076+0000","last_fullsized":"2023-12-08T08:38:23.656076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:43.336635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661985+0000","last_change":"2023-12-08T08:38:19.627606+0000","last_active":"2023-12-08T08:38:23.661985+0000","last_peered":"2023-12-08T08:38:23.661985+0000","last_clean":"2023-12-08T08:38:23.661985+0000","last_became_active":"2023-12-08T08:38:19.627345+0000","last_became_peered":"2023-12-08T08:38:19.627345+0000","last_unstale":"2023-12-08T08:38:23.661985+0000","last_undegraded":"2023-12-08T08:38:23.661985+0000","last_fullsized":"2023-12-08T08:38:23.661985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:00:43.336635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.657865+0000","last_change":"2023-12-08T08:38:19.623195+0000","last_active":"2023-12-08T08:38:23.657865+0000","last_peered":"2023-12-08T08:38:23.657865+0000","last_clean":"2023-12-08T08:38:23.657865+0000","last_became_active":"2023-12-08T08:38:19.622896+0000","last_became_peered":"2023-12-08T08:38:19.622896+0000","last_unstale":"2023-12-08T08:38:23.657865+0000","last_undegraded":"2023-12-08T08:38:23.657865+0000","last_fullsized":"2023-12-08T08:38:23.657865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:59:11.073543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661771+0000","last_change":"2023-12-08T08:38:19.632017+0000","last_active":"2023-12-08T08:38:23.661771+0000","last_peered":"2023-12-08T08:38:23.661771+0000","last_clean":"2023-12-08T08:38:23.661771+0000","last_became_active":"2023-12-08T08:38:19.631771+0000","last_became_peered":"2023-12-08T08:38:19.631771+0000","last_unstale":"2023-12-08T08:38:23.661771+0000","last_undegraded":"2023-12-08T08:38:23.661771+0000","last_fullsized":"2023-12-08T08:38:23.661771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:59:11.073543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.660849+0000","last_change":"2023-12-08T08:38:19.618872+0000","last_active":"2023-12-08T08:38:23.660849+0000","last_peered":"2023-12-08T08:38:23.660849+0000","last_clean":"2023-12-08T08:38:23.660849+0000","last_became_active":"2023-12-08T08:38:19.617964+0000","last_became_peered":"2023-12-08T08:38:19.617964+0000","last_unstale":"2023-12-08T08:38:23.660849+0000","last_undegraded":"2023-12-08T08:38:23.660849+0000","last_fullsized":"2023-12-08T08:38:23.660849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665351+0000","last_change":"2023-12-08T08:38:19.622440+0000","last_active":"2023-12-08T08:38:23.665351+0000","last_peered":"2023-12-08T08:38:23.665351+0000","last_clean":"2023-12-08T08:38:23.665351+0000","last_became_active":"2023-12-08T08:38:19.621997+0000","last_became_peered":"2023-12-08T08:38:19.621997+0000","last_unstale":"2023-12-08T08:38:23.665351+0000","last_undegraded":"2023-12-08T08:38:23.665351+0000","last_fullsized":"2023-12-08T08:38:23.665351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852435+0000","last_change":"2023-12-08T08:38:19.621395+0000","last_active":"2023-12-08T08:38:23.852435+0000","last_peered":"2023-12-08T08:38:23.852435+0000","last_clean":"2023-12-08T08:38:23.852435+0000","last_became_active":"2023-12-08T08:38:19.621148+0000","last_became_peered":"2023-12-08T08:38:19.621148+0000","last_unstale":"2023-12-08T08:38:23.852435+0000","last_undegraded":"2023-12-08T08:38:23.852435+0000","last_fullsized":"2023-12-08T08:38:23.852435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:44.319210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659499+0000","last_change":"2023-12-08T08:38:19.626108+0000","last_active":"2023-12-08T08:38:23.659499+0000","last_peered":"2023-12-08T08:38:23.659499+0000","last_clean":"2023-12-08T08:38:23.659499+0000","last_became_active":"2023-12-08T08:38:19.625773+0000","last_became_peered":"2023-12-08T08:38:19.625773+0000","last_unstale":"2023-12-08T08:38:23.659499+0000","last_undegraded":"2023-12-08T08:38:23.659499+0000","last_fullsized":"2023-12-08T08:38:23.659499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:05:08.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.662077+0000","last_change":"2023-12-08T08:38:19.631410+0000","last_active":"2023-12-08T08:38:23.662077+0000","last_peered":"2023-12-08T08:38:23.662077+0000","last_clean":"2023-12-08T08:38:23.662077+0000","last_became_active":"2023-12-08T08:38:19.630653+0000","last_became_peered":"2023-12-08T08:38:19.630653+0000","last_unstale":"2023-12-08T08:38:23.662077+0000","last_undegraded":"2023-12-08T08:38:23.662077+0000","last_fullsized":"2023-12-08T08:38:23.662077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:44.319210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661818+0000","last_change":"2023-12-08T08:38:19.630190+0000","last_active":"2023-12-08T08:38:23.661818+0000","last_peered":"2023-12-08T08:38:23.661818+0000","last_clean":"2023-12-08T08:38:23.661818+0000","last_became_active":"2023-12-08T08:38:19.629923+0000","last_became_peered":"2023-12-08T08:38:19.629923+0000","last_unstale":"2023-12-08T08:38:23.661818+0000","last_undegraded":"2023-12-08T08:38:23.661818+0000","last_fullsized":"2023-12-08T08:38:23.661818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664602+0000","last_change":"2023-12-08T08:38:19.620155+0000","last_active":"2023-12-08T08:38:23.664602+0000","last_peered":"2023-12-08T08:38:23.664602+0000","last_clean":"2023-12-08T08:38:23.664602+0000","last_became_active":"2023-12-08T08:38:19.619814+0000","last_became_peered":"2023-12-08T08:38:19.619814+0000","last_unstale":"2023-12-08T08:38:23.664602+0000","last_undegraded":"2023-12-08T08:38:23.664602+0000","last_fullsized":"2023-12-08T08:38:23.664602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:24:58.531616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664183+0000","last_change":"2023-12-08T08:38:19.616693+0000","last_active":"2023-12-08T08:38:23.664183+0000","last_peered":"2023-12-08T08:38:23.664183+0000","last_clean":"2023-12-08T08:38:23.664183+0000","last_became_active":"2023-12-08T08:38:19.616369+0000","last_became_peered":"2023-12-08T08:38:19.616369+0000","last_unstale":"2023-12-08T08:38:23.664183+0000","last_undegraded":"2023-12-08T08:38:23.664183+0000","last_fullsized":"2023-12-08T08:38:23.664183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T08:39:38.322348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655532+0000","last_change":"2023-12-08T08:38:19.625929+0000","last_active":"2023-12-08T08:38:23.655532+0000","last_peered":"2023-12-08T08:38:23.655532+0000","last_clean":"2023-12-08T08:38:23.655532+0000","last_became_active":"2023-12-08T08:38:19.624245+0000","last_became_peered":"2023-12-08T08:38:19.624245+0000","last_unstale":"2023-12-08T08:38:23.655532+0000","last_undegraded":"2023-12-08T08:38:23.655532+0000","last_fullsized":"2023-12-08T08:38:23.655532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.659198+0000","last_change":"2023-12-08T08:38:19.625517+0000","last_active":"2023-12-08T08:38:23.659198+0000","last_peered":"2023-12-08T08:38:23.659198+0000","last_clean":"2023-12-08T08:38:23.659198+0000","last_became_active":"2023-12-08T08:38:19.625079+0000","last_became_peered":"2023-12-08T08:38:19.625079+0000","last_unstale":"2023-12-08T08:38:23.659198+0000","last_undegraded":"2023-12-08T08:38:23.659198+0000","last_fullsized":"2023-12-08T08:38:23.659198+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:39:12.072225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665048+0000","last_change":"2023-12-08T08:38:19.632480+0000","last_active":"2023-12-08T08:38:23.665048+0000","last_peered":"2023-12-08T08:38:23.665048+0000","last_clean":"2023-12-08T08:38:23.665048+0000","last_became_active":"2023-12-08T08:38:19.632215+0000","last_became_peered":"2023-12-08T08:38:19.632215+0000","last_unstale":"2023-12-08T08:38:23.665048+0000","last_undegraded":"2023-12-08T08:38:23.665048+0000","last_fullsized":"2023-12-08T08:38:23.665048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:55:15.347176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.665415+0000","last_change":"2023-12-08T08:38:19.635234+0000","last_active":"2023-12-08T08:38:23.665415+0000","last_peered":"2023-12-08T08:38:23.665415+0000","last_clean":"2023-12-08T08:38:23.665415+0000","last_became_active":"2023-12-08T08:38:19.634638+0000","last_became_peered":"2023-12-08T08:38:19.634638+0000","last_unstale":"2023-12-08T08:38:23.665415+0000","last_undegraded":"2023-12-08T08:38:23.665415+0000","last_fullsized":"2023-12-08T08:38:23.665415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:16:00.196691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.853718+0000","last_change":"2023-12-08T08:38:19.633557+0000","last_active":"2023-12-08T08:38:23.853718+0000","last_peered":"2023-12-08T08:38:23.853718+0000","last_clean":"2023-12-08T08:38:23.853718+0000","last_became_active":"2023-12-08T08:38:19.633168+0000","last_became_peered":"2023-12-08T08:38:19.633168+0000","last_unstale":"2023-12-08T08:38:23.853718+0000","last_undegraded":"2023-12-08T08:38:23.853718+0000","last_fullsized":"2023-12-08T08:38:23.853718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:18:09.854138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.655258+0000","last_change":"2023-12-08T08:38:19.635104+0000","last_active":"2023-12-08T08:38:23.655258+0000","last_peered":"2023-12-08T08:38:23.655258+0000","last_clean":"2023-12-08T08:38:23.655258+0000","last_became_active":"2023-12-08T08:38:19.634711+0000","last_became_peered":"2023-12-08T08:38:19.634711+0000","last_unstale":"2023-12-08T08:38:23.655258+0000","last_undegraded":"2023-12-08T08:38:23.655258+0000","last_fullsized":"2023-12-08T08:38:23.655258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:00:14.812487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851929+0000","last_change":"2023-12-08T08:38:19.618128+0000","last_active":"2023-12-08T08:38:23.851929+0000","last_peered":"2023-12-08T08:38:23.851929+0000","last_clean":"2023-12-08T08:38:23.851929+0000","last_became_active":"2023-12-08T08:38:19.617870+0000","last_became_peered":"2023-12-08T08:38:19.617870+0000","last_unstale":"2023-12-08T08:38:23.851929+0000","last_undegraded":"2023-12-08T08:38:23.851929+0000","last_fullsized":"2023-12-08T08:38:23.851929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:59:32.116348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661665+0000","last_change":"2023-12-08T08:38:19.631953+0000","last_active":"2023-12-08T08:38:23.661665+0000","last_peered":"2023-12-08T08:38:23.661665+0000","last_clean":"2023-12-08T08:38:23.661665+0000","last_became_active":"2023-12-08T08:38:19.631638+0000","last_became_peered":"2023-12-08T08:38:19.631638+0000","last_unstale":"2023-12-08T08:38:23.661665+0000","last_undegraded":"2023-12-08T08:38:23.661665+0000","last_fullsized":"2023-12-08T08:38:23.661665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.851995+0000","last_change":"2023-12-08T08:38:19.621369+0000","last_active":"2023-12-08T08:38:23.851995+0000","last_peered":"2023-12-08T08:38:23.851995+0000","last_clean":"2023-12-08T08:38:23.851995+0000","last_became_active":"2023-12-08T08:38:19.621103+0000","last_became_peered":"2023-12-08T08:38:19.621103+0000","last_unstale":"2023-12-08T08:38:23.851995+0000","last_undegraded":"2023-12-08T08:38:23.851995+0000","last_fullsized":"2023-12-08T08:38:23.851995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:46:57.645302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.661240+0000","last_change":"2023-12-08T08:38:19.630541+0000","last_active":"2023-12-08T08:38:23.661240+0000","last_peered":"2023-12-08T08:38:23.661240+0000","last_clean":"2023-12-08T08:38:23.661240+0000","last_became_active":"2023-12-08T08:38:19.630175+0000","last_became_peered":"2023-12-08T08:38:19.630175+0000","last_unstale":"2023-12-08T08:38:23.661240+0000","last_undegraded":"2023-12-08T08:38:23.661240+0000","last_fullsized":"2023-12-08T08:38:23.661240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:49:29.898351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664269+0000","last_change":"2023-12-08T08:38:19.626884+0000","last_active":"2023-12-08T08:38:23.664269+0000","last_peered":"2023-12-08T08:38:23.664269+0000","last_clean":"2023-12-08T08:38:23.664269+0000","last_became_active":"2023-12-08T08:38:19.626525+0000","last_became_peered":"2023-12-08T08:38:19.626525+0000","last_unstale":"2023-12-08T08:38:23.664269+0000","last_undegraded":"2023-12-08T08:38:23.664269+0000","last_fullsized":"2023-12-08T08:38:23.664269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:35:08.501715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.664251+0000","last_change":"2023-12-08T08:38:19.621125+0000","last_active":"2023-12-08T08:38:23.664251+0000","last_peered":"2023-12-08T08:38:23.664251+0000","last_clean":"2023-12-08T08:38:23.664251+0000","last_became_active":"2023-12-08T08:38:19.620831+0000","last_became_peered":"2023-12-08T08:38:19.620831+0000","last_unstale":"2023-12-08T08:38:23.664251+0000","last_undegraded":"2023-12-08T08:38:23.664251+0000","last_fullsized":"2023-12-08T08:38:23.664251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:21:57.522251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-12-08T08:38:23.852760+0000","last_change":"2023-12-08T08:38:19.624484+0000","last_active":"2023-12-08T08:38:23.852760+0000","last_peered":"2023-12-08T08:38:23.852760+0000","last_clean":"2023-12-08T08:38:23.852760+0000","last_became_active":"2023-12-08T08:38:19.623921+0000","last_became_peered":"2023-12-08T08:38:19.623921+0000","last_unstale":"2023-12-08T08:38:23.852760+0000","last_undegraded":"2023-12-08T08:38:23.852760+0000","last_fullsized":"2023-12-08T08:38:23.852760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:18.612058+0000","last_clean_scrub_stamp":"2023-12-08T08:38:18.612058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:32:54.146197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4637,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":851968,"data_stored":690742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":99882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":99918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":266240,"data_stored":103631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":253952,"data_stored":99792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":99952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6779999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.498}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4730000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":690798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4159999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4850000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5920000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-08T08:38:47.499 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-08T08:38:47.499 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-08T08:38:47.499 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-12-08T08:38:47.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-12-08T08:38:47.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.638+0000 7fd553978640 1 Processor -- start 2023-12-08T08:38:47.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.638+0000 7fd553978640 1 -- start start 2023-12-08T08:38:47.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd553978640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd54c15e7f0 0x7fd54c0d3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd553978640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd54c0cff60 0x7fd54c0b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd553978640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd54c0b2eb0 0x7fd54c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd553978640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54c0cdd80 con 0x7fd54c15e7f0 2023-12-08T08:38:47.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd553978640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54c161ed0 con 0x7fd54c0b2eb0 2023-12-08T08:38:47.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd553978640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c0d04d0 con 0x7fd54c0cff60 2023-12-08T08:38:47.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd551714640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd54c15e7f0 0x7fd54c0d3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd551714640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd54c15e7f0 0x7fd54c0d3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53488/0 (socket says 172.21.15.16:53488) 2023-12-08T08:38:47.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd551714640 1 -- 172.21.15.16:0/3101094868 learned_addr learned my addr 172.21.15.16:0/3101094868 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:47.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd551f15640 1 --2- 172.21.15.16:0/3101094868 >> v2:172.21.15.112:3300/0 conn(0x7fd54c0b2eb0 0x7fd54c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd550f13640 1 --2- 172.21.15.16:0/3101094868 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd54c0cff60 0x7fd54c0b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd551714640 1 -- 172.21.15.16:0/3101094868 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd54c0cff60 msgr2=0x7fd54c0b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:47.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd551714640 1 --2- 172.21.15.16:0/3101094868 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd54c0cff60 0x7fd54c0b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd551714640 1 -- 172.21.15.16:0/3101094868 >> v2:172.21.15.112:3300/0 conn(0x7fd54c0b2eb0 msgr2=0x7fd54c0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:47.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd551714640 1 --2- 172.21.15.16:0/3101094868 >> v2:172.21.15.112:3300/0 conn(0x7fd54c0b2eb0 0x7fd54c0ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.639+0000 7fd551714640 1 -- 172.21.15.16:0/3101094868 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54c0ce020 con 0x7fd54c15e7f0 2023-12-08T08:38:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.640+0000 7fd551714640 1 --2- 172.21.15.16:0/3101094868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd54c15e7f0 0x7fd54c0d3b30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0ca060 tx=0x7fd54802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3389585d2debda0b server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.640+0000 7fd543fff640 1 -- 172.21.15.16:0/3101094868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548031870 con 0x7fd54c15e7f0 2023-12-08T08:38:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.640+0000 7fd543fff640 1 -- 172.21.15.16:0/3101094868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5480319d0 con 0x7fd54c15e7f0 2023-12-08T08:38:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.640+0000 7fd543fff640 1 -- 172.21.15.16:0/3101094868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd548031d60 con 0x7fd54c15e7f0 2023-12-08T08:38:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.640+0000 7fd553978640 1 -- 172.21.15.16:0/3101094868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd54c15e7f0 msgr2=0x7fd54c0d3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.640+0000 7fd553978640 1 --2- 172.21.15.16:0/3101094868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd54c15e7f0 0x7fd54c0d3b30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd54c0ca060 tx=0x7fd54802b910 comp rx=0 tx=0).stop 2023-12-08T08:38:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.641+0000 7fd553978640 1 -- 172.21.15.16:0/3101094868 shutdown_connections 2023-12-08T08:38:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.641+0000 7fd553978640 1 --2- 172.21.15.16:0/3101094868 >> v2:172.21.15.112:3300/0 conn(0x7fd54c0b2eb0 0x7fd54c0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.641+0000 7fd553978640 1 --2- 172.21.15.16:0/3101094868 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd54c0cff60 0x7fd54c0b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.641+0000 7fd553978640 1 --2- 172.21.15.16:0/3101094868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd54c15e7f0 0x7fd54c0d3b30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.641+0000 7fd553978640 1 -- 172.21.15.16:0/3101094868 >> 172.21.15.16:0/3101094868 conn(0x7fd54c0d3010 msgr2=0x7fd54c0d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.641+0000 7fd553978640 1 -- 172.21.15.16:0/3101094868 shutdown_connections 2023-12-08T08:38:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.641+0000 7fd553978640 1 -- 172.21.15.16:0/3101094868 wait complete. 2023-12-08T08:38:47.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.642+0000 7fd553978640 1 Processor -- start 2023-12-08T08:38:47.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.642+0000 7fd553978640 1 -- start start 2023-12-08T08:38:47.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd553978640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd54c0b2eb0 0x7fd54c15c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd553978640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd54c0cff60 0x7fd54c0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd553978640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd54c15e7f0 0x7fd54c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd553978640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c0ca0e0 con 0x7fd54c15e7f0 2023-12-08T08:38:47.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd553978640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c0ca3c0 con 0x7fd54c0b2eb0 2023-12-08T08:38:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd553978640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c0ca6a0 con 0x7fd54c0cff60 2023-12-08T08:38:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd551f15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd54c15e7f0 0x7fd54c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd551f15640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd54c15e7f0 0x7fd54c15c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53500/0 (socket says 172.21.15.16:53500) 2023-12-08T08:38:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd551f15640 1 -- 172.21.15.16:0/3871524081 learned_addr learned my addr 172.21.15.16:0/3871524081 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd551f15640 1 -- 172.21.15.16:0/3871524081 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd54c0cff60 msgr2=0x7fd54c0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd550f13640 1 --2- 172.21.15.16:0/3871524081 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd54c0cff60 0x7fd54c0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd551714640 1 --2- 172.21.15.16:0/3871524081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd54c0b2eb0 0x7fd54c15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd551f15640 1 --2- 172.21.15.16:0/3871524081 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd54c0cff60 0x7fd54c0cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd551f15640 1 -- 172.21.15.16:0/3871524081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd54c0b2eb0 msgr2=0x7fd54c15c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd551f15640 1 --2- 172.21.15.16:0/3871524081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd54c0b2eb0 0x7fd54c15c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd551f15640 1 -- 172.21.15.16:0/3871524081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5480094d0 con 0x7fd54c15e7f0 2023-12-08T08:38:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd551f15640 1 --2- 172.21.15.16:0/3871524081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd54c15e7f0 0x7fd54c15c9a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd54400cd20 tx=0x7fd54400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd541ffb640 1 -- 172.21.15.16:0/3871524081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544003b90 con 0x7fd54c15e7f0 2023-12-08T08:38:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd541ffb640 1 -- 172.21.15.16:0/3871524081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd544003cf0 con 0x7fd54c15e7f0 2023-12-08T08:38:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd553978640 1 -- 172.21.15.16:0/3871524081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54c0ca980 con 0x7fd54c15e7f0 2023-12-08T08:38:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd541ffb640 1 -- 172.21.15.16:0/3871524081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd544004240 con 0x7fd54c15e7f0 2023-12-08T08:38:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.643+0000 7fd553978640 1 -- 172.21.15.16:0/3871524081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54c0cb6c0 con 0x7fd54c15e7f0 2023-12-08T08:38:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.644+0000 7fd553978640 1 -- 172.21.15.16:0/3871524081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54c0caea0 con 0x7fd54c15e7f0 2023-12-08T08:38:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.645+0000 7fd541ffb640 1 -- 172.21.15.16:0/3871524081 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd5440043a0 con 0x7fd54c15e7f0 2023-12-08T08:38:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.645+0000 7fd541ffb640 1 --2- 172.21.15.16:0/3871524081 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd53006ae40 0x7fd53006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.645+0000 7fd541ffb640 1 -- 172.21.15.16:0/3871524081 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd544012070 con 0x7fd54c15e7f0 2023-12-08T08:38:47.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.645+0000 7fd551714640 1 --2- 172.21.15.16:0/3871524081 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd53006ae40 0x7fd53006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.646+0000 7fd551714640 1 --2- 172.21.15.16:0/3871524081 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd53006ae40 0x7fd53006d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd548036000 tx=0x7fd54803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:47.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.649+0000 7fd541ffb640 1 -- 172.21.15.16:0/3871524081 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd54c0caea0 con 0x7fd54c15e7f0 2023-12-08T08:38:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.869+0000 7fd553978640 1 -- 172.21.15.16:0/3871524081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd54c0caea0 con 0x7fd54c15e7f0 2023-12-08T08:38:47.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.869+0000 7fd541ffb640 1 -- 172.21.15.16:0/3871524081 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd54c0caea0 con 0x7fd54c15e7f0 2023-12-08T08:38:47.872 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:47.872 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-08T08:38:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.871+0000 7fd553978640 1 -- 172.21.15.16:0/3871524081 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd53006ae40 msgr2=0x7fd53006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.871+0000 7fd553978640 1 --2- 172.21.15.16:0/3871524081 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd53006ae40 0x7fd53006d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd548036000 tx=0x7fd54803c040 comp rx=0 tx=0).stop 2023-12-08T08:38:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.871+0000 7fd553978640 1 -- 172.21.15.16:0/3871524081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd54c15e7f0 msgr2=0x7fd54c15c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.871+0000 7fd553978640 1 --2- 172.21.15.16:0/3871524081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd54c15e7f0 0x7fd54c15c9a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd54400cd20 tx=0x7fd54400a660 comp rx=0 tx=0).stop 2023-12-08T08:38:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.871+0000 7fd553978640 1 -- 172.21.15.16:0/3871524081 shutdown_connections 2023-12-08T08:38:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.871+0000 7fd553978640 1 --2- 172.21.15.16:0/3871524081 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd53006ae40 0x7fd53006d300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.871+0000 7fd553978640 1 --2- 172.21.15.16:0/3871524081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd54c15e7f0 0x7fd54c15c9a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.871+0000 7fd553978640 1 --2- 172.21.15.16:0/3871524081 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd54c0cff60 0x7fd54c0cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.871+0000 7fd553978640 1 --2- 172.21.15.16:0/3871524081 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd54c0b2eb0 0x7fd54c15c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.871+0000 7fd553978640 1 -- 172.21.15.16:0/3871524081 >> 172.21.15.16:0/3871524081 conn(0x7fd54c0d3010 msgr2=0x7fd54c161060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.872+0000 7fd553978640 1 -- 172.21.15.16:0/3871524081 shutdown_connections 2023-12-08T08:38:47.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.872+0000 7fd553978640 1 -- 172.21.15.16:0/3871524081 wait complete. 2023-12-08T08:38:47.885 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-12-08T08:38:47.885 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.989+0000 7f5a7e3dc640 1 Processor -- start 2023-12-08T08:38:47.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.989+0000 7f5a7e3dc640 1 -- start start 2023-12-08T08:38:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a7e3dc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a780b6ac0 0x7f5a780cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a7e3dc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a780cd660 0x7f5a780cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a7e3dc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5a780cdfc0 0x7f5a7806b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a7e3dc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a78160cb0 con 0x7f5a780b6ac0 2023-12-08T08:38:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a7e3dc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a780d0070 con 0x7f5a780cdfc0 2023-12-08T08:38:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a7e3dc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780d02b0 con 0x7f5a780cd660 2023-12-08T08:38:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a77fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a780b6ac0 0x7f5a780cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a77fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a780b6ac0 0x7f5a780cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53510/0 (socket says 172.21.15.16:53510) 2023-12-08T08:38:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a77fff640 1 -- 172.21.15.16:0/833432497 learned_addr learned my addr 172.21.15.16:0/833432497 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a777fe640 1 --2- 172.21.15.16:0/833432497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a780cd660 0x7f5a780cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a7c979640 1 --2- 172.21.15.16:0/833432497 >> v2:172.21.15.112:3300/0 conn(0x7f5a780cdfc0 0x7f5a7806b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a77fff640 1 -- 172.21.15.16:0/833432497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a780cd660 msgr2=0x7f5a780cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a77fff640 1 --2- 172.21.15.16:0/833432497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a780cd660 0x7f5a780cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a77fff640 1 -- 172.21.15.16:0/833432497 >> v2:172.21.15.112:3300/0 conn(0x7f5a780cdfc0 msgr2=0x7f5a7806b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a77fff640 1 --2- 172.21.15.16:0/833432497 >> v2:172.21.15.112:3300/0 conn(0x7f5a780cdfc0 0x7f5a7806b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.990+0000 7f5a77fff640 1 -- 172.21.15.16:0/833432497 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a780cb430 con 0x7f5a780b6ac0 2023-12-08T08:38:47.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.991+0000 7f5a77fff640 1 --2- 172.21.15.16:0/833432497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a780b6ac0 0x7f5a780cfb30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5a7806e0d0 tx=0x7f5a6c02b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53419d4da4777ce6 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.991+0000 7f5a76ffd640 1 -- 172.21.15.16:0/833432497 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c031740 con 0x7f5a780b6ac0 2023-12-08T08:38:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.991+0000 7f5a76ffd640 1 -- 172.21.15.16:0/833432497 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c0318a0 con 0x7f5a780b6ac0 2023-12-08T08:38:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.991+0000 7f5a76ffd640 1 -- 172.21.15.16:0/833432497 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c031c30 con 0x7f5a780b6ac0 2023-12-08T08:38:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.992+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/833432497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a780b6ac0 msgr2=0x7f5a780cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.992+0000 7f5a7e3dc640 1 --2- 172.21.15.16:0/833432497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a780b6ac0 0x7f5a780cfb30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5a7806e0d0 tx=0x7f5a6c02b7e0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.992+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/833432497 shutdown_connections 2023-12-08T08:38:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.992+0000 7f5a7e3dc640 1 --2- 172.21.15.16:0/833432497 >> v2:172.21.15.112:3300/0 conn(0x7f5a780cdfc0 0x7f5a7806b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.992+0000 7f5a7e3dc640 1 --2- 172.21.15.16:0/833432497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a780cd660 0x7f5a780cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.992+0000 7f5a7e3dc640 1 --2- 172.21.15.16:0/833432497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a780b6ac0 0x7f5a780cfb30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.992+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/833432497 >> 172.21.15.16:0/833432497 conn(0x7f5a780d0de0 msgr2=0x7f5a780d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.992+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/833432497 shutdown_connections 2023-12-08T08:38:47.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.992+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/833432497 wait complete. 2023-12-08T08:38:47.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.993+0000 7f5a7e3dc640 1 Processor -- start 2023-12-08T08:38:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.993+0000 7f5a7e3dc640 1 -- start start 2023-12-08T08:38:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a7e3dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a780b6ac0 0x7f5a780b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a7e3dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a780cd660 0x7f5a7809e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a7e3dc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a780cdfc0 0x7f5a7815ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a7e3dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7815a430 con 0x7f5a780b6ac0 2023-12-08T08:38:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a7e3dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a781f3250 con 0x7f5a780cd660 2023-12-08T08:38:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a7e3dc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780b5cd0 con 0x7f5a780cdfc0 2023-12-08T08:38:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a780b6ac0 0x7f5a780b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a77fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a780b6ac0 0x7f5a780b5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53526/0 (socket says 172.21.15.16:53526) 2023-12-08T08:38:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a77fff640 1 -- 172.21.15.16:0/3485469398 learned_addr learned my addr 172.21.15.16:0/3485469398 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:47.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a77fff640 1 -- 172.21.15.16:0/3485469398 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a780cdfc0 msgr2=0x7f5a7815ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a77fff640 1 --2- 172.21.15.16:0/3485469398 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a780cdfc0 0x7f5a7815ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a77fff640 1 -- 172.21.15.16:0/3485469398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a780cd660 msgr2=0x7f5a7809e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a77fff640 1 --2- 172.21.15.16:0/3485469398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a780cd660 0x7f5a7809e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a77fff640 1 -- 172.21.15.16:0/3485469398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c0094d0 con 0x7f5a780b6ac0 2023-12-08T08:38:48.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a77fff640 1 --2- 172.21.15.16:0/3485469398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a780b6ac0 0x7f5a780b5650 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c02b820 tx=0x7f5a6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:48.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.994+0000 7f5a74ff9640 1 -- 172.21.15.16:0/3485469398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c033c70 con 0x7f5a780b6ac0 2023-12-08T08:38:48.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.995+0000 7f5a74ff9640 1 -- 172.21.15.16:0/3485469398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c033df0 con 0x7f5a780b6ac0 2023-12-08T08:38:48.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.995+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/3485469398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7815a640 con 0x7f5a780b6ac0 2023-12-08T08:38:48.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.995+0000 7f5a74ff9640 1 -- 172.21.15.16:0/3485469398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c03c640 con 0x7f5a780b6ac0 2023-12-08T08:38:48.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.995+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/3485469398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7815ac00 con 0x7f5a780b6ac0 2023-12-08T08:38:48.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.996+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/3485469398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7815adf0 con 0x7f5a780b6ac0 2023-12-08T08:38:48.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.996+0000 7f5a74ff9640 1 -- 172.21.15.16:0/3485469398 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a6c03a040 con 0x7f5a780b6ac0 2023-12-08T08:38:48.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.996+0000 7f5a74ff9640 1 --2- 172.21.15.16:0/3485469398 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5a6406ae40 0x7f5a6406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.996+0000 7f5a74ff9640 1 -- 172.21.15.16:0/3485469398 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5a6c0ab320 con 0x7f5a780b6ac0 2023-12-08T08:38:48.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.997+0000 7f5a777fe640 1 --2- 172.21.15.16:0/3485469398 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5a6406ae40 0x7f5a6406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:47.997+0000 7f5a777fe640 1 --2- 172.21.15.16:0/3485469398 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5a6406ae40 0x7f5a6406d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5a781f63f0 tx=0x7f5a68008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:48.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.001+0000 7f5a74ff9640 1 -- 172.21.15.16:0/3485469398 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a6c0760f0 con 0x7f5a780b6ac0 2023-12-08T08:38:48.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.177+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/3485469398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a78067430 con 0x7f5a780b6ac0 2023-12-08T08:38:48.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.178+0000 7f5a74ff9640 1 -- 172.21.15.16:0/3485469398 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f5a6c079710 con 0x7f5a780b6ac0 2023-12-08T08:38:48.181 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:48.181 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:24.030625+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.180+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/3485469398 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5a6406ae40 msgr2=0x7f5a6406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.180+0000 7f5a7e3dc640 1 --2- 172.21.15.16:0/3485469398 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5a6406ae40 0x7f5a6406d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5a781f63f0 tx=0x7f5a68008040 comp rx=0 tx=0).stop 2023-12-08T08:38:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.180+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/3485469398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a780b6ac0 msgr2=0x7f5a780b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.180+0000 7f5a7e3dc640 1 --2- 172.21.15.16:0/3485469398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a780b6ac0 0x7f5a780b5650 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c02b820 tx=0x7f5a6c0046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.180+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/3485469398 shutdown_connections 2023-12-08T08:38:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.180+0000 7f5a7e3dc640 1 --2- 172.21.15.16:0/3485469398 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5a6406ae40 0x7f5a6406d300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.180+0000 7f5a7e3dc640 1 --2- 172.21.15.16:0/3485469398 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5a780cdfc0 0x7f5a7815ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.180+0000 7f5a7e3dc640 1 --2- 172.21.15.16:0/3485469398 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5a780cd660 0x7f5a7809e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.180+0000 7f5a7e3dc640 1 --2- 172.21.15.16:0/3485469398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a780b6ac0 0x7f5a780b5650 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.180+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/3485469398 >> 172.21.15.16:0/3485469398 conn(0x7f5a780d0de0 msgr2=0x7f5a7815f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.180+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/3485469398 shutdown_connections 2023-12-08T08:38:48.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.180+0000 7f5a7e3dc640 1 -- 172.21.15.16:0/3485469398 wait complete. 2023-12-08T08:38:48.194 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:38:48.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:38:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.301+0000 7fd05c389640 1 Processor -- start 2023-12-08T08:38:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.301+0000 7fd05c389640 1 -- start start 2023-12-08T08:38:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.302+0000 7fd05c389640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0540d3b80 0x7fd0540ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.302+0000 7fd05c389640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0540b5ec0 con 0x7fd0540cfa80 2023-12-08T08:38:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.302+0000 7fd05c389640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0540d3fe0 con 0x7fd0540b6bf0 2023-12-08T08:38:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.302+0000 7fd05c389640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0540cebd0 con 0x7fd0540d3b80 2023-12-08T08:38:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.302+0000 7fd059924640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd0540d3b80 0x7fd0540ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.302+0000 7fd05a125640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd0540cfa80 0x7fd0540cfe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37500/0 (socket says 172.21.15.16:37500) 2023-12-08T08:38:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.302+0000 7fd05a125640 1 -- 172.21.15.16:0/930205259 learned_addr learned my addr 172.21.15.16:0/930205259 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.302+0000 7fd059924640 1 -- 172.21.15.16:0/930205259 >> v1:172.21.15.112:6789/0 conn(0x7fd0540b6bf0 legacy=0x7fd0540b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.302+0000 7fd059924640 1 -- 172.21.15.16:0/930205259 >> v1:172.21.15.16:6789/0 conn(0x7fd0540cfa80 legacy=0x7fd0540cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.302+0000 7fd059924640 1 -- 172.21.15.16:0/930205259 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0540cede0 con 0x7fd0540d3b80 2023-12-08T08:38:48.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.302+0000 7fd059123640 1 -- 172.21.15.16:0/930205259 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445588800 0 0) 0x7fd0540b5ec0 con 0x7fd0540cfa80 2023-12-08T08:38:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.303+0000 7fd059924640 1 --2- 172.21.15.16:0/930205259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540d3b80 0x7fd0540ce690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd0480097a0 tx=0x7fd04802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f683035e4ebcd2 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.303+0000 7fd059123640 1 -- 172.21.15.16:0/930205259 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04802e070 con 0x7fd0540d3b80 2023-12-08T08:38:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.303+0000 7fd059123640 1 -- 172.21.15.16:0/930205259 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0480315a0 con 0x7fd0540d3b80 2023-12-08T08:38:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.303+0000 7fd059123640 1 -- 172.21.15.16:0/930205259 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048031930 con 0x7fd0540d3b80 2023-12-08T08:38:48.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.304+0000 7fd05c389640 1 -- 172.21.15.16:0/930205259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540d3b80 msgr2=0x7fd0540ce690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.304+0000 7fd05c389640 1 --2- 172.21.15.16:0/930205259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540d3b80 0x7fd0540ce690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd0480097a0 tx=0x7fd04802b840 comp rx=0 tx=0).stop 2023-12-08T08:38:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.304+0000 7fd05c389640 1 -- 172.21.15.16:0/930205259 shutdown_connections 2023-12-08T08:38:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.304+0000 7fd05c389640 1 --2- 172.21.15.16:0/930205259 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540d3b80 0x7fd0540ce690 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.304+0000 7fd05c389640 1 -- 172.21.15.16:0/930205259 >> 172.21.15.16:0/930205259 conn(0x7fd05409c340 msgr2=0x7fd0540d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.304+0000 7fd05c389640 1 -- 172.21.15.16:0/930205259 shutdown_connections 2023-12-08T08:38:48.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.304+0000 7fd05c389640 1 -- 172.21.15.16:0/930205259 wait complete. 2023-12-08T08:38:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.305+0000 7fd05c389640 1 Processor -- start 2023-12-08T08:38:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.305+0000 7fd05c389640 1 -- start start 2023-12-08T08:38:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05c389640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0540b6bf0 0x7fd0540c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05c389640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0540cfa80 0x7fd05414be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05c389640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540d3b80 0x7fd05414f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05c389640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05414c7a0 con 0x7fd0540d3b80 2023-12-08T08:38:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05c389640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05414ca40 con 0x7fd0540cfa80 2023-12-08T08:38:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05c389640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd05414cd20 con 0x7fd0540b6bf0 2023-12-08T08:38:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05a926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540d3b80 0x7fd05414f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05a926640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540d3b80 0x7fd05414f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53544/0 (socket says 172.21.15.16:53544) 2023-12-08T08:38:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05a926640 1 -- 172.21.15.16:0/1933975251 learned_addr learned my addr 172.21.15.16:0/1933975251 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd059924640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0540cfa80 0x7fd05414be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05a125640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0540b6bf0 0x7fd0540c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05a926640 1 -- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0540b6bf0 msgr2=0x7fd0540c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05a926640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0540b6bf0 0x7fd0540c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05a926640 1 -- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0540cfa80 msgr2=0x7fd05414be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05a926640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0540cfa80 0x7fd05414be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05a926640 1 -- 172.21.15.16:0/1933975251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd048009400 con 0x7fd0540d3b80 2023-12-08T08:38:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd059924640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0540cfa80 0x7fd05414be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05a125640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0540b6bf0 0x7fd0540c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.306+0000 7fd05a926640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540d3b80 0x7fd05414f0d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00b490 tx=0x7fd04c00b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.307+0000 7fd052ffd640 1 -- 172.21.15.16:0/1933975251 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c011830 con 0x7fd0540d3b80 2023-12-08T08:38:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.307+0000 7fd052ffd640 1 -- 172.21.15.16:0/1933975251 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04c0119b0 con 0x7fd0540d3b80 2023-12-08T08:38:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.307+0000 7fd05c389640 1 -- 172.21.15.16:0/1933975251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd05414d000 con 0x7fd0540d3b80 2023-12-08T08:38:48.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.307+0000 7fd052ffd640 1 -- 172.21.15.16:0/1933975251 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c011d60 con 0x7fd0540d3b80 2023-12-08T08:38:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.307+0000 7fd05c389640 1 -- 172.21.15.16:0/1933975251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd05414c360 con 0x7fd0540d3b80 2023-12-08T08:38:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.307+0000 7fd05c389640 1 -- 172.21.15.16:0/1933975251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05414d240 con 0x7fd0540d3b80 2023-12-08T08:38:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.308+0000 7fd052ffd640 1 -- 172.21.15.16:0/1933975251 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd04c01a400 con 0x7fd0540d3b80 2023-12-08T08:38:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.308+0000 7fd052ffd640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd04006ae40 0x7fd04006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.308+0000 7fd052ffd640 1 -- 172.21.15.16:0/1933975251 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd04c089a50 con 0x7fd0540d3b80 2023-12-08T08:38:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.309+0000 7fd05a125640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd04006ae40 0x7fd04006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.309+0000 7fd05a125640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd04006ae40 0x7fd04006d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd0440012e0 tx=0x7fd044009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:48.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.313+0000 7fd052ffd640 1 -- 172.21.15.16:0/1933975251 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd05414d240 con 0x7fd0540d3b80 2023-12-08T08:38:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.486+0000 7fd05c389640 1 -- 172.21.15.16:0/1933975251 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd054067430 con 0x7fd0540d3b80 2023-12-08T08:38:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.486+0000 7fd052ffd640 1 -- 172.21.15.16:0/1933975251 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd04c015020 con 0x7fd0540d3b80 2023-12-08T08:38:48.490 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:38:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.488+0000 7fd05c389640 1 -- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd04006ae40 msgr2=0x7fd04006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.488+0000 7fd05c389640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd04006ae40 0x7fd04006d300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd0440012e0 tx=0x7fd044009040 comp rx=0 tx=0).stop 2023-12-08T08:38:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.488+0000 7fd05c389640 1 -- 172.21.15.16:0/1933975251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540d3b80 msgr2=0x7fd05414f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.488+0000 7fd05c389640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540d3b80 0x7fd05414f0d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00b490 tx=0x7fd04c00b950 comp rx=0 tx=0).stop 2023-12-08T08:38:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.488+0000 7fd05c389640 1 -- 172.21.15.16:0/1933975251 shutdown_connections 2023-12-08T08:38:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.488+0000 7fd05c389640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd04006ae40 0x7fd04006d300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.488+0000 7fd05c389640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0540d3b80 0x7fd05414f0d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.488+0000 7fd05c389640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0540cfa80 0x7fd05414be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.488+0000 7fd05c389640 1 --2- 172.21.15.16:0/1933975251 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0540b6bf0 0x7fd0540c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.488+0000 7fd05c389640 1 -- 172.21.15.16:0/1933975251 >> 172.21.15.16:0/1933975251 conn(0x7fd05409c340 msgr2=0x7fd05415c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.488+0000 7fd05c389640 1 -- 172.21.15.16:0/1933975251 shutdown_connections 2023-12-08T08:38:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.488+0000 7fd05c389640 1 -- 172.21.15.16:0/1933975251 wait complete. 2023-12-08T08:38:48.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:38:48.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.608+0000 7fd22f52c640 1 Processor -- start 2023-12-08T08:38:48.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.609+0000 7fd22f52c640 1 -- start start 2023-12-08T08:38:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.609+0000 7fd22f52c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd2280cfa30 0x7fd2280b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.609+0000 7fd22f52c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2280c6be0 con 0x7fd2280b4150 2023-12-08T08:38:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.609+0000 7fd22f52c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2280d3ff0 con 0x7fd2280cfe10 2023-12-08T08:38:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.609+0000 7fd22f52c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2280b4860 con 0x7fd2280cfa30 2023-12-08T08:38:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.610+0000 7fd22cac7640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd2280b4150 0x7fd2280b3460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37502/0 (socket says 172.21.15.16:37502) 2023-12-08T08:38:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.610+0000 7fd22cac7640 1 -- 172.21.15.16:0/3385935253 learned_addr learned my addr 172.21.15.16:0/3385935253 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.610+0000 7fd22d2c8640 1 --2- 172.21.15.16:0/3385935253 >> v2:172.21.15.112:3300/0 conn(0x7fd2280cfa30 0x7fd2280b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.610+0000 7fd21ffff640 1 -- 172.21.15.16:0/3385935253 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844623538 0 0) 0x7fd2280c6be0 con 0x7fd2280b4150 2023-12-08T08:38:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.610+0000 7fd21ffff640 1 -- 172.21.15.16:0/3385935253 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd214003520 con 0x7fd2280b4150 2023-12-08T08:38:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.610+0000 7fd22d2c8640 1 -- 172.21.15.16:0/3385935253 >> v1:172.21.15.112:6789/0 conn(0x7fd2280cfe10 legacy=0x7fd2280cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.610+0000 7fd21ffff640 1 -- 172.21.15.16:0/3385935253 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3525403909 0 0) 0x7fd214003520 con 0x7fd2280b4150 2023-12-08T08:38:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.610+0000 7fd22d2c8640 1 -- 172.21.15.16:0/3385935253 >> v1:172.21.15.16:6789/0 conn(0x7fd2280b4150 legacy=0x7fd2280b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.610+0000 7fd22d2c8640 1 -- 172.21.15.16:0/3385935253 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2280d3d50 con 0x7fd2280cfa30 2023-12-08T08:38:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.611+0000 7fd22d2c8640 1 --2- 172.21.15.16:0/3385935253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2280cfa30 0x7fd2280b2f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd2240097a0 tx=0x7fd22402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acfecd3864d09a server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.611+0000 7fd21ffff640 1 -- 172.21.15.16:0/3385935253 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218001670 con 0x7fd2280cfa30 2023-12-08T08:38:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.611+0000 7fd21ffff640 1 -- 172.21.15.16:0/3385935253 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22402e050 con 0x7fd2280cfa30 2023-12-08T08:38:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.611+0000 7fd21ffff640 1 -- 172.21.15.16:0/3385935253 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd224030710 con 0x7fd2280cfa30 2023-12-08T08:38:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.612+0000 7fd22f52c640 1 -- 172.21.15.16:0/3385935253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2280cfa30 msgr2=0x7fd2280b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.612+0000 7fd22f52c640 1 --2- 172.21.15.16:0/3385935253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2280cfa30 0x7fd2280b2f20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd2240097a0 tx=0x7fd22402b790 comp rx=0 tx=0).stop 2023-12-08T08:38:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.612+0000 7fd22f52c640 1 -- 172.21.15.16:0/3385935253 shutdown_connections 2023-12-08T08:38:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.612+0000 7fd22f52c640 1 --2- 172.21.15.16:0/3385935253 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2280cfa30 0x7fd2280b2f20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.612+0000 7fd22f52c640 1 -- 172.21.15.16:0/3385935253 >> 172.21.15.16:0/3385935253 conn(0x7fd2280d3090 msgr2=0x7fd2280d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.612+0000 7fd22f52c640 1 -- 172.21.15.16:0/3385935253 shutdown_connections 2023-12-08T08:38:48.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.612+0000 7fd22f52c640 1 -- 172.21.15.16:0/3385935253 wait complete. 2023-12-08T08:38:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.613+0000 7fd22f52c640 1 Processor -- start 2023-12-08T08:38:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.613+0000 7fd22f52c640 1 -- start start 2023-12-08T08:38:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22f52c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2280b4150 0x7fd2280ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22f52c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2280cfa30 0x7fd2281540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22f52c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2280cfe10 0x7fd228152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22f52c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2281f6900 con 0x7fd2280cfa30 2023-12-08T08:38:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22f52c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2281f6be0 con 0x7fd2280cfe10 2023-12-08T08:38:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22f52c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2281f6ec0 con 0x7fd2280b4150 2023-12-08T08:38:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22cac7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2280cfa30 0x7fd2281540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22cac7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2280cfa30 0x7fd2281540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53558/0 (socket says 172.21.15.16:53558) 2023-12-08T08:38:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22cac7640 1 -- 172.21.15.16:0/565595012 learned_addr learned my addr 172.21.15.16:0/565595012 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22d2c8640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2280b4150 0x7fd2280ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22dac9640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2280cfe10 0x7fd228152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22cac7640 1 -- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2280b4150 msgr2=0x7fd2280ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22cac7640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2280b4150 0x7fd2280ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22cac7640 1 -- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2280cfe10 msgr2=0x7fd228152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22cac7640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2280cfe10 0x7fd228152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22cac7640 1 -- 172.21.15.16:0/565595012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd224009400 con 0x7fd2280cfa30 2023-12-08T08:38:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22dac9640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2280cfe10 0x7fd228152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.614+0000 7fd22cac7640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2280cfa30 0x7fd2281540b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd2180011e0 tx=0x7fd21800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.615+0000 7fd21dffb640 1 -- 172.21.15.16:0/565595012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218005ad0 con 0x7fd2280cfa30 2023-12-08T08:38:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.615+0000 7fd21dffb640 1 -- 172.21.15.16:0/565595012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd218005c50 con 0x7fd2280cfa30 2023-12-08T08:38:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.615+0000 7fd21dffb640 1 -- 172.21.15.16:0/565595012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2180072b0 con 0x7fd2280cfa30 2023-12-08T08:38:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.615+0000 7fd22f52c640 1 -- 172.21.15.16:0/565595012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2281f71a0 con 0x7fd2280cfa30 2023-12-08T08:38:48.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.615+0000 7fd22f52c640 1 -- 172.21.15.16:0/565595012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2281f6590 con 0x7fd2280cfa30 2023-12-08T08:38:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.616+0000 7fd22f52c640 1 -- 172.21.15.16:0/565595012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2280b5c80 con 0x7fd2280cfa30 2023-12-08T08:38:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.616+0000 7fd21dffb640 1 -- 172.21.15.16:0/565595012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd218007410 con 0x7fd2280cfa30 2023-12-08T08:38:48.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.616+0000 7fd21dffb640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd20c06ae70 0x7fd20c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.617+0000 7fd21dffb640 1 -- 172.21.15.16:0/565595012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd218088e80 con 0x7fd2280cfa30 2023-12-08T08:38:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.617+0000 7fd22d2c8640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd20c06ae70 0x7fd20c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.617+0000 7fd22d2c8640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd20c06ae70 0x7fd20c06d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd224034000 tx=0x7fd22402f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:48.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.621+0000 7fd21dffb640 1 -- 172.21.15.16:0/565595012 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd2280b5c80 con 0x7fd2280cfa30 2023-12-08T08:38:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.798+0000 7fd22f52c640 1 -- 172.21.15.16:0/565595012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2280b5c80 con 0x7fd2280cfa30 2023-12-08T08:38:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.798+0000 7fd21dffb640 1 -- 172.21.15.16:0/565595012 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd2280b5c80 con 0x7fd2280cfa30 2023-12-08T08:38:48.802 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:38:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.800+0000 7fd22f52c640 1 -- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd20c06ae70 msgr2=0x7fd20c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.800+0000 7fd22f52c640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd20c06ae70 0x7fd20c06d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd224034000 tx=0x7fd22402f000 comp rx=0 tx=0).stop 2023-12-08T08:38:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.800+0000 7fd22f52c640 1 -- 172.21.15.16:0/565595012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2280cfa30 msgr2=0x7fd2281540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.800+0000 7fd22f52c640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2280cfa30 0x7fd2281540b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd2180011e0 tx=0x7fd21800cb90 comp rx=0 tx=0).stop 2023-12-08T08:38:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.800+0000 7fd22f52c640 1 -- 172.21.15.16:0/565595012 shutdown_connections 2023-12-08T08:38:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.800+0000 7fd22f52c640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd20c06ae70 0x7fd20c06d330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.800+0000 7fd22f52c640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd2280cfe10 0x7fd228152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.800+0000 7fd22f52c640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2280cfa30 0x7fd2281540b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.800+0000 7fd22f52c640 1 --2- 172.21.15.16:0/565595012 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd2280b4150 0x7fd2280ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.800+0000 7fd22f52c640 1 -- 172.21.15.16:0/565595012 >> 172.21.15.16:0/565595012 conn(0x7fd2280d3090 msgr2=0x7fd22815d210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.800+0000 7fd22f52c640 1 -- 172.21.15.16:0/565595012 shutdown_connections 2023-12-08T08:38:48.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.800+0000 7fd22f52c640 1 -- 172.21.15.16:0/565595012 wait complete. 2023-12-08T08:38:48.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:38:48.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.920+0000 7f08814c9640 1 Processor -- start 2023-12-08T08:38:48.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.921+0000 7f08814c9640 1 -- start start 2023-12-08T08:38:48.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.921+0000 7f08814c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f087c0ce690 0x7f087c0b7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.921+0000 7f08814c9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087c0b5ec0 con 0x7f087c0cfa80 2023-12-08T08:38:48.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.921+0000 7f08814c9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087c0d3fe0 con 0x7f087c0d3b80 2023-12-08T08:38:48.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.921+0000 7f08814c9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f087c0ceb20 con 0x7f087c0ce690 2023-12-08T08:38:48.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.921+0000 7f087b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f087c0ce690 0x7f087c0b7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.922+0000 7f087affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f087c0cfa80 0x7f087c0cfe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37506/0 (socket says 172.21.15.16:37506) 2023-12-08T08:38:48.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.922+0000 7f087affd640 1 -- 172.21.15.16:0/3212399850 learned_addr learned my addr 172.21.15.16:0/3212399850 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:48.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.922+0000 7f087b7fe640 1 -- 172.21.15.16:0/3212399850 >> v1:172.21.15.112:6789/0 conn(0x7f087c0d3b80 legacy=0x7f087c0b6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:48.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.922+0000 7f087b7fe640 1 -- 172.21.15.16:0/3212399850 >> v1:172.21.15.16:6789/0 conn(0x7f087c0cfa80 legacy=0x7f087c0cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.922+0000 7f0879ffb640 1 -- 172.21.15.16:0/3212399850 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079710477 0 0) 0x7f087c0b5ec0 con 0x7f087c0cfa80 2023-12-08T08:38:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.922+0000 7f087b7fe640 1 -- 172.21.15.16:0/3212399850 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c0cede0 con 0x7f087c0ce690 2023-12-08T08:38:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.923+0000 7f087b7fe640 1 --2- 172.21.15.16:0/3212399850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c0ce690 0x7f087c0b7300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0874009d90 tx=0x7f087402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b200f1e3f025e63 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.923+0000 7f0879ffb640 1 -- 172.21.15.16:0/3212399850 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087402cc60 con 0x7f087c0ce690 2023-12-08T08:38:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.923+0000 7f0879ffb640 1 -- 172.21.15.16:0/3212399850 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087402cdc0 con 0x7f087c0ce690 2023-12-08T08:38:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.923+0000 7f0879ffb640 1 -- 172.21.15.16:0/3212399850 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0874034660 con 0x7f087c0ce690 2023-12-08T08:38:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.923+0000 7f08814c9640 1 -- 172.21.15.16:0/3212399850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c0ce690 msgr2=0x7f087c0b7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.923+0000 7f08814c9640 1 --2- 172.21.15.16:0/3212399850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c0ce690 0x7f087c0b7300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0874009d90 tx=0x7f087402beb0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.923+0000 7f08814c9640 1 -- 172.21.15.16:0/3212399850 shutdown_connections 2023-12-08T08:38:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.923+0000 7f08814c9640 1 --2- 172.21.15.16:0/3212399850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c0ce690 0x7f087c0b7300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.923+0000 7f08814c9640 1 -- 172.21.15.16:0/3212399850 >> 172.21.15.16:0/3212399850 conn(0x7f087c09c340 msgr2=0x7f087c0d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.923+0000 7f08814c9640 1 -- 172.21.15.16:0/3212399850 shutdown_connections 2023-12-08T08:38:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.923+0000 7f08814c9640 1 -- 172.21.15.16:0/3212399850 wait complete. 2023-12-08T08:38:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.925+0000 7f08814c9640 1 Processor -- start 2023-12-08T08:38:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.925+0000 7f08814c9640 1 -- start start 2023-12-08T08:38:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.925+0000 7f08814c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f087c0ce690 0x7f087c14e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.925+0000 7f08814c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c0cfa80 0x7f087c0c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.925+0000 7f08814c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f087c0d3b80 0x7f087c0c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.925+0000 7f08814c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c160240 con 0x7f087c0cfa80 2023-12-08T08:38:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.925+0000 7f08814c9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c160520 con 0x7f087c0ce690 2023-12-08T08:38:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.925+0000 7f08814c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0ca410 con 0x7f087c0d3b80 2023-12-08T08:38:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.925+0000 7f087a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c0cfa80 0x7f087c0c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f087a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c0cfa80 0x7f087c0c6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53576/0 (socket says 172.21.15.16:53576) 2023-12-08T08:38:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f087a7fc640 1 -- 172.21.15.16:0/2983475747 learned_addr learned my addr 172.21.15.16:0/2983475747 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f087b7fe640 1 --2- 172.21.15.16:0/2983475747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f087c0d3b80 0x7f087c0c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f087affd640 1 --2- 172.21.15.16:0/2983475747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f087c0ce690 0x7f087c14e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f087a7fc640 1 -- 172.21.15.16:0/2983475747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f087c0d3b80 msgr2=0x7f087c0c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f087a7fc640 1 --2- 172.21.15.16:0/2983475747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f087c0d3b80 0x7f087c0c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f087a7fc640 1 -- 172.21.15.16:0/2983475747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f087c0ce690 msgr2=0x7f087c14e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f087a7fc640 1 --2- 172.21.15.16:0/2983475747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f087c0ce690 0x7f087c14e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f087a7fc640 1 -- 172.21.15.16:0/2983475747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0874009a70 con 0x7f087c0cfa80 2023-12-08T08:38:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f087a7fc640 1 --2- 172.21.15.16:0/2983475747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c0cfa80 0x7f087c0c6f10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f087000c450 tx=0x7f087000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:48.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f08637fe640 1 -- 172.21.15.16:0/2983475747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08700052d0 con 0x7f087c0cfa80 2023-12-08T08:38:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f08637fe640 1 -- 172.21.15.16:0/2983475747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0870005430 con 0x7f087c0cfa80 2023-12-08T08:38:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f08814c9640 1 -- 172.21.15.16:0/2983475747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087c0ca6f0 con 0x7f087c0cfa80 2023-12-08T08:38:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f08637fe640 1 -- 172.21.15.16:0/2983475747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870012870 con 0x7f087c0cfa80 2023-12-08T08:38:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.926+0000 7f08814c9640 1 -- 172.21.15.16:0/2983475747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087c14ebf0 con 0x7f087c0cfa80 2023-12-08T08:38:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.927+0000 7f08814c9640 1 -- 172.21.15.16:0/2983475747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087c0ca8e0 con 0x7f087c0cfa80 2023-12-08T08:38:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.928+0000 7f08637fe640 1 -- 172.21.15.16:0/2983475747 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f08700129d0 con 0x7f087c0cfa80 2023-12-08T08:38:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.928+0000 7f08637fe640 1 --2- 172.21.15.16:0/2983475747 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f086806ae40 0x7f086806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.928+0000 7f08637fe640 1 -- 172.21.15.16:0/2983475747 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0870089260 con 0x7f087c0cfa80 2023-12-08T08:38:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.928+0000 7f087affd640 1 --2- 172.21.15.16:0/2983475747 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f086806ae40 0x7f086806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:48.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.929+0000 7f087affd640 1 --2- 172.21.15.16:0/2983475747 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f086806ae40 0x7f086806d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f086c007f70 tx=0x7f086c007f00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:48.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:48.932+0000 7f08637fe640 1 -- 172.21.15.16:0/2983475747 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0870054160 con 0x7f087c0cfa80 2023-12-08T08:38:49.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.107+0000 7f08814c9640 1 -- 172.21.15.16:0/2983475747 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f087c0d2e50 con 0x7f087c0cfa80 2023-12-08T08:38:49.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.108+0000 7f08637fe640 1 -- 172.21.15.16:0/2983475747 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0870057780 con 0x7f087c0cfa80 2023-12-08T08:38:49.111 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:38:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.109+0000 7f08814c9640 1 -- 172.21.15.16:0/2983475747 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f086806ae40 msgr2=0x7f086806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.109+0000 7f08814c9640 1 --2- 172.21.15.16:0/2983475747 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f086806ae40 0x7f086806d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f086c007f70 tx=0x7f086c007f00 comp rx=0 tx=0).stop 2023-12-08T08:38:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.109+0000 7f08814c9640 1 -- 172.21.15.16:0/2983475747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c0cfa80 msgr2=0x7f087c0c6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.109+0000 7f08814c9640 1 --2- 172.21.15.16:0/2983475747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c0cfa80 0x7f087c0c6f10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f087000c450 tx=0x7f087000c910 comp rx=0 tx=0).stop 2023-12-08T08:38:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.110+0000 7f08814c9640 1 -- 172.21.15.16:0/2983475747 shutdown_connections 2023-12-08T08:38:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.110+0000 7f08814c9640 1 --2- 172.21.15.16:0/2983475747 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f086806ae40 0x7f086806d300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.110+0000 7f08814c9640 1 --2- 172.21.15.16:0/2983475747 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f087c0d3b80 0x7f087c0c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.110+0000 7f08814c9640 1 --2- 172.21.15.16:0/2983475747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f087c0cfa80 0x7f087c0c6f10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.110+0000 7f08814c9640 1 --2- 172.21.15.16:0/2983475747 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f087c0ce690 0x7f087c14e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.110+0000 7f08814c9640 1 -- 172.21.15.16:0/2983475747 >> 172.21.15.16:0/2983475747 conn(0x7f087c09c340 msgr2=0x7f087c06b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.110+0000 7f08814c9640 1 -- 172.21.15.16:0/2983475747 shutdown_connections 2023-12-08T08:38:49.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.110+0000 7f08814c9640 1 -- 172.21.15.16:0/2983475747 wait complete. 2023-12-08T08:38:49.123 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:38:49.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.229+0000 7fa3e7583640 1 Processor -- start 2023-12-08T08:38:49.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.230+0000 7fa3e7583640 1 -- start start 2023-12-08T08:38:49.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.230+0000 7fa3e7583640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e00cf570 0x7fa3e00cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.230+0000 7fa3e7583640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e00cfef0 con 0x7fa3e00b6c50 2023-12-08T08:38:49.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.231+0000 7fa3e7583640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e00b5f00 con 0x7fa3e00b8950 2023-12-08T08:38:49.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.231+0000 7fa3e7583640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00b6170 con 0x7fa3e00cf570 2023-12-08T08:38:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.231+0000 7fa3e5b20640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa3e00b6c50 0x7fa3e00b70e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37508/0 (socket says 172.21.15.16:37508) 2023-12-08T08:38:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.231+0000 7fa3e5b20640 1 -- 172.21.15.16:0/1408503539 learned_addr learned my addr 172.21.15.16:0/1408503539 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.231+0000 7fa3e4b1e640 1 --2- 172.21.15.16:0/1408503539 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e00cf570 0x7fa3e00cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.231+0000 7fa3dffff640 1 -- 172.21.15.16:0/1408503539 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285812069 0 0) 0x7fa3e00cfef0 con 0x7fa3e00b6c50 2023-12-08T08:38:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.231+0000 7fa3dffff640 1 -- 172.21.15.16:0/1408503539 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c8003520 con 0x7fa3e00b6c50 2023-12-08T08:38:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.231+0000 7fa3dffff640 1 -- 172.21.15.16:0/1408503539 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322309111 0 0) 0x7fa3e00b5f00 con 0x7fa3e00b8950 2023-12-08T08:38:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.231+0000 7fa3dffff640 1 -- 172.21.15.16:0/1408503539 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3e00cfef0 con 0x7fa3e00b8950 2023-12-08T08:38:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.231+0000 7fa3e4b1e640 1 -- 172.21.15.16:0/1408503539 >> v1:172.21.15.112:6789/0 conn(0x7fa3e00b8950 legacy=0x7fa3e00ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.231+0000 7fa3e4b1e640 1 -- 172.21.15.16:0/1408503539 >> v1:172.21.15.16:6789/0 conn(0x7fa3e00b6c50 legacy=0x7fa3e00b70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.231+0000 7fa3e4b1e640 1 -- 172.21.15.16:0/1408503539 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e00b6450 con 0x7fa3e00cf570 2023-12-08T08:38:49.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.232+0000 7fa3e4b1e640 1 --2- 172.21.15.16:0/1408503539 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e00cf570 0x7fa3e00cf9b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40097d0 tx=0x7fa3d402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6ad00063507cbe4 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:49.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.232+0000 7fa3dffff640 1 -- 172.21.15.16:0/1408503539 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4031800 con 0x7fa3e00cf570 2023-12-08T08:38:49.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.232+0000 7fa3dffff640 1 -- 172.21.15.16:0/1408503539 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d4031960 con 0x7fa3e00cf570 2023-12-08T08:38:49.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.232+0000 7fa3dffff640 1 -- 172.21.15.16:0/1408503539 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4031cd0 con 0x7fa3e00cf570 2023-12-08T08:38:49.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.233+0000 7fa3e7583640 1 -- 172.21.15.16:0/1408503539 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e00cf570 msgr2=0x7fa3e00cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.233+0000 7fa3e7583640 1 --2- 172.21.15.16:0/1408503539 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e00cf570 0x7fa3e00cf9b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40097d0 tx=0x7fa3d402b8a0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.233+0000 7fa3e7583640 1 -- 172.21.15.16:0/1408503539 shutdown_connections 2023-12-08T08:38:49.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.233+0000 7fa3e7583640 1 --2- 172.21.15.16:0/1408503539 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e00cf570 0x7fa3e00cf9b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.233+0000 7fa3e7583640 1 -- 172.21.15.16:0/1408503539 >> 172.21.15.16:0/1408503539 conn(0x7fa3e00d3010 msgr2=0x7fa3e00d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:49.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.233+0000 7fa3e7583640 1 -- 172.21.15.16:0/1408503539 shutdown_connections 2023-12-08T08:38:49.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.233+0000 7fa3e7583640 1 -- 172.21.15.16:0/1408503539 wait complete. 2023-12-08T08:38:49.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.234+0000 7fa3e7583640 1 Processor -- start 2023-12-08T08:38:49.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.234+0000 7fa3e7583640 1 -- start start 2023-12-08T08:38:49.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e7583640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e00b6c50 0x7fa3e015f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e7583640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e00b8950 0x7fa3e00b2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e7583640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e00cf570 0x7fa3e009f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e7583640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00b2300 con 0x7fa3e00b6c50 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e7583640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e006c570 con 0x7fa3e00cf570 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e7583640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e00b19d0 con 0x7fa3e00b8950 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e531f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e00b6c50 0x7fa3e015f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e531f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e00b6c50 0x7fa3e015f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53592/0 (socket says 172.21.15.16:53592) 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e531f640 1 -- 172.21.15.16:0/4005717616 learned_addr learned my addr 172.21.15.16:0/4005717616 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e5b20640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e00cf570 0x7fa3e009f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e4b1e640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e00b8950 0x7fa3e00b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e531f640 1 -- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e00b8950 msgr2=0x7fa3e00b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e531f640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e00b8950 0x7fa3e00b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e531f640 1 -- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e00cf570 msgr2=0x7fa3e009f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e531f640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e00cf570 0x7fa3e009f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e531f640 1 -- 172.21.15.16:0/4005717616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d4009400 con 0x7fa3e00b6c50 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e5b20640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e00cf570 0x7fa3e009f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:49.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e531f640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e00b6c50 0x7fa3e015f120 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800c500 tx=0x7fa3d800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:49.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3ddffb640 1 -- 172.21.15.16:0/4005717616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d80052d0 con 0x7fa3e00b6c50 2023-12-08T08:38:49.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3ddffb640 1 -- 172.21.15.16:0/4005717616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d8005450 con 0x7fa3e00b6c50 2023-12-08T08:38:49.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3e7583640 1 -- 172.21.15.16:0/4005717616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e00b1cb0 con 0x7fa3e00b6c50 2023-12-08T08:38:49.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.235+0000 7fa3ddffb640 1 -- 172.21.15.16:0/4005717616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8012870 con 0x7fa3e00b6c50 2023-12-08T08:38:49.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.236+0000 7fa3e7583640 1 -- 172.21.15.16:0/4005717616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e015f660 con 0x7fa3e00b6c50 2023-12-08T08:38:49.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.237+0000 7fa3e7583640 1 -- 172.21.15.16:0/4005717616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e0067430 con 0x7fa3e00b6c50 2023-12-08T08:38:49.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.237+0000 7fa3ddffb640 1 -- 172.21.15.16:0/4005717616 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa3d80129d0 con 0x7fa3e00b6c50 2023-12-08T08:38:49.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.237+0000 7fa3ddffb640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3cc06ae40 0x7fa3cc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.237+0000 7fa3ddffb640 1 -- 172.21.15.16:0/4005717616 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa3d80895c0 con 0x7fa3e00b6c50 2023-12-08T08:38:49.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.238+0000 7fa3e4b1e640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3cc06ae40 0x7fa3cc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.238+0000 7fa3e4b1e640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3cc06ae40 0x7fa3cc06d300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4036000 tx=0x7fa3d4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:49.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.241+0000 7fa3ddffb640 1 -- 172.21.15.16:0/4005717616 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3d80542f0 con 0x7fa3e00b6c50 2023-12-08T08:38:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.417+0000 7fa3e7583640 1 -- 172.21.15.16:0/4005717616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa3e00d1280 con 0x7fa3e00b6c50 2023-12-08T08:38:49.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.417+0000 7fa3ddffb640 1 -- 172.21.15.16:0/4005717616 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa3d8057910 con 0x7fa3e00b6c50 2023-12-08T08:38:49.420 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:38:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.419+0000 7fa3e7583640 1 -- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3cc06ae40 msgr2=0x7fa3cc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.419+0000 7fa3e7583640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3cc06ae40 0x7fa3cc06d300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4036000 tx=0x7fa3d4008030 comp rx=0 tx=0).stop 2023-12-08T08:38:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.419+0000 7fa3e7583640 1 -- 172.21.15.16:0/4005717616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e00b6c50 msgr2=0x7fa3e015f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.419+0000 7fa3e7583640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e00b6c50 0x7fa3e015f120 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800c500 tx=0x7fa3d800c9c0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.419+0000 7fa3e7583640 1 -- 172.21.15.16:0/4005717616 shutdown_connections 2023-12-08T08:38:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.419+0000 7fa3e7583640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3cc06ae40 0x7fa3cc06d300 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.419+0000 7fa3e7583640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3e00cf570 0x7fa3e009f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.419+0000 7fa3e7583640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa3e00b8950 0x7fa3e00b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.419+0000 7fa3e7583640 1 --2- 172.21.15.16:0/4005717616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3e00b6c50 0x7fa3e015f120 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.419+0000 7fa3e7583640 1 -- 172.21.15.16:0/4005717616 >> 172.21.15.16:0/4005717616 conn(0x7fa3e00d3010 msgr2=0x7fa3e00b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.419+0000 7fa3e7583640 1 -- 172.21.15.16:0/4005717616 shutdown_connections 2023-12-08T08:38:49.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.419+0000 7fa3e7583640 1 -- 172.21.15.16:0/4005717616 wait complete. 2023-12-08T08:38:49.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-08T08:38:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.536+0000 7f9e35e4f640 1 Processor -- start 2023-12-08T08:38:49.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.536+0000 7f9e35e4f640 1 -- start start 2023-12-08T08:38:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.537+0000 7f9e35e4f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f9e3015d580 0x7f9e300d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.537+0000 7f9e35e4f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e3015f890 0x7f9e3015d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.537+0000 7f9e35e4f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3015ee90 con 0x7f9e300d3ad0 2023-12-08T08:38:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.537+0000 7f9e35e4f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e300d0fe0 con 0x7f9e3015f890 2023-12-08T08:38:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.537+0000 7f9e35e4f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e300d1280 con 0x7f9e3015d580 2023-12-08T08:38:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.537+0000 7f9e2ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e3015f890 0x7f9e3015d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.537+0000 7f9e2ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e3015f890 0x7f9e3015d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53602/0 (socket says 172.21.15.16:53602) 2023-12-08T08:38:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.537+0000 7f9e2ffff640 1 -- 172.21.15.16:0/3556452021 learned_addr learned my addr 172.21.15.16:0/3556452021 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.537+0000 7f9e2f7fe640 1 --2- 172.21.15.16:0/3556452021 >> v2:172.21.15.112:3300/0 conn(0x7f9e3015d580 0x7f9e300d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.538+0000 7f9e2ffff640 1 -- 172.21.15.16:0/3556452021 >> v2:172.21.15.112:3300/0 conn(0x7f9e3015d580 msgr2=0x7f9e300d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.538+0000 7f9e2ffff640 1 --2- 172.21.15.16:0/3556452021 >> v2:172.21.15.112:3300/0 conn(0x7f9e3015d580 0x7f9e300d1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.538+0000 7f9e2ffff640 1 -- 172.21.15.16:0/3556452021 >> v1:172.21.15.112:6789/0 conn(0x7f9e300d3ad0 legacy=0x7f9e3015f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.538+0000 7f9e2ffff640 1 -- 172.21.15.16:0/3556452021 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e300d1560 con 0x7f9e3015f890 2023-12-08T08:38:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.538+0000 7f9e2ffff640 1 --2- 172.21.15.16:0/3556452021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e3015f890 0x7f9e3015d960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009e50 tx=0x7f9e2802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6047b8f4889a269 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.538+0000 7f9e2e7fc640 1 -- 172.21.15.16:0/3556452021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2802ca80 con 0x7f9e3015f890 2023-12-08T08:38:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.538+0000 7f9e2e7fc640 1 -- 172.21.15.16:0/3556452021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2802cbe0 con 0x7f9e3015f890 2023-12-08T08:38:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.539+0000 7f9e2e7fc640 1 -- 172.21.15.16:0/3556452021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e280343f0 con 0x7f9e3015f890 2023-12-08T08:38:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.539+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3556452021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e3015f890 msgr2=0x7f9e3015d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.539+0000 7f9e35e4f640 1 --2- 172.21.15.16:0/3556452021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e3015f890 0x7f9e3015d960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9e28009e50 tx=0x7f9e2802bee0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.539+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3556452021 shutdown_connections 2023-12-08T08:38:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.539+0000 7f9e35e4f640 1 --2- 172.21.15.16:0/3556452021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e3015f890 0x7f9e3015d960 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.539+0000 7f9e35e4f640 1 --2- 172.21.15.16:0/3556452021 >> v2:172.21.15.112:3300/0 conn(0x7f9e3015d580 0x7f9e300d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.539+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3556452021 >> 172.21.15.16:0/3556452021 conn(0x7f9e300d2fb0 msgr2=0x7f9e300d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:49.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.539+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3556452021 shutdown_connections 2023-12-08T08:38:49.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.539+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3556452021 wait complete. 2023-12-08T08:38:49.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.540+0000 7f9e35e4f640 1 Processor -- start 2023-12-08T08:38:49.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.541+0000 7f9e35e4f640 1 -- start start 2023-12-08T08:38:49.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.541+0000 7f9e35e4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e300d3ad0 0x7f9e301f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.541+0000 7f9e35e4f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e3015d580 0x7f9e300cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.541+0000 7f9e35e4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e3015f890 0x7f9e300cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.541+0000 7f9e35e4f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e300cb070 con 0x7f9e300d3ad0 2023-12-08T08:38:49.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.541+0000 7f9e35e4f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e300ca3f0 con 0x7f9e3015f890 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.541+0000 7f9e35e4f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e300ca6d0 con 0x7f9e3015d580 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.541+0000 7f9e2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e300d3ad0 0x7f9e301f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.541+0000 7f9e2f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e300d3ad0 0x7f9e301f26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53604/0 (socket says 172.21.15.16:53604) 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.541+0000 7f9e2f7fe640 1 -- 172.21.15.16:0/3624246806 learned_addr learned my addr 172.21.15.16:0/3624246806 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.542+0000 7f9e2ffff640 1 --2- 172.21.15.16:0/3624246806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e3015f890 0x7f9e300cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.542+0000 7f9e2effd640 1 --2- 172.21.15.16:0/3624246806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e3015d580 0x7f9e300cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.542+0000 7f9e2f7fe640 1 -- 172.21.15.16:0/3624246806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e3015d580 msgr2=0x7f9e300cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.542+0000 7f9e2f7fe640 1 --2- 172.21.15.16:0/3624246806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e3015d580 0x7f9e300cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.542+0000 7f9e2f7fe640 1 -- 172.21.15.16:0/3624246806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e3015f890 msgr2=0x7f9e300cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.542+0000 7f9e2f7fe640 1 --2- 172.21.15.16:0/3624246806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e3015f890 0x7f9e300cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.542+0000 7f9e2f7fe640 1 -- 172.21.15.16:0/3624246806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e28009b30 con 0x7f9e300d3ad0 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.542+0000 7f9e2f7fe640 1 --2- 172.21.15.16:0/3624246806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e300d3ad0 0x7f9e301f26a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000e9b0 tx=0x7f9e2000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.542+0000 7f9e17fff640 1 -- 172.21.15.16:0/3624246806 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20009670 con 0x7f9e300d3ad0 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.542+0000 7f9e17fff640 1 -- 172.21.15.16:0/3624246806 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2000c9a0 con 0x7f9e300d3ad0 2023-12-08T08:38:49.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.542+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3624246806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e300ca9b0 con 0x7f9e300d3ad0 2023-12-08T08:38:49.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.542+0000 7f9e17fff640 1 -- 172.21.15.16:0/3624246806 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000cd50 con 0x7f9e300d3ad0 2023-12-08T08:38:49.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.542+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3624246806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e300ca080 con 0x7f9e300d3ad0 2023-12-08T08:38:49.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.543+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3624246806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e300cb2e0 con 0x7f9e300d3ad0 2023-12-08T08:38:49.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.543+0000 7f9e17fff640 1 -- 172.21.15.16:0/3624246806 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e20013470 con 0x7f9e300d3ad0 2023-12-08T08:38:49.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.544+0000 7f9e17fff640 1 --2- 172.21.15.16:0/3624246806 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9e1c06ae40 0x7f9e1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.544+0000 7f9e17fff640 1 -- 172.21.15.16:0/3624246806 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9e20089a80 con 0x7f9e300d3ad0 2023-12-08T08:38:49.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.544+0000 7f9e2effd640 1 --2- 172.21.15.16:0/3624246806 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9e1c06ae40 0x7f9e1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.545+0000 7f9e2effd640 1 --2- 172.21.15.16:0/3624246806 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9e1c06ae40 0x7f9e1c06d300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9e30152de0 tx=0x7f9e24009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:49.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.548+0000 7f9e17fff640 1 -- 172.21.15.16:0/3624246806 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9e200548d0 con 0x7f9e300d3ad0 2023-12-08T08:38:49.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.761+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3624246806 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e300d09d0 con 0x7f9e300d3ad0 2023-12-08T08:38:49.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.761+0000 7f9e17fff640 1 -- 172.21.15.16:0/3624246806 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f9e20017090 con 0x7f9e300d3ad0 2023-12-08T08:38:49.764 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:49.764 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/307039451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":307039451},{"type":"v1","addr":"172.21.15.112:6836","nonce":307039451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1424152341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1424152341},{"type":"v1","addr":"172.21.15.16:6833","nonce":1424152341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3597811670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3597811670},{"type":"v1","addr":"172.21.15.16:6835","nonce":3597811670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-08T08:38:18.991090+0000","modified":"2023-12-08T08:38:41.075742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1082241707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1082241707},{"type":"v1","addr":"172.21.15.112:6837","nonce":1082241707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-08T08:38:49.765 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-12-08T08:38:49.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.763+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3624246806 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9e1c06ae40 msgr2=0x7f9e1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.763+0000 7f9e35e4f640 1 --2- 172.21.15.16:0/3624246806 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9e1c06ae40 0x7f9e1c06d300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9e30152de0 tx=0x7f9e24009040 comp rx=0 tx=0).stop 2023-12-08T08:38:49.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.763+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3624246806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e300d3ad0 msgr2=0x7f9e301f26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.763+0000 7f9e35e4f640 1 --2- 172.21.15.16:0/3624246806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e300d3ad0 0x7f9e301f26a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000e9b0 tx=0x7f9e2000ee70 comp rx=0 tx=0).stop 2023-12-08T08:38:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.763+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3624246806 shutdown_connections 2023-12-08T08:38:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.763+0000 7f9e35e4f640 1 --2- 172.21.15.16:0/3624246806 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9e1c06ae40 0x7f9e1c06d300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.763+0000 7f9e35e4f640 1 --2- 172.21.15.16:0/3624246806 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9e3015f890 0x7f9e300cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.763+0000 7f9e35e4f640 1 --2- 172.21.15.16:0/3624246806 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9e3015d580 0x7f9e300cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.763+0000 7f9e35e4f640 1 --2- 172.21.15.16:0/3624246806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e300d3ad0 0x7f9e301f26a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.763+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3624246806 >> 172.21.15.16:0/3624246806 conn(0x7f9e300d2fb0 msgr2=0x7f9e300d1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.763+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3624246806 shutdown_connections 2023-12-08T08:38:49.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.763+0000 7f9e35e4f640 1 -- 172.21.15.16:0/3624246806 wait complete. 2023-12-08T08:38:49.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:49.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.881+0000 7fa1edf68640 1 Processor -- start 2023-12-08T08:38:49.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.881+0000 7fa1edf68640 1 -- start start 2023-12-08T08:38:49.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1edf68640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1e80cf760 0x7fa1e806a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1edf68640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1e815e150 0x7fa1e8161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1edf68640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e806b070 con 0x7fa1e80cf380 2023-12-08T08:38:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1edf68640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e8072290 con 0x7fa1e815e150 2023-12-08T08:38:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1edf68640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80d0d00 con 0x7fa1e80cf760 2023-12-08T08:38:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1e7fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa1e815e150 0x7fa1e8161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1e77fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa1e80cf380 0x7fa1e8071b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37514/0 (socket says 172.21.15.16:37514) 2023-12-08T08:38:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1e77fe640 1 -- 172.21.15.16:0/402738298 learned_addr learned my addr 172.21.15.16:0/402738298 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1e6ffd640 1 --2- 172.21.15.16:0/402738298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1e80cf760 0x7fa1e806a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1e7fff640 1 -- 172.21.15.16:0/402738298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1e80cf760 msgr2=0x7fa1e806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1e7fff640 1 --2- 172.21.15.16:0/402738298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1e80cf760 0x7fa1e806a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1e7fff640 1 -- 172.21.15.16:0/402738298 >> v1:172.21.15.16:6789/0 conn(0x7fa1e80cf380 legacy=0x7fa1e8071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1e7fff640 1 -- 172.21.15.16:0/402738298 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e806aab0 con 0x7fa1e815e150 2023-12-08T08:38:49.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1e67fc640 1 -- 172.21.15.16:0/402738298 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472254276 0 0) 0x7fa1e806b070 con 0x7fa1e80cf380 2023-12-08T08:38:49.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.882+0000 7fa1e6ffd640 1 --2- 172.21.15.16:0/402738298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1e80cf760 0x7fa1e806a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:49.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.883+0000 7fa1e7fff640 1 --2- 172.21.15.16:0/402738298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e815e150 0x7fa1e8161690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009dc0 tx=0x7fa1e002be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e885d8e9b678d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:49.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.883+0000 7fa1e67fc640 1 -- 172.21.15.16:0/402738298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e002cbb0 con 0x7fa1e815e150 2023-12-08T08:38:49.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.883+0000 7fa1e67fc640 1 -- 172.21.15.16:0/402738298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e002cd10 con 0x7fa1e815e150 2023-12-08T08:38:49.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.883+0000 7fa1e67fc640 1 -- 172.21.15.16:0/402738298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0034660 con 0x7fa1e815e150 2023-12-08T08:38:49.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.884+0000 7fa1edf68640 1 -- 172.21.15.16:0/402738298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e815e150 msgr2=0x7fa1e8161690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.884+0000 7fa1edf68640 1 --2- 172.21.15.16:0/402738298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e815e150 0x7fa1e8161690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0009dc0 tx=0x7fa1e002be00 comp rx=0 tx=0).stop 2023-12-08T08:38:49.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.884+0000 7fa1edf68640 1 -- 172.21.15.16:0/402738298 shutdown_connections 2023-12-08T08:38:49.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.884+0000 7fa1edf68640 1 --2- 172.21.15.16:0/402738298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e815e150 0x7fa1e8161690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.884+0000 7fa1edf68640 1 --2- 172.21.15.16:0/402738298 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1e80cf760 0x7fa1e806a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.884+0000 7fa1edf68640 1 -- 172.21.15.16:0/402738298 >> 172.21.15.16:0/402738298 conn(0x7fa1e80cfda0 msgr2=0x7fa1e80d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.884+0000 7fa1edf68640 1 -- 172.21.15.16:0/402738298 shutdown_connections 2023-12-08T08:38:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.884+0000 7fa1edf68640 1 -- 172.21.15.16:0/402738298 wait complete. 2023-12-08T08:38:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.885+0000 7fa1edf68640 1 Processor -- start 2023-12-08T08:38:49.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.885+0000 7fa1edf68640 1 -- start start 2023-12-08T08:38:49.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1edf68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e80cf380 0x7fa1e815c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1edf68640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1e80cf760 0x7fa1e80ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1edf68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1e815e150 0x7fa1e80b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1edf68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8152180 con 0x7fa1e80cf380 2023-12-08T08:38:49.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1edf68640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e81602b0 con 0x7fa1e815e150 2023-12-08T08:38:49.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1edf68640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8160590 con 0x7fa1e80cf760 2023-12-08T08:38:49.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e80cf380 0x7fa1e815c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1e77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e80cf380 0x7fa1e815c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53610/0 (socket says 172.21.15.16:53610) 2023-12-08T08:38:49.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1e77fe640 1 -- 172.21.15.16:0/1379849944 learned_addr learned my addr 172.21.15.16:0/1379849944 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:49.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1e7fff640 1 --2- 172.21.15.16:0/1379849944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1e815e150 0x7fa1e80b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1e77fe640 1 -- 172.21.15.16:0/1379849944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1e80cf760 msgr2=0x7fa1e80ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1e6ffd640 1 --2- 172.21.15.16:0/1379849944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1e80cf760 0x7fa1e80ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1e77fe640 1 --2- 172.21.15.16:0/1379849944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1e80cf760 0x7fa1e80ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1e77fe640 1 -- 172.21.15.16:0/1379849944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1e815e150 msgr2=0x7fa1e80b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:49.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1e77fe640 1 --2- 172.21.15.16:0/1379849944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1e815e150 0x7fa1e80b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:49.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1e77fe640 1 -- 172.21.15.16:0/1379849944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e0009a70 con 0x7fa1e80cf380 2023-12-08T08:38:49.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.886+0000 7fa1e77fe640 1 --2- 172.21.15.16:0/1379849944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e80cf380 0x7fa1e815c7d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800c4d0 tx=0x7fa1d800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:49.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.887+0000 7fa1cffff640 1 -- 172.21.15.16:0/1379849944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d80074e0 con 0x7fa1e80cf380 2023-12-08T08:38:49.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.887+0000 7fa1cffff640 1 -- 172.21.15.16:0/1379849944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d8007660 con 0x7fa1e80cf380 2023-12-08T08:38:49.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.887+0000 7fa1edf68640 1 -- 172.21.15.16:0/1379849944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e8160870 con 0x7fa1e80cf380 2023-12-08T08:38:49.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.887+0000 7fa1cffff640 1 -- 172.21.15.16:0/1379849944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8004a20 con 0x7fa1e80cf380 2023-12-08T08:38:49.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.887+0000 7fa1edf68640 1 -- 172.21.15.16:0/1379849944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e80ca8c0 con 0x7fa1e80cf380 2023-12-08T08:38:49.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.888+0000 7fa1edf68640 1 -- 172.21.15.16:0/1379849944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e8067430 con 0x7fa1e80cf380 2023-12-08T08:38:49.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.888+0000 7fa1cffff640 1 -- 172.21.15.16:0/1379849944 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1d8004b80 con 0x7fa1e80cf380 2023-12-08T08:38:49.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.888+0000 7fa1cffff640 1 --2- 172.21.15.16:0/1379849944 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa1d406ae40 0x7fa1d406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:49.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.888+0000 7fa1cffff640 1 -- 172.21.15.16:0/1379849944 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa1d8088b10 con 0x7fa1e80cf380 2023-12-08T08:38:49.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.889+0000 7fa1e6ffd640 1 --2- 172.21.15.16:0/1379849944 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa1d406ae40 0x7fa1d406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:49.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.889+0000 7fa1e6ffd640 1 --2- 172.21.15.16:0/1379849944 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa1d406ae40 0x7fa1d406d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa1e815be60 tx=0x7fa1dc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:49.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:49.893+0000 7fa1cffff640 1 -- 172.21.15.16:0/1379849944 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1d8053850 con 0x7fa1e80cf380 2023-12-08T08:38:50.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.070+0000 7fa1edf68640 1 -- 172.21.15.16:0/1379849944 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1e81f5b90 con 0x7fa1e80cf380 2023-12-08T08:38:50.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.071+0000 7fa1cffff640 1 -- 172.21.15.16:0/1379849944 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fa1d8088020 con 0x7fa1e80cf380 2023-12-08T08:38:50.074 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:50.074 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:24.030625+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:50.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.073+0000 7fa1edf68640 1 -- 172.21.15.16:0/1379849944 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa1d406ae40 msgr2=0x7fa1d406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.073+0000 7fa1edf68640 1 --2- 172.21.15.16:0/1379849944 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa1d406ae40 0x7fa1d406d300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa1e815be60 tx=0x7fa1dc008040 comp rx=0 tx=0).stop 2023-12-08T08:38:50.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.073+0000 7fa1edf68640 1 -- 172.21.15.16:0/1379849944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e80cf380 msgr2=0x7fa1e815c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.073+0000 7fa1edf68640 1 --2- 172.21.15.16:0/1379849944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e80cf380 0x7fa1e815c7d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800c4d0 tx=0x7fa1d800c990 comp rx=0 tx=0).stop 2023-12-08T08:38:50.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.073+0000 7fa1edf68640 1 -- 172.21.15.16:0/1379849944 shutdown_connections 2023-12-08T08:38:50.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.073+0000 7fa1edf68640 1 --2- 172.21.15.16:0/1379849944 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa1d406ae40 0x7fa1d406d300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.073+0000 7fa1edf68640 1 --2- 172.21.15.16:0/1379849944 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa1e815e150 0x7fa1e80b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.073+0000 7fa1edf68640 1 --2- 172.21.15.16:0/1379849944 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa1e80cf760 0x7fa1e80ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.073+0000 7fa1edf68640 1 --2- 172.21.15.16:0/1379849944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa1e80cf380 0x7fa1e815c7d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.073+0000 7fa1edf68640 1 -- 172.21.15.16:0/1379849944 >> 172.21.15.16:0/1379849944 conn(0x7fa1e80cfda0 msgr2=0x7fa1e80d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:50.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.073+0000 7fa1edf68640 1 -- 172.21.15.16:0/1379849944 shutdown_connections 2023-12-08T08:38:50.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.073+0000 7fa1edf68640 1 -- 172.21.15.16:0/1379849944 wait complete. 2023-12-08T08:38:50.087 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:38:50.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:38:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.194+0000 7f520d31d640 1 Processor -- start 2023-12-08T08:38:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.194+0000 7f520d31d640 1 -- start start 2023-12-08T08:38:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.194+0000 7f520d31d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080b8950 0x7f52080ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.194+0000 7f520d31d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f52080cebc0 0x7f52080cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.194+0000 7f520d31d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f52080cfab0 0x7f52080d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.194+0000 7f520d31d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52080cff40 con 0x7f52080cebc0 2023-12-08T08:38:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.194+0000 7f520d31d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52080b5f00 con 0x7f52080cfab0 2023-12-08T08:38:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.194+0000 7f520d31d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52080b6170 con 0x7f52080b8950 2023-12-08T08:38:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.195+0000 7f51fffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f52080cebc0 0x7f52080cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.195+0000 7f51fffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f52080cebc0 0x7f52080cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53624/0 (socket says 172.21.15.16:53624) 2023-12-08T08:38:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.195+0000 7f51fffff640 1 -- 172.21.15.16:0/2190259340 learned_addr learned my addr 172.21.15.16:0/2190259340 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.195+0000 7f5206ffd640 1 --2- 172.21.15.16:0/2190259340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080b8950 0x7f52080ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.195+0000 7f52077fe640 1 --2- 172.21.15.16:0/2190259340 >> v2:172.21.15.112:3300/0 conn(0x7f52080cfab0 0x7f52080d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.195+0000 7f51fffff640 1 -- 172.21.15.16:0/2190259340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080b8950 msgr2=0x7f52080ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.195+0000 7f51fffff640 1 --2- 172.21.15.16:0/2190259340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080b8950 0x7f52080ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.195+0000 7f51fffff640 1 -- 172.21.15.16:0/2190259340 >> v2:172.21.15.112:3300/0 conn(0x7f52080cfab0 msgr2=0x7f52080d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.195+0000 7f51fffff640 1 --2- 172.21.15.16:0/2190259340 >> v2:172.21.15.112:3300/0 conn(0x7f52080cfab0 0x7f52080d0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.195+0000 7f51fffff640 1 -- 172.21.15.16:0/2190259340 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52080b6450 con 0x7f52080cebc0 2023-12-08T08:38:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.196+0000 7f51fffff640 1 --2- 172.21.15.16:0/2190259340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52080cebc0 0x7f52080cf570 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f520808c470 tx=0x7f51f002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5d3494a4dd4712 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.196+0000 7f52067fc640 1 -- 172.21.15.16:0/2190259340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f00317e0 con 0x7f52080cebc0 2023-12-08T08:38:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.196+0000 7f52067fc640 1 -- 172.21.15.16:0/2190259340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f0031940 con 0x7f52080cebc0 2023-12-08T08:38:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.196+0000 7f52067fc640 1 -- 172.21.15.16:0/2190259340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0031cd0 con 0x7f52080cebc0 2023-12-08T08:38:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.196+0000 7f520d31d640 1 -- 172.21.15.16:0/2190259340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52080cebc0 msgr2=0x7f52080cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.196+0000 7f520d31d640 1 --2- 172.21.15.16:0/2190259340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52080cebc0 0x7f52080cf570 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f520808c470 tx=0x7f51f002b880 comp rx=0 tx=0).stop 2023-12-08T08:38:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.196+0000 7f520d31d640 1 -- 172.21.15.16:0/2190259340 shutdown_connections 2023-12-08T08:38:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.196+0000 7f520d31d640 1 --2- 172.21.15.16:0/2190259340 >> v2:172.21.15.112:3300/0 conn(0x7f52080cfab0 0x7f52080d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.196+0000 7f520d31d640 1 --2- 172.21.15.16:0/2190259340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52080cebc0 0x7f52080cf570 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.196+0000 7f520d31d640 1 --2- 172.21.15.16:0/2190259340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080b8950 0x7f52080ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.196+0000 7f520d31d640 1 -- 172.21.15.16:0/2190259340 >> 172.21.15.16:0/2190259340 conn(0x7f52080d3010 msgr2=0x7f52080d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.197+0000 7f520d31d640 1 -- 172.21.15.16:0/2190259340 shutdown_connections 2023-12-08T08:38:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.197+0000 7f520d31d640 1 -- 172.21.15.16:0/2190259340 wait complete. 2023-12-08T08:38:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.198+0000 7f520d31d640 1 Processor -- start 2023-12-08T08:38:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.198+0000 7f520d31d640 1 -- start start 2023-12-08T08:38:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.198+0000 7f520d31d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080b8950 0x7f520815f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.198+0000 7f520d31d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52080cebc0 0x7f520809ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.198+0000 7f520d31d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52080cfab0 0x7f520806aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.198+0000 7f520d31d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52081f3150 con 0x7f52080cebc0 2023-12-08T08:38:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.198+0000 7f520d31d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52080a0a50 con 0x7f52080cfab0 2023-12-08T08:38:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.198+0000 7f520d31d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520815fc20 con 0x7f52080b8950 2023-12-08T08:38:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.198+0000 7f51fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52080cebc0 0x7f520809ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f51fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52080cebc0 0x7f520809ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53628/0 (socket says 172.21.15.16:53628) 2023-12-08T08:38:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f51fffff640 1 -- 172.21.15.16:0/2152086940 learned_addr learned my addr 172.21.15.16:0/2152086940 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f52077fe640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52080cfab0 0x7f520806aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f5206ffd640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080b8950 0x7f520815f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f51fffff640 1 -- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080b8950 msgr2=0x7f520815f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f51fffff640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080b8950 0x7f520815f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f51fffff640 1 -- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52080cfab0 msgr2=0x7f520806aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f51fffff640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52080cfab0 0x7f520806aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f51fffff640 1 -- 172.21.15.16:0/2152086940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f00094d0 con 0x7f52080cebc0 2023-12-08T08:38:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f5206ffd640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080b8950 0x7f520815f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f51fffff640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52080cebc0 0x7f520809ff20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f51f002b8c0 tx=0x7f51f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f51ff7fe640 1 -- 172.21.15.16:0/2152086940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0032880 con 0x7f52080cebc0 2023-12-08T08:38:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f51ff7fe640 1 -- 172.21.15.16:0/2152086940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f0032a00 con 0x7f52080cebc0 2023-12-08T08:38:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f520d31d640 1 -- 172.21.15.16:0/2152086940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520809eb60 con 0x7f52080cebc0 2023-12-08T08:38:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f51ff7fe640 1 -- 172.21.15.16:0/2152086940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0032db0 con 0x7f52080cebc0 2023-12-08T08:38:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.199+0000 7f520d31d640 1 -- 172.21.15.16:0/2152086940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520809e5a0 con 0x7f52080cebc0 2023-12-08T08:38:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.200+0000 7f520d31d640 1 -- 172.21.15.16:0/2152086940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f520808b150 con 0x7f52080cebc0 2023-12-08T08:38:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.201+0000 7f51ff7fe640 1 -- 172.21.15.16:0/2152086940 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f51f004e450 con 0x7f52080cebc0 2023-12-08T08:38:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.201+0000 7f51ff7fe640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f51e806ae40 0x7f51e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.201+0000 7f51ff7fe640 1 -- 172.21.15.16:0/2152086940 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f51f00ab840 con 0x7f52080cebc0 2023-12-08T08:38:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.201+0000 7f5206ffd640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f51e806ae40 0x7f51e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.202+0000 7f5206ffd640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f51e806ae40 0x7f51e806d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009a40 tx=0x7f51f8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.206+0000 7f51ff7fe640 1 -- 172.21.15.16:0/2152086940 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f51f0076600 con 0x7f52080cebc0 2023-12-08T08:38:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.380+0000 7f520d31d640 1 -- 172.21.15.16:0/2152086940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52081f3290 con 0x7f52080cebc0 2023-12-08T08:38:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.380+0000 7f51ff7fe640 1 -- 172.21.15.16:0/2152086940 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f52081f3290 con 0x7f52080cebc0 2023-12-08T08:38:50.383 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:38:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.382+0000 7f520d31d640 1 -- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f51e806ae40 msgr2=0x7f51e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.382+0000 7f520d31d640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f51e806ae40 0x7f51e806d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009a40 tx=0x7f51f8008040 comp rx=0 tx=0).stop 2023-12-08T08:38:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.382+0000 7f520d31d640 1 -- 172.21.15.16:0/2152086940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52080cebc0 msgr2=0x7f520809ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.382+0000 7f520d31d640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52080cebc0 0x7f520809ff20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f51f002b8c0 tx=0x7f51f00046c0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.382+0000 7f520d31d640 1 -- 172.21.15.16:0/2152086940 shutdown_connections 2023-12-08T08:38:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.382+0000 7f520d31d640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f51e806ae40 0x7f51e806d300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.382+0000 7f520d31d640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f52080cfab0 0x7f520806aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.382+0000 7f520d31d640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52080cebc0 0x7f520809ff20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.382+0000 7f520d31d640 1 --2- 172.21.15.16:0/2152086940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f52080b8950 0x7f520815f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.382+0000 7f520d31d640 1 -- 172.21.15.16:0/2152086940 >> 172.21.15.16:0/2152086940 conn(0x7f52080d3010 msgr2=0x7f52080ca520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.382+0000 7f520d31d640 1 -- 172.21.15.16:0/2152086940 shutdown_connections 2023-12-08T08:38:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.382+0000 7f520d31d640 1 -- 172.21.15.16:0/2152086940 wait complete. 2023-12-08T08:38:50.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:38:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.502+0000 7f881f373640 1 Processor -- start 2023-12-08T08:38:50.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.503+0000 7f881f373640 1 -- start start 2023-12-08T08:38:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.503+0000 7f881f373640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f881815e6a0 0x7f88180ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.503+0000 7f881f373640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88180d1950 0x7f88180cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.503+0000 7f881f373640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88180ce790 con 0x7f88180cc9f0 2023-12-08T08:38:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.503+0000 7f881f373640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88180ceba0 con 0x7f88180d1950 2023-12-08T08:38:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.503+0000 7f881f373640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88180cee80 con 0x7f881815e6a0 2023-12-08T08:38:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.503+0000 7f881c90e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88180d1950 0x7f88180cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.503+0000 7f881d910640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f88180cc9f0 0x7f88180b3c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37520/0 (socket says 172.21.15.16:37520) 2023-12-08T08:38:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.504+0000 7f881d910640 1 -- 172.21.15.16:0/1611458381 learned_addr learned my addr 172.21.15.16:0/1611458381 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.504+0000 7f881d10f640 1 --2- 172.21.15.16:0/1611458381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f881815e6a0 0x7f88180ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.504+0000 7f881c90e640 1 -- 172.21.15.16:0/1611458381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f881815e6a0 msgr2=0x7f88180ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:50.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.504+0000 7f881c90e640 1 --2- 172.21.15.16:0/1611458381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f881815e6a0 0x7f88180ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.504+0000 7f881c90e640 1 -- 172.21.15.16:0/1611458381 >> v1:172.21.15.16:6789/0 conn(0x7f88180cc9f0 legacy=0x7f88180b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.504+0000 7f881c90e640 1 -- 172.21.15.16:0/1611458381 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88180cbcb0 con 0x7f88180d1950 2023-12-08T08:38:50.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.505+0000 7f881c90e640 1 --2- 172.21.15.16:0/1611458381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88180d1950 0x7f88180cc4b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f880c0097a0 tx=0x7f880c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d773027f21892332 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:50.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.505+0000 7f8817fff640 1 -- 172.21.15.16:0/1611458381 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880c031830 con 0x7f88180d1950 2023-12-08T08:38:50.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.505+0000 7f8817fff640 1 -- 172.21.15.16:0/1611458381 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f880c031990 con 0x7f88180d1950 2023-12-08T08:38:50.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.505+0000 7f8817fff640 1 -- 172.21.15.16:0/1611458381 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f880c031d20 con 0x7f88180d1950 2023-12-08T08:38:50.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.505+0000 7f881f373640 1 -- 172.21.15.16:0/1611458381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88180d1950 msgr2=0x7f88180cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.505+0000 7f881f373640 1 --2- 172.21.15.16:0/1611458381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88180d1950 0x7f88180cc4b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f880c0097a0 tx=0x7f880c02b8d0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.505+0000 7f881f373640 1 -- 172.21.15.16:0/1611458381 shutdown_connections 2023-12-08T08:38:50.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.505+0000 7f881f373640 1 --2- 172.21.15.16:0/1611458381 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88180d1950 0x7f88180cc4b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.505+0000 7f881f373640 1 --2- 172.21.15.16:0/1611458381 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f881815e6a0 0x7f88180ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.505+0000 7f881f373640 1 -- 172.21.15.16:0/1611458381 >> 172.21.15.16:0/1611458381 conn(0x7f88180d0e30 msgr2=0x7f88180d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:50.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.505+0000 7f881f373640 1 -- 172.21.15.16:0/1611458381 shutdown_connections 2023-12-08T08:38:50.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.505+0000 7f881f373640 1 -- 172.21.15.16:0/1611458381 wait complete. 2023-12-08T08:38:50.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.506+0000 7f881f373640 1 Processor -- start 2023-12-08T08:38:50.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.506+0000 7f881f373640 1 -- start start 2023-12-08T08:38:50.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.507+0000 7f881f373640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88180cc9f0 0x7f8818154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.507+0000 7f881f373640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88180d1950 0x7f881806b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.507+0000 7f881f373640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f881815e6a0 0x7f881814ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.507+0000 7f881f373640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881806cb60 con 0x7f88180cc9f0 2023-12-08T08:38:50.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.507+0000 7f881f373640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881806bf50 con 0x7f88180d1950 2023-12-08T08:38:50.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.507+0000 7f881f373640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881806c230 con 0x7f881815e6a0 2023-12-08T08:38:50.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.507+0000 7f881d10f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88180cc9f0 0x7f8818154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.507+0000 7f881d10f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88180cc9f0 0x7f8818154840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53642/0 (socket says 172.21.15.16:53642) 2023-12-08T08:38:50.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.507+0000 7f881d10f640 1 -- 172.21.15.16:0/1434907700 learned_addr learned my addr 172.21.15.16:0/1434907700 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.507+0000 7f881c90e640 1 --2- 172.21.15.16:0/1434907700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88180d1950 0x7f881806b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.507+0000 7f881d910640 1 --2- 172.21.15.16:0/1434907700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f881815e6a0 0x7f881814ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.508+0000 7f881d10f640 1 -- 172.21.15.16:0/1434907700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f881815e6a0 msgr2=0x7f881814ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.508+0000 7f881d10f640 1 --2- 172.21.15.16:0/1434907700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f881815e6a0 0x7f881814ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.508+0000 7f881d10f640 1 -- 172.21.15.16:0/1434907700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88180d1950 msgr2=0x7f881806b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.508+0000 7f881d10f640 1 --2- 172.21.15.16:0/1434907700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88180d1950 0x7f881806b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.508+0000 7f881d10f640 1 -- 172.21.15.16:0/1434907700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c009400 con 0x7f88180cc9f0 2023-12-08T08:38:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.508+0000 7f881d10f640 1 --2- 172.21.15.16:0/1434907700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88180cc9f0 0x7f8818154840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f881000e9b0 tx=0x7f881000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.508+0000 7f8815ffb640 1 -- 172.21.15.16:0/1434907700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810009670 con 0x7f88180cc9f0 2023-12-08T08:38:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.508+0000 7f8815ffb640 1 -- 172.21.15.16:0/1434907700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881000c9a0 con 0x7f88180cc9f0 2023-12-08T08:38:50.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.508+0000 7f881f373640 1 -- 172.21.15.16:0/1434907700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881806c510 con 0x7f88180cc9f0 2023-12-08T08:38:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.508+0000 7f8815ffb640 1 -- 172.21.15.16:0/1434907700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881000cd50 con 0x7f88180cc9f0 2023-12-08T08:38:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.508+0000 7f881f373640 1 -- 172.21.15.16:0/1434907700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f881806cff0 con 0x7f88180cc9f0 2023-12-08T08:38:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.509+0000 7f881f373640 1 -- 172.21.15.16:0/1434907700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8818067ad0 con 0x7f88180cc9f0 2023-12-08T08:38:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.510+0000 7f8815ffb640 1 -- 172.21.15.16:0/1434907700 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8810013470 con 0x7f88180cc9f0 2023-12-08T08:38:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.510+0000 7f8815ffb640 1 --2- 172.21.15.16:0/1434907700 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f880406ae70 0x7f880406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.510+0000 7f8815ffb640 1 -- 172.21.15.16:0/1434907700 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8810089a60 con 0x7f88180cc9f0 2023-12-08T08:38:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.510+0000 7f881c90e640 1 --2- 172.21.15.16:0/1434907700 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f880406ae70 0x7f880406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.511+0000 7f881c90e640 1 --2- 172.21.15.16:0/1434907700 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f880406ae70 0x7f880406d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f880c036000 tx=0x7f880c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:50.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.514+0000 7f8815ffb640 1 -- 172.21.15.16:0/1434907700 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88100548d0 con 0x7f88180cc9f0 2023-12-08T08:38:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.691+0000 7f881f373640 1 -- 172.21.15.16:0/1434907700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f881806dbe0 con 0x7f88180cc9f0 2023-12-08T08:38:50.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.692+0000 7f8815ffb640 1 -- 172.21.15.16:0/1434907700 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8810017020 con 0x7f88180cc9f0 2023-12-08T08:38:50.695 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:38:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.694+0000 7f881f373640 1 -- 172.21.15.16:0/1434907700 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f880406ae70 msgr2=0x7f880406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.694+0000 7f881f373640 1 --2- 172.21.15.16:0/1434907700 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f880406ae70 0x7f880406d330 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f880c036000 tx=0x7f880c03c040 comp rx=0 tx=0).stop 2023-12-08T08:38:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.694+0000 7f881f373640 1 -- 172.21.15.16:0/1434907700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88180cc9f0 msgr2=0x7f8818154840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.694+0000 7f881f373640 1 --2- 172.21.15.16:0/1434907700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88180cc9f0 0x7f8818154840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f881000e9b0 tx=0x7f881000ee70 comp rx=0 tx=0).stop 2023-12-08T08:38:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.694+0000 7f881f373640 1 -- 172.21.15.16:0/1434907700 shutdown_connections 2023-12-08T08:38:50.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.694+0000 7f881f373640 1 --2- 172.21.15.16:0/1434907700 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f880406ae70 0x7f880406d330 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.694+0000 7f881f373640 1 --2- 172.21.15.16:0/1434907700 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f881815e6a0 0x7f881814ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.694+0000 7f881f373640 1 --2- 172.21.15.16:0/1434907700 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88180d1950 0x7f881806b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.694+0000 7f881f373640 1 --2- 172.21.15.16:0/1434907700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88180cc9f0 0x7f8818154840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.694+0000 7f881f373640 1 -- 172.21.15.16:0/1434907700 >> 172.21.15.16:0/1434907700 conn(0x7f88180d0e30 msgr2=0x7f88180cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.694+0000 7f881f373640 1 -- 172.21.15.16:0/1434907700 shutdown_connections 2023-12-08T08:38:50.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.694+0000 7f881f373640 1 -- 172.21.15.16:0/1434907700 wait complete. 2023-12-08T08:38:50.708 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:38:50.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.814+0000 7fedf530f640 1 Processor -- start 2023-12-08T08:38:50.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.814+0000 7fedf530f640 1 -- start start 2023-12-08T08:38:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.815+0000 7fedf530f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedf00caae0 0x7fedf0161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.815+0000 7fedf530f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf00caec0 0x7fedf015d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.815+0000 7fedf530f640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf015e600 con 0x7fedf015dbc0 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.815+0000 7fedf530f640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf015e040 con 0x7fedf00caae0 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.815+0000 7fedf530f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf015e320 con 0x7fedf00caec0 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.815+0000 7fedeeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedf00caae0 0x7fedf0161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.815+0000 7fedeeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedf00caae0 0x7fedf0161510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53646/0 (socket says 172.21.15.16:53646) 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.815+0000 7fedeeffd640 1 -- 172.21.15.16:0/1718889436 learned_addr learned my addr 172.21.15.16:0/1718889436 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.815+0000 7fedee7fc640 1 --2- 172.21.15.16:0/1718889436 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf00caec0 0x7fedf015d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.816+0000 7fedeeffd640 1 -- 172.21.15.16:0/1718889436 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf00caec0 msgr2=0x7fedf015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.816+0000 7fedeeffd640 1 --2- 172.21.15.16:0/1718889436 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf00caec0 0x7fedf015d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.816+0000 7fedeeffd640 1 -- 172.21.15.16:0/1718889436 >> v1:172.21.15.112:6789/0 conn(0x7fedf015dbc0 legacy=0x7fedf006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.816+0000 7fedeeffd640 1 -- 172.21.15.16:0/1718889436 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf0160d10 con 0x7fedf00caae0 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.816+0000 7fedeeffd640 1 --2- 172.21.15.16:0/1718889436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00caae0 0x7fedf0161510 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fede8009870 tx=0x7fede802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a55097b00a8ef6 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.817+0000 7fededffb640 1 -- 172.21.15.16:0/1718889436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8030840 con 0x7fedf00caae0 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.817+0000 7fededffb640 1 -- 172.21.15.16:0/1718889436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede80309a0 con 0x7fedf00caae0 2023-12-08T08:38:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.817+0000 7fededffb640 1 -- 172.21.15.16:0/1718889436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8030d30 con 0x7fedf00caae0 2023-12-08T08:38:50.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.817+0000 7fedf530f640 1 -- 172.21.15.16:0/1718889436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00caae0 msgr2=0x7fedf0161510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.817+0000 7fedf530f640 1 --2- 172.21.15.16:0/1718889436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00caae0 0x7fedf0161510 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fede8009870 tx=0x7fede802b980 comp rx=0 tx=0).stop 2023-12-08T08:38:50.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.817+0000 7fedf530f640 1 -- 172.21.15.16:0/1718889436 shutdown_connections 2023-12-08T08:38:50.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.817+0000 7fedf530f640 1 --2- 172.21.15.16:0/1718889436 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf00caec0 0x7fedf015d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.817+0000 7fedf530f640 1 --2- 172.21.15.16:0/1718889436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00caae0 0x7fedf0161510 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.817+0000 7fedf530f640 1 -- 172.21.15.16:0/1718889436 >> 172.21.15.16:0/1718889436 conn(0x7fedf00cfcd0 msgr2=0x7fedf00d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:50.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.817+0000 7fedf530f640 1 -- 172.21.15.16:0/1718889436 shutdown_connections 2023-12-08T08:38:50.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.817+0000 7fedf530f640 1 -- 172.21.15.16:0/1718889436 wait complete. 2023-12-08T08:38:50.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.818+0000 7fedf530f640 1 Processor -- start 2023-12-08T08:38:50.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.819+0000 7fedf530f640 1 -- start start 2023-12-08T08:38:50.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.819+0000 7fedf530f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf00caae0 0x7fedf01f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.819+0000 7fedf530f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00caec0 0x7fedf0156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.819+0000 7fedf530f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf015dbc0 0x7fedf01f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.819+0000 7fedf530f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf01f7b30 con 0x7fedf00caec0 2023-12-08T08:38:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.819+0000 7fedf530f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf01f6ba0 con 0x7fedf015dbc0 2023-12-08T08:38:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.819+0000 7fedf530f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf01f4100 con 0x7fedf00caae0 2023-12-08T08:38:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.819+0000 7fedee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00caec0 0x7fedf0156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.819+0000 7fedee7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00caec0 0x7fedf0156b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53652/0 (socket says 172.21.15.16:53652) 2023-12-08T08:38:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.819+0000 7fedee7fc640 1 -- 172.21.15.16:0/1803033682 learned_addr learned my addr 172.21.15.16:0/1803033682 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.819+0000 7fedef7fe640 1 --2- 172.21.15.16:0/1803033682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf015dbc0 0x7fedf01f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.819+0000 7fedeeffd640 1 --2- 172.21.15.16:0/1803033682 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf00caae0 0x7fedf01f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.820+0000 7fedee7fc640 1 -- 172.21.15.16:0/1803033682 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf00caae0 msgr2=0x7fedf01f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.820+0000 7fedee7fc640 1 --2- 172.21.15.16:0/1803033682 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf00caae0 0x7fedf01f75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.820+0000 7fedee7fc640 1 -- 172.21.15.16:0/1803033682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf015dbc0 msgr2=0x7fedf01f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.820+0000 7fedee7fc640 1 --2- 172.21.15.16:0/1803033682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf015dbc0 0x7fedf01f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.820+0000 7fedee7fc640 1 -- 172.21.15.16:0/1803033682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede80094d0 con 0x7fedf00caec0 2023-12-08T08:38:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.820+0000 7fedee7fc640 1 --2- 172.21.15.16:0/1803033682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00caec0 0x7fedf0156b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fedd800d800 tx=0x7fedd800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.820+0000 7fede77fe640 1 -- 172.21.15.16:0/1803033682 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd800b6c0 con 0x7fedf00caec0 2023-12-08T08:38:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.820+0000 7fede77fe640 1 -- 172.21.15.16:0/1803033682 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd800b820 con 0x7fedf00caec0 2023-12-08T08:38:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.820+0000 7fedf530f640 1 -- 172.21.15.16:0/1803033682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf0161bd0 con 0x7fedf00caec0 2023-12-08T08:38:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.820+0000 7fede77fe640 1 -- 172.21.15.16:0/1803033682 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd800bbd0 con 0x7fedf00caec0 2023-12-08T08:38:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.820+0000 7fedf530f640 1 -- 172.21.15.16:0/1803033682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf01f6df0 con 0x7fedf00caec0 2023-12-08T08:38:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.821+0000 7fedf530f640 1 -- 172.21.15.16:0/1803033682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedf00cbe50 con 0x7fedf00caec0 2023-12-08T08:38:50.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.822+0000 7fede77fe640 1 -- 172.21.15.16:0/1803033682 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fedd800bd30 con 0x7fedf00caec0 2023-12-08T08:38:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.822+0000 7fede77fe640 1 --2- 172.21.15.16:0/1803033682 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fedd006ae70 0x7fedd006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.822+0000 7fede77fe640 1 -- 172.21.15.16:0/1803033682 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fedd8089c70 con 0x7fedf00caec0 2023-12-08T08:38:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.822+0000 7fedeeffd640 1 --2- 172.21.15.16:0/1803033682 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fedd006ae70 0x7fedd006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:50.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.823+0000 7fedeeffd640 1 --2- 172.21.15.16:0/1803033682 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fedd006ae70 0x7fedd006d330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fede8036000 tx=0x7fede8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:50.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:50.826+0000 7fede77fe640 1 -- 172.21.15.16:0/1803033682 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fedd8054b70 con 0x7fedf00caec0 2023-12-08T08:38:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.001+0000 7fedf530f640 1 -- 172.21.15.16:0/1803033682 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fedf0067650 con 0x7fedf00caec0 2023-12-08T08:38:51.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.001+0000 7fede77fe640 1 -- 172.21.15.16:0/1803033682 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fedd8058190 con 0x7fedf00caec0 2023-12-08T08:38:51.005 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:38:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.003+0000 7fedf530f640 1 -- 172.21.15.16:0/1803033682 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fedd006ae70 msgr2=0x7fedd006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.003+0000 7fedf530f640 1 --2- 172.21.15.16:0/1803033682 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fedd006ae70 0x7fedd006d330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fede8036000 tx=0x7fede8008030 comp rx=0 tx=0).stop 2023-12-08T08:38:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.003+0000 7fedf530f640 1 -- 172.21.15.16:0/1803033682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00caec0 msgr2=0x7fedf0156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.003+0000 7fedf530f640 1 --2- 172.21.15.16:0/1803033682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00caec0 0x7fedf0156b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fedd800d800 tx=0x7fedd800dcc0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.003+0000 7fedf530f640 1 -- 172.21.15.16:0/1803033682 shutdown_connections 2023-12-08T08:38:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.003+0000 7fedf530f640 1 --2- 172.21.15.16:0/1803033682 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fedd006ae70 0x7fedd006d330 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.003+0000 7fedf530f640 1 --2- 172.21.15.16:0/1803033682 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fedf015dbc0 0x7fedf01f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.003+0000 7fedf530f640 1 --2- 172.21.15.16:0/1803033682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedf00caec0 0x7fedf0156b20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.003+0000 7fedf530f640 1 --2- 172.21.15.16:0/1803033682 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fedf00caae0 0x7fedf01f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.003+0000 7fedf530f640 1 -- 172.21.15.16:0/1803033682 >> 172.21.15.16:0/1803033682 conn(0x7fedf00cfcd0 msgr2=0x7fedf00aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.003+0000 7fedf530f640 1 -- 172.21.15.16:0/1803033682 shutdown_connections 2023-12-08T08:38:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.003+0000 7fedf530f640 1 -- 172.21.15.16:0/1803033682 wait complete. 2023-12-08T08:38:51.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:38:51.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.123+0000 7f02afa07640 1 Processor -- start 2023-12-08T08:38:51.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.123+0000 7f02afa07640 1 -- start start 2023-12-08T08:38:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.124+0000 7f02afa07640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f02a8160980 0x7f02a80cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.124+0000 7f02afa07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02a806c3b0 0x7f02a80b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.124+0000 7f02afa07640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a80b1260 con 0x7f02a806b6e0 2023-12-08T08:38:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.124+0000 7f02afa07640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a80d04f0 con 0x7f02a8160980 2023-12-08T08:38:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.124+0000 7f02afa07640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a80b0930 con 0x7f02a806c3b0 2023-12-08T08:38:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.125+0000 7f02adfa4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02a806c3b0 0x7f02a80b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.125+0000 7f02ad7a3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f02a8160980 0x7f02a80cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.125+0000 7f02adfa4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02a806c3b0 0x7f02a80b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:48880/0 (socket says 172.21.15.16:48880) 2023-12-08T08:38:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.125+0000 7f02acfa2640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f02a806b6e0 0x7f02a806bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:49550/0 (socket says 172.21.15.16:49550) 2023-12-08T08:38:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.125+0000 7f02adfa4640 1 -- 172.21.15.16:0/3606921994 learned_addr learned my addr 172.21.15.16:0/3606921994 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.125+0000 7f02adfa4640 1 -- 172.21.15.16:0/3606921994 >> v2:172.21.15.112:3300/0 conn(0x7f02a8160980 msgr2=0x7f02a80cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.125+0000 7f02adfa4640 1 --2- 172.21.15.16:0/3606921994 >> v2:172.21.15.112:3300/0 conn(0x7f02a8160980 0x7f02a80cfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.125+0000 7f02adfa4640 1 -- 172.21.15.16:0/3606921994 >> v1:172.21.15.112:6789/0 conn(0x7f02a806b6e0 legacy=0x7f02a806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.125+0000 7f02adfa4640 1 -- 172.21.15.16:0/3606921994 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a80b0c10 con 0x7f02a806c3b0 2023-12-08T08:38:51.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.125+0000 7f02ad7a3640 1 --2- 172.21.15.16:0/3606921994 >> v2:172.21.15.112:3300/0 conn(0x7f02a8160980 0x7f02a80cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:51.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.126+0000 7f02adfa4640 1 --2- 172.21.15.16:0/3606921994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02a806c3b0 0x7f02a80b16f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0298009e50 tx=0x7f029802bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d3f9c01e141a93 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.126+0000 7f02a7fff640 1 -- 172.21.15.16:0/3606921994 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029802cb90 con 0x7f02a806c3b0 2023-12-08T08:38:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.126+0000 7f02a7fff640 1 -- 172.21.15.16:0/3606921994 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029802ccf0 con 0x7f02a806c3b0 2023-12-08T08:38:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.126+0000 7f02a7fff640 1 -- 172.21.15.16:0/3606921994 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0298034610 con 0x7f02a806c3b0 2023-12-08T08:38:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.126+0000 7f02afa07640 1 -- 172.21.15.16:0/3606921994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02a806c3b0 msgr2=0x7f02a80b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.126+0000 7f02afa07640 1 --2- 172.21.15.16:0/3606921994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02a806c3b0 0x7f02a80b16f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0298009e50 tx=0x7f029802bff0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.126+0000 7f02afa07640 1 -- 172.21.15.16:0/3606921994 shutdown_connections 2023-12-08T08:38:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.126+0000 7f02afa07640 1 --2- 172.21.15.16:0/3606921994 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02a806c3b0 0x7f02a80b16f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.126+0000 7f02afa07640 1 --2- 172.21.15.16:0/3606921994 >> v2:172.21.15.112:3300/0 conn(0x7f02a8160980 0x7f02a80cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.126+0000 7f02afa07640 1 -- 172.21.15.16:0/3606921994 >> 172.21.15.16:0/3606921994 conn(0x7f02a80d0e30 msgr2=0x7f02a80d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:51.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.127+0000 7f02afa07640 1 -- 172.21.15.16:0/3606921994 shutdown_connections 2023-12-08T08:38:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.127+0000 7f02afa07640 1 -- 172.21.15.16:0/3606921994 wait complete. 2023-12-08T08:38:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.128+0000 7f02afa07640 1 Processor -- start 2023-12-08T08:38:51.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.128+0000 7f02afa07640 1 -- start start 2023-12-08T08:38:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.128+0000 7f02afa07640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02a806b6e0 0x7f02a80c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.128+0000 7f02afa07640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02a806c3b0 0x7f02a80b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02afa07640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a8160980 0x7f02a80b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02afa07640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a80c74b0 con 0x7f02a8160980 2023-12-08T08:38:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02afa07640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a814d120 con 0x7f02a806c3b0 2023-12-08T08:38:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02afa07640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a814d400 con 0x7f02a806b6e0 2023-12-08T08:38:51.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02adfa4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a8160980 0x7f02a80b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02adfa4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a8160980 0x7f02a80b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53662/0 (socket says 172.21.15.16:53662) 2023-12-08T08:38:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02adfa4640 1 -- 172.21.15.16:0/2452223905 learned_addr learned my addr 172.21.15.16:0/2452223905 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02ad7a3640 1 --2- 172.21.15.16:0/2452223905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02a806b6e0 0x7f02a80c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02acfa2640 1 --2- 172.21.15.16:0/2452223905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02a806c3b0 0x7f02a80b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02adfa4640 1 -- 172.21.15.16:0/2452223905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02a806b6e0 msgr2=0x7f02a80c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02adfa4640 1 --2- 172.21.15.16:0/2452223905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02a806b6e0 0x7f02a80c6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02adfa4640 1 -- 172.21.15.16:0/2452223905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02a806c3b0 msgr2=0x7f02a80b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02adfa4640 1 --2- 172.21.15.16:0/2452223905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02a806c3b0 0x7f02a80b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02adfa4640 1 -- 172.21.15.16:0/2452223905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a0009400 con 0x7f02a8160980 2023-12-08T08:38:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02adfa4640 1 --2- 172.21.15.16:0/2452223905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a8160980 0x7f02a80b1c30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0298009e50 tx=0x7f0298004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02a5ffb640 1 -- 172.21.15.16:0/2452223905 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0298009510 con 0x7f02a8160980 2023-12-08T08:38:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02a5ffb640 1 -- 172.21.15.16:0/2452223905 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029803e450 con 0x7f02a8160980 2023-12-08T08:38:51.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02afa07640 1 -- 172.21.15.16:0/2452223905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0298009b30 con 0x7f02a8160980 2023-12-08T08:38:51.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.129+0000 7f02a5ffb640 1 -- 172.21.15.16:0/2452223905 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029803e800 con 0x7f02a8160980 2023-12-08T08:38:51.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.130+0000 7f02afa07640 1 -- 172.21.15.16:0/2452223905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a814d9c0 con 0x7f02a8160980 2023-12-08T08:38:51.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.131+0000 7f02afa07640 1 -- 172.21.15.16:0/2452223905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a814dbb0 con 0x7f02a8160980 2023-12-08T08:38:51.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.131+0000 7f02a5ffb640 1 -- 172.21.15.16:0/2452223905 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f029803e960 con 0x7f02a8160980 2023-12-08T08:38:51.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.132+0000 7f02a5ffb640 1 --2- 172.21.15.16:0/2452223905 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f029406ae40 0x7f029406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.132+0000 7f02a5ffb640 1 -- 172.21.15.16:0/2452223905 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f02980ac310 con 0x7f02a8160980 2023-12-08T08:38:51.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.132+0000 7f02ad7a3640 1 --2- 172.21.15.16:0/2452223905 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f029406ae40 0x7f029406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.132+0000 7f02ad7a3640 1 --2- 172.21.15.16:0/2452223905 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f029406ae40 0x7f029406d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009c30 tx=0x7f02a0009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:51.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.136+0000 7f02a5ffb640 1 -- 172.21.15.16:0/2452223905 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0298077180 con 0x7f02a8160980 2023-12-08T08:38:51.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.310+0000 7f02afa07640 1 -- 172.21.15.16:0/2452223905 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f02a8067ad0 con 0x7f02a8160980 2023-12-08T08:38:51.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.311+0000 7f02a5ffb640 1 -- 172.21.15.16:0/2452223905 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f029802ce60 con 0x7f02a8160980 2023-12-08T08:38:51.314 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:38:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.312+0000 7f02afa07640 1 -- 172.21.15.16:0/2452223905 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f029406ae40 msgr2=0x7f029406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.312+0000 7f02afa07640 1 --2- 172.21.15.16:0/2452223905 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f029406ae40 0x7f029406d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009c30 tx=0x7f02a0009190 comp rx=0 tx=0).stop 2023-12-08T08:38:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.312+0000 7f02afa07640 1 -- 172.21.15.16:0/2452223905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a8160980 msgr2=0x7f02a80b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.312+0000 7f02afa07640 1 --2- 172.21.15.16:0/2452223905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a8160980 0x7f02a80b1c30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0298009e50 tx=0x7f0298004a40 comp rx=0 tx=0).stop 2023-12-08T08:38:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.313+0000 7f02afa07640 1 -- 172.21.15.16:0/2452223905 shutdown_connections 2023-12-08T08:38:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.313+0000 7f02afa07640 1 --2- 172.21.15.16:0/2452223905 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f029406ae40 0x7f029406d300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.313+0000 7f02afa07640 1 --2- 172.21.15.16:0/2452223905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02a8160980 0x7f02a80b1c30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.313+0000 7f02afa07640 1 --2- 172.21.15.16:0/2452223905 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f02a806c3b0 0x7f02a80b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.313+0000 7f02afa07640 1 --2- 172.21.15.16:0/2452223905 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f02a806b6e0 0x7f02a80c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.313+0000 7f02afa07640 1 -- 172.21.15.16:0/2452223905 >> 172.21.15.16:0/2452223905 conn(0x7f02a80d0e30 msgr2=0x7f02a815d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.313+0000 7f02afa07640 1 -- 172.21.15.16:0/2452223905 shutdown_connections 2023-12-08T08:38:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.313+0000 7f02afa07640 1 -- 172.21.15.16:0/2452223905 wait complete. 2023-12-08T08:38:51.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-08T08:38:51.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.430+0000 7f5cdb516640 1 Processor -- start 2023-12-08T08:38:51.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.430+0000 7f5cdb516640 1 -- start start 2023-12-08T08:38:51.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.431+0000 7f5cdb516640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5cd40d1d60 0x7f5cd40d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.431+0000 7f5cdb516640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd415ee90 con 0x7f5cd415f890 2023-12-08T08:38:51.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.431+0000 7f5cdb516640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cd40d0fe0 con 0x7f5cd415d580 2023-12-08T08:38:51.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.431+0000 7f5cdb516640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cd40d1280 con 0x7f5cd40d1d60 2023-12-08T08:38:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.431+0000 7f5cd8ab1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5cd40d1d60 0x7f5cd40d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.431+0000 7f5cd9ab3640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5cd415f890 0x7f5cd415d960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37536/0 (socket says 172.21.15.16:37536) 2023-12-08T08:38:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.431+0000 7f5cd9ab3640 1 -- 172.21.15.16:0/3069293583 learned_addr learned my addr 172.21.15.16:0/3069293583 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.431+0000 7f5cd8ab1640 1 -- 172.21.15.16:0/3069293583 >> v1:172.21.15.112:6789/0 conn(0x7f5cd415d580 legacy=0x7f5cd415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.431+0000 7f5cd8ab1640 1 -- 172.21.15.16:0/3069293583 >> v1:172.21.15.16:6789/0 conn(0x7f5cd415f890 legacy=0x7f5cd415d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.432+0000 7f5cd8ab1640 1 -- 172.21.15.16:0/3069293583 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd40d1560 con 0x7f5cd40d1d60 2023-12-08T08:38:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.432+0000 7f5cd8ab1640 1 --2- 172.21.15.16:0/3069293583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd40d1d60 0x7f5cd40d3ad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40097d0 tx=0x7f5cc402b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b26b89e89ded6 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.432+0000 7f5ccbfff640 1 -- 172.21.15.16:0/3069293583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4030710 con 0x7f5cd40d1d60 2023-12-08T08:38:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.432+0000 7f5ccbfff640 1 -- 172.21.15.16:0/3069293583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cc402e050 con 0x7f5cd40d1d60 2023-12-08T08:38:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.432+0000 7f5ccbfff640 1 -- 172.21.15.16:0/3069293583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4030a90 con 0x7f5cd40d1d60 2023-12-08T08:38:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.433+0000 7f5cdb516640 1 -- 172.21.15.16:0/3069293583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd40d1d60 msgr2=0x7f5cd40d3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.433+0000 7f5cdb516640 1 --2- 172.21.15.16:0/3069293583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd40d1d60 0x7f5cd40d3ad0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5cc40097d0 tx=0x7f5cc402b710 comp rx=0 tx=0).stop 2023-12-08T08:38:51.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.433+0000 7f5cdb516640 1 -- 172.21.15.16:0/3069293583 shutdown_connections 2023-12-08T08:38:51.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.433+0000 7f5cdb516640 1 --2- 172.21.15.16:0/3069293583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd40d1d60 0x7f5cd40d3ad0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.433+0000 7f5cdb516640 1 -- 172.21.15.16:0/3069293583 >> 172.21.15.16:0/3069293583 conn(0x7f5cd40d2fb0 msgr2=0x7f5cd40d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:51.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.433+0000 7f5cdb516640 1 -- 172.21.15.16:0/3069293583 shutdown_connections 2023-12-08T08:38:51.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.433+0000 7f5cdb516640 1 -- 172.21.15.16:0/3069293583 wait complete. 2023-12-08T08:38:51.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.434+0000 7f5cdb516640 1 Processor -- start 2023-12-08T08:38:51.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.434+0000 7f5cdb516640 1 -- start start 2023-12-08T08:38:51.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cdb516640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd40d1d60 0x7f5cd40b8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cdb516640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415d580 0x7f5cd40cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cdb516640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd415f890 0x7f5cd40ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cdb516640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40cb800 con 0x7f5cd40d1d60 2023-12-08T08:38:51.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cdb516640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd4151fa0 con 0x7f5cd415f890 2023-12-08T08:38:51.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cdb516640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd40cabf0 con 0x7f5cd415d580 2023-12-08T08:38:51.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cd92b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd40d1d60 0x7f5cd40b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cd92b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd40d1d60 0x7f5cd40b8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53674/0 (socket says 172.21.15.16:53674) 2023-12-08T08:38:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cd92b2640 1 -- 172.21.15.16:0/65068665 learned_addr learned my addr 172.21.15.16:0/65068665 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cd8ab1640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415d580 0x7f5cd40cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cd9ab3640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd415f890 0x7f5cd40ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cd92b2640 1 -- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415d580 msgr2=0x7f5cd40cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cd92b2640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415d580 0x7f5cd40cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cd92b2640 1 -- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd415f890 msgr2=0x7f5cd40ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cd92b2640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd415f890 0x7f5cd40ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cd92b2640 1 -- 172.21.15.16:0/65068665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc4009400 con 0x7f5cd40d1d60 2023-12-08T08:38:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.435+0000 7f5cd9ab3640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd415f890 0x7f5cd40ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.436+0000 7f5cd92b2640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd40d1d60 0x7f5cd40b8680 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00c480 tx=0x7f5ccc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.436+0000 7f5cc9ffb640 1 -- 172.21.15.16:0/65068665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0052d0 con 0x7f5cd40d1d60 2023-12-08T08:38:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.436+0000 7f5cc9ffb640 1 -- 172.21.15.16:0/65068665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc005430 con 0x7f5cd40d1d60 2023-12-08T08:38:51.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.436+0000 7f5cdb516640 1 -- 172.21.15.16:0/65068665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd40caed0 con 0x7f5cd40d1d60 2023-12-08T08:38:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.436+0000 7f5cc9ffb640 1 -- 172.21.15.16:0/65068665 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc012870 con 0x7f5cd40d1d60 2023-12-08T08:38:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.436+0000 7f5cdb516640 1 -- 172.21.15.16:0/65068665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd40cb490 con 0x7f5cd40d1d60 2023-12-08T08:38:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.437+0000 7f5cdb516640 1 -- 172.21.15.16:0/65068665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd40d09d0 con 0x7f5cd40d1d60 2023-12-08T08:38:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.437+0000 7f5cc9ffb640 1 -- 172.21.15.16:0/65068665 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ccc0129d0 con 0x7f5cd40d1d60 2023-12-08T08:38:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.438+0000 7f5cc9ffb640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5cb806ae40 0x7f5cb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.438+0000 7f5cc9ffb640 1 -- 172.21.15.16:0/65068665 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5ccc089370 con 0x7f5cd40d1d60 2023-12-08T08:38:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.438+0000 7f5cd8ab1640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5cb806ae40 0x7f5cb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.438+0000 7f5cd8ab1640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5cb806ae40 0x7f5cb806d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4034000 tx=0x7f5cc4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.442+0000 7f5cc9ffb640 1 -- 172.21.15.16:0/65068665 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ccc0541c0 con 0x7f5cd40d1d60 2023-12-08T08:38:51.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.656+0000 7f5cdb516640 1 -- 172.21.15.16:0/65068665 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5cd4067430 con 0x7f5cd40d1d60 2023-12-08T08:38:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.657+0000 7f5cc9ffb640 1 -- 172.21.15.16:0/65068665 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f5ccc0577e0 con 0x7f5cd40d1d60 2023-12-08T08:38:51.660 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:51.660 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/307039451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":307039451},{"type":"v1","addr":"172.21.15.112:6836","nonce":307039451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1424152341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1424152341},{"type":"v1","addr":"172.21.15.16:6833","nonce":1424152341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3597811670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3597811670},{"type":"v1","addr":"172.21.15.16:6835","nonce":3597811670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-08T08:38:18.991090+0000","modified":"2023-12-08T08:38:41.075742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1082241707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1082241707},{"type":"v1","addr":"172.21.15.112:6837","nonce":1082241707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-08T08:38:51.660 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-12-08T08:38:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.659+0000 7f5cdb516640 1 -- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5cb806ae40 msgr2=0x7f5cb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.659+0000 7f5cdb516640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5cb806ae40 0x7f5cb806d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4034000 tx=0x7f5cc4008030 comp rx=0 tx=0).stop 2023-12-08T08:38:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.659+0000 7f5cdb516640 1 -- 172.21.15.16:0/65068665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd40d1d60 msgr2=0x7f5cd40b8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.659+0000 7f5cdb516640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd40d1d60 0x7f5cd40b8680 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00c480 tx=0x7f5ccc00c940 comp rx=0 tx=0).stop 2023-12-08T08:38:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.659+0000 7f5cdb516640 1 -- 172.21.15.16:0/65068665 shutdown_connections 2023-12-08T08:38:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.659+0000 7f5cdb516640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5cb806ae40 0x7f5cb806d300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.659+0000 7f5cdb516640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5cd415f890 0x7f5cd40ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.659+0000 7f5cdb516640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5cd415d580 0x7f5cd40cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.659+0000 7f5cdb516640 1 --2- 172.21.15.16:0/65068665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5cd40d1d60 0x7f5cd40b8680 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.659+0000 7f5cdb516640 1 -- 172.21.15.16:0/65068665 >> 172.21.15.16:0/65068665 conn(0x7f5cd40d2fb0 msgr2=0x7f5cd406b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.659+0000 7f5cdb516640 1 -- 172.21.15.16:0/65068665 shutdown_connections 2023-12-08T08:38:51.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.659+0000 7f5cdb516640 1 -- 172.21.15.16:0/65068665 wait complete. 2023-12-08T08:38:51.672 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:38:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.776+0000 7fe79f0f3640 1 Processor -- start 2023-12-08T08:38:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.776+0000 7fe79f0f3640 1 -- start start 2023-12-08T08:38:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.777+0000 7fe79f0f3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe798158ee0 0x7fe798070e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.777+0000 7fe79f0f3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7980720d0 con 0x7fe7980cbd10 2023-12-08T08:38:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.777+0000 7fe79f0f3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7980cc890 con 0x7fe79815b750 2023-12-08T08:38:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.777+0000 7fe79f0f3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7980714c0 con 0x7fe798158ee0 2023-12-08T08:38:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.777+0000 7fe79d690640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe7980cbd10 0x7fe79806c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37548/0 (socket says 172.21.15.16:37548) 2023-12-08T08:38:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.777+0000 7fe79d690640 1 -- 172.21.15.16:0/4279029988 learned_addr learned my addr 172.21.15.16:0/4279029988 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.777+0000 7fe79ce8f640 1 --2- 172.21.15.16:0/4279029988 >> v2:172.21.15.112:3300/0 conn(0x7fe798158ee0 0x7fe798070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.777+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736066276 0 0) 0x7fe7980720d0 con 0x7fe7980cbd10 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.777+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe788003520 con 0x7fe7980cbd10 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226766822 0 0) 0x7fe7980cc890 con 0x7fe79815b750 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7980720d0 con 0x7fe79815b750 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 647237527 0 0) 0x7fe788003520 con 0x7fe7980cbd10 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 >> v2:172.21.15.112:3300/0 conn(0x7fe798158ee0 msgr2=0x7fe798070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 --2- 172.21.15.16:0/4279029988 >> v2:172.21.15.112:3300/0 conn(0x7fe798158ee0 0x7fe798070e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 >> v1:172.21.15.112:6789/0 conn(0x7fe79815b750 legacy=0x7fe7980cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7980717a0 con 0x7fe7980cbd10 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7fe7900019f0 con 0x7fe7980cbd10 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe79ce8f640 1 --2- 172.21.15.16:0/4279029988 >> v2:172.21.15.112:3300/0 conn(0x7fe798158ee0 0x7fe798070e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 >> v1:172.21.15.16:6789/0 conn(0x7fe7980cbd10 legacy=0x7fe79806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 --2- 172.21.15.16:0/4279029988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe788003b00 0x7fe788003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 --2- 172.21.15.16:0/4279029988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7880047c0 0x7fe788004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 --2- 172.21.15.16:0/4279029988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe788005180 0x7fe78800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980714c0 con 0x7fe788003b00 2023-12-08T08:38:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78800c0b0 con 0x7fe788005180 2023-12-08T08:38:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe78800c270 con 0x7fe7880047c0 2023-12-08T08:38:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe79ce8f640 1 --2- 172.21.15.16:0/4279029988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe788003b00 0x7fe788003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe797fff640 1 --2- 172.21.15.16:0/4279029988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7880047c0 0x7fe788004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe79d690640 1 --2- 172.21.15.16:0/4279029988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe788005180 0x7fe78800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe79ce8f640 1 -- 172.21.15.16:0/4279029988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7880047c0 msgr2=0x7fe788004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe79ce8f640 1 --2- 172.21.15.16:0/4279029988 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7880047c0 0x7fe788004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe79ce8f640 1 -- 172.21.15.16:0/4279029988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe788005180 msgr2=0x7fe78800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe79ce8f640 1 --2- 172.21.15.16:0/4279029988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe788005180 0x7fe78800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe79ce8f640 1 -- 172.21.15.16:0/4279029988 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe788003520 con 0x7fe788003b00 2023-12-08T08:38:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe79d690640 1 --2- 172.21.15.16:0/4279029988 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe788005180 0x7fe78800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.778+0000 7fe79d690640 1 -- 172.21.15.16:0/4279029988 reap_dead start 2023-12-08T08:38:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.779+0000 7fe79ce8f640 1 --2- 172.21.15.16:0/4279029988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe788003b00 0x7fe788003f00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009c90 tx=0x7fe7880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76285d4dcf6ba3fe server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.779+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c02c8f0 con 0x7fe788003b00 2023-12-08T08:38:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.779+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c02ca50 con 0x7fe788003b00 2023-12-08T08:38:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.779+0000 7fe7977fe640 1 -- 172.21.15.16:0/4279029988 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c02ce40 con 0x7fe788003b00 2023-12-08T08:38:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.780+0000 7fe79f0f3640 1 -- 172.21.15.16:0/4279029988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe788003b00 msgr2=0x7fe788003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.780+0000 7fe79f0f3640 1 --2- 172.21.15.16:0/4279029988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe788003b00 0x7fe788003f00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009c90 tx=0x7fe7880057f0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.780+0000 7fe79f0f3640 1 -- 172.21.15.16:0/4279029988 shutdown_connections 2023-12-08T08:38:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.780+0000 7fe79f0f3640 1 --2- 172.21.15.16:0/4279029988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe788003b00 0x7fe788003f00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.780+0000 7fe79f0f3640 1 -- 172.21.15.16:0/4279029988 >> 172.21.15.16:0/4279029988 conn(0x7fe7980b09f0 msgr2=0x7fe7980b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.780+0000 7fe79f0f3640 1 -- 172.21.15.16:0/4279029988 shutdown_connections 2023-12-08T08:38:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.780+0000 7fe79f0f3640 1 -- 172.21.15.16:0/4279029988 wait complete. 2023-12-08T08:38:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.781+0000 7fe79f0f3640 1 Processor -- start 2023-12-08T08:38:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.781+0000 7fe79f0f3640 1 -- start start 2023-12-08T08:38:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe79f0f3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe788003b00 0x7fe7980b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe79f0f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7980cf640 0x7fe7981495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe79f0f3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7980ccd80 0x7fe79815a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe79f0f3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980ce030 con 0x7fe7980cf640 2023-12-08T08:38:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe79f0f3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe798149b00 con 0x7fe788003b00 2023-12-08T08:38:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe79f0f3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980cfac0 con 0x7fe7980ccd80 2023-12-08T08:38:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe797fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7980cf640 0x7fe7981495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe797fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7980cf640 0x7fe7981495c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53700/0 (socket says 172.21.15.16:53700) 2023-12-08T08:38:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe797fff640 1 -- 172.21.15.16:0/3825134276 learned_addr learned my addr 172.21.15.16:0/3825134276 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe79d690640 1 --2- 172.21.15.16:0/3825134276 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7980ccd80 0x7fe79815a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe79ce8f640 1 --2- 172.21.15.16:0/3825134276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe788003b00 0x7fe7980b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe797fff640 1 -- 172.21.15.16:0/3825134276 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7980ccd80 msgr2=0x7fe79815a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe797fff640 1 --2- 172.21.15.16:0/3825134276 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7980ccd80 0x7fe79815a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe797fff640 1 -- 172.21.15.16:0/3825134276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe788003b00 msgr2=0x7fe7980b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe797fff640 1 --2- 172.21.15.16:0/3825134276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe788003b00 0x7fe7980b3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe797fff640 1 -- 172.21.15.16:0/3825134276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7980b3c20 con 0x7fe7980cf640 2023-12-08T08:38:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe797fff640 1 --2- 172.21.15.16:0/3825134276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7980cf640 0x7fe7981495c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe78400f7a0 tx=0x7fe78400fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.782+0000 7fe7957fa640 1 -- 172.21.15.16:0/3825134276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78400fe50 con 0x7fe7980cf640 2023-12-08T08:38:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.783+0000 7fe7957fa640 1 -- 172.21.15.16:0/3825134276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7840052e0 con 0x7fe7980cf640 2023-12-08T08:38:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.783+0000 7fe79f0f3640 1 -- 172.21.15.16:0/3825134276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7980ceb60 con 0x7fe7980cf640 2023-12-08T08:38:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.783+0000 7fe7957fa640 1 -- 172.21.15.16:0/3825134276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78400d970 con 0x7fe7980cf640 2023-12-08T08:38:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.783+0000 7fe79f0f3640 1 -- 172.21.15.16:0/3825134276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7980cf120 con 0x7fe7980cf640 2023-12-08T08:38:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.784+0000 7fe79f0f3640 1 -- 172.21.15.16:0/3825134276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7980cf340 con 0x7fe7980cf640 2023-12-08T08:38:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.784+0000 7fe7957fa640 1 -- 172.21.15.16:0/3825134276 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe78400dad0 con 0x7fe7980cf640 2023-12-08T08:38:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.784+0000 7fe7957fa640 1 --2- 172.21.15.16:0/3825134276 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe77c06ae70 0x7fe77c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.785+0000 7fe7957fa640 1 -- 172.21.15.16:0/3825134276 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe7840898e0 con 0x7fe7980cf640 2023-12-08T08:38:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.785+0000 7fe79ce8f640 1 --2- 172.21.15.16:0/3825134276 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe77c06ae70 0x7fe77c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.785+0000 7fe79ce8f640 1 --2- 172.21.15.16:0/3825134276 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe77c06ae70 0x7fe77c06d330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe78c036000 tx=0x7fe78c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.789+0000 7fe7957fa640 1 -- 172.21.15.16:0/3825134276 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe784054730 con 0x7fe7980cf640 2023-12-08T08:38:51.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.965+0000 7fe79f0f3640 1 -- 172.21.15.16:0/3825134276 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe798067430 con 0x7fe7980cf640 2023-12-08T08:38:51.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.966+0000 7fe7957fa640 1 -- 172.21.15.16:0/3825134276 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fe784057d50 con 0x7fe7980cf640 2023-12-08T08:38:51.970 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:51.970 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":24,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:38:24.030625+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:38:51.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.968+0000 7fe79f0f3640 1 -- 172.21.15.16:0/3825134276 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe77c06ae70 msgr2=0x7fe77c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.968+0000 7fe79f0f3640 1 --2- 172.21.15.16:0/3825134276 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe77c06ae70 0x7fe77c06d330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe78c036000 tx=0x7fe78c030000 comp rx=0 tx=0).stop 2023-12-08T08:38:51.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.968+0000 7fe79f0f3640 1 -- 172.21.15.16:0/3825134276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7980cf640 msgr2=0x7fe7981495c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:51.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.968+0000 7fe79f0f3640 1 --2- 172.21.15.16:0/3825134276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7980cf640 0x7fe7981495c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe78400f7a0 tx=0x7fe78400fc60 comp rx=0 tx=0).stop 2023-12-08T08:38:51.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.968+0000 7fe79f0f3640 1 -- 172.21.15.16:0/3825134276 shutdown_connections 2023-12-08T08:38:51.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.968+0000 7fe79f0f3640 1 --2- 172.21.15.16:0/3825134276 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe77c06ae70 0x7fe77c06d330 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.968+0000 7fe79f0f3640 1 --2- 172.21.15.16:0/3825134276 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe7980ccd80 0x7fe79815a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.968+0000 7fe79f0f3640 1 --2- 172.21.15.16:0/3825134276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7980cf640 0x7fe7981495c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.968+0000 7fe79f0f3640 1 --2- 172.21.15.16:0/3825134276 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe788003b00 0x7fe7980b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:51.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.968+0000 7fe79f0f3640 1 -- 172.21.15.16:0/3825134276 >> 172.21.15.16:0/3825134276 conn(0x7fe7980b09f0 msgr2=0x7fe79806c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:51.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.968+0000 7fe79f0f3640 1 -- 172.21.15.16:0/3825134276 shutdown_connections 2023-12-08T08:38:51.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:51.968+0000 7fe79f0f3640 1 -- 172.21.15.16:0/3825134276 wait complete. 2023-12-08T08:38:51.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-08T08:38:52.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.087+0000 7f87b9b43640 1 Processor -- start 2023-12-08T08:38:52.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.087+0000 7f87b9b43640 1 -- start start 2023-12-08T08:38:52.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.087+0000 7f87b9b43640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b40b0cf0 0x7f87b40d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.087+0000 7f87b9b43640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87b40b1d70 0x7f87b40cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.087+0000 7f87b9b43640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b40d20b0 con 0x7f87b41608a0 2023-12-08T08:38:52.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.087+0000 7f87b9b43640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b40d0620 con 0x7f87b40b1d70 2023-12-08T08:38:52.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.087+0000 7f87b9b43640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40d1b80 con 0x7f87b40b0cf0 2023-12-08T08:38:52.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.088+0000 7f87b3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87b40b1d70 0x7f87b40cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.088+0000 7f87b3fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87b40b1d70 0x7f87b40cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53704/0 (socket says 172.21.15.16:53704) 2023-12-08T08:38:52.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.088+0000 7f87b3fff640 1 -- 172.21.15.16:0/788220654 learned_addr learned my addr 172.21.15.16:0/788220654 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:52.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.088+0000 7f87b2ffd640 1 --2- 172.21.15.16:0/788220654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b40b0cf0 0x7f87b40d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.088+0000 7f87b3fff640 1 -- 172.21.15.16:0/788220654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b40b0cf0 msgr2=0x7f87b40d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:52.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.088+0000 7f87b3fff640 1 --2- 172.21.15.16:0/788220654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b40b0cf0 0x7f87b40d1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.088+0000 7f87b3fff640 1 -- 172.21.15.16:0/788220654 >> v1:172.21.15.112:6789/0 conn(0x7f87b41608a0 legacy=0x7f87b40d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.088+0000 7f87b3fff640 1 -- 172.21.15.16:0/788220654 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b40d1dd0 con 0x7f87b40b1d70 2023-12-08T08:38:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.089+0000 7f87b3fff640 1 --2- 172.21.15.16:0/788220654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40b1d70 0x7f87b40cdc40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009e50 tx=0x7f87ac02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5577ee8a13bcf028 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.089+0000 7f87b27fc640 1 -- 172.21.15.16:0/788220654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac02ca70 con 0x7f87b40b1d70 2023-12-08T08:38:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.089+0000 7f87b27fc640 1 -- 172.21.15.16:0/788220654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac02cbd0 con 0x7f87b40b1d70 2023-12-08T08:38:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.089+0000 7f87b27fc640 1 -- 172.21.15.16:0/788220654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac0343f0 con 0x7f87b40b1d70 2023-12-08T08:38:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.089+0000 7f87b9b43640 1 -- 172.21.15.16:0/788220654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40b1d70 msgr2=0x7f87b40cdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.089+0000 7f87b9b43640 1 --2- 172.21.15.16:0/788220654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40b1d70 0x7f87b40cdc40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f87ac009e50 tx=0x7f87ac02bed0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.089+0000 7f87b9b43640 1 -- 172.21.15.16:0/788220654 shutdown_connections 2023-12-08T08:38:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.089+0000 7f87b9b43640 1 --2- 172.21.15.16:0/788220654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40b1d70 0x7f87b40cdc40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.089+0000 7f87b9b43640 1 --2- 172.21.15.16:0/788220654 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b40b0cf0 0x7f87b40d1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.089+0000 7f87b9b43640 1 -- 172.21.15.16:0/788220654 >> 172.21.15.16:0/788220654 conn(0x7f87b40d3030 msgr2=0x7f87b40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.089+0000 7f87b9b43640 1 -- 172.21.15.16:0/788220654 shutdown_connections 2023-12-08T08:38:52.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.090+0000 7f87b9b43640 1 -- 172.21.15.16:0/788220654 wait complete. 2023-12-08T08:38:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.091+0000 7f87b9b43640 1 Processor -- start 2023-12-08T08:38:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.091+0000 7f87b9b43640 1 -- start start 2023-12-08T08:38:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.091+0000 7f87b9b43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40b0cf0 0x7f87b40b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.091+0000 7f87b9b43640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b40b1d70 0x7f87b40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b9b43640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b41608a0 0x7f87b40c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b9b43640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40b8e90 con 0x7f87b40b0cf0 2023-12-08T08:38:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b9b43640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40c7a00 con 0x7f87b41608a0 2023-12-08T08:38:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b9b43640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b40c9790 con 0x7f87b40b1d70 2023-12-08T08:38:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40b0cf0 0x7f87b40b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40b0cf0 0x7f87b40b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53716/0 (socket says 172.21.15.16:53716) 2023-12-08T08:38:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b37fe640 1 -- 172.21.15.16:0/1439665707 learned_addr learned my addr 172.21.15.16:0/1439665707 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b2ffd640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b40b1d70 0x7f87b40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b3fff640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b41608a0 0x7f87b40c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b37fe640 1 -- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b40b1d70 msgr2=0x7f87b40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b37fe640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b40b1d70 0x7f87b40b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b37fe640 1 -- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b41608a0 msgr2=0x7f87b40c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b37fe640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b41608a0 0x7f87b40c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b37fe640 1 -- 172.21.15.16:0/1439665707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac009b30 con 0x7f87b40b0cf0 2023-12-08T08:38:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b3fff640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b41608a0 0x7f87b40c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b37fe640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40b0cf0 0x7f87b40b3de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c400 tx=0x7f87a400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f879bfff640 1 -- 172.21.15.16:0/1439665707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a40052d0 con 0x7f87b40b0cf0 2023-12-08T08:38:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f879bfff640 1 -- 172.21.15.16:0/1439665707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a4005430 con 0x7f87b40b0cf0 2023-12-08T08:38:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f879bfff640 1 -- 172.21.15.16:0/1439665707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4003830 con 0x7f87b40b0cf0 2023-12-08T08:38:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b9b43640 1 -- 172.21.15.16:0/1439665707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b40c6c40 con 0x7f87b40b0cf0 2023-12-08T08:38:52.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.092+0000 7f87b9b43640 1 -- 172.21.15.16:0/1439665707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b40c7200 con 0x7f87b40b0cf0 2023-12-08T08:38:52.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.093+0000 7f87b9b43640 1 -- 172.21.15.16:0/1439665707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b4067430 con 0x7f87b40b0cf0 2023-12-08T08:38:52.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.094+0000 7f879bfff640 1 -- 172.21.15.16:0/1439665707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f87a4003990 con 0x7f87b40b0cf0 2023-12-08T08:38:52.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.094+0000 7f879bfff640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f87a006ae40 0x7f87a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.094+0000 7f879bfff640 1 -- 172.21.15.16:0/1439665707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f87a4088dc0 con 0x7f87b40b0cf0 2023-12-08T08:38:52.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.094+0000 7f87b2ffd640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f87a006ae40 0x7f87a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.095+0000 7f87b2ffd640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f87a006ae40 0x7f87a006d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f87b40cf0e0 tx=0x7f87a8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:52.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.098+0000 7f879bfff640 1 -- 172.21.15.16:0/1439665707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f87a4053c40 con 0x7f87b40b0cf0 2023-12-08T08:38:52.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.314+0000 7f87b9b43640 1 -- 172.21.15.16:0/1439665707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87b415e310 con 0x7f87b40b0cf0 2023-12-08T08:38:52.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.314+0000 7f879bfff640 1 -- 172.21.15.16:0/1439665707 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f87a4057260 con 0x7f87b40b0cf0 2023-12-08T08:38:52.318 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:38:52.318 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/307039451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":307039451},{"type":"v1","addr":"172.21.15.112:6836","nonce":307039451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1424152341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1424152341},{"type":"v1","addr":"172.21.15.16:6833","nonce":1424152341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3597811670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3597811670},{"type":"v1","addr":"172.21.15.16:6835","nonce":3597811670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-08T08:38:18.991090+0000","modified":"2023-12-08T08:38:41.075742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1082241707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1082241707},{"type":"v1","addr":"172.21.15.112:6837","nonce":1082241707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-08T08:38:52.318 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-12-08T08:38:52.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.316+0000 7f87b9b43640 1 -- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f87a006ae40 msgr2=0x7f87a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.316+0000 7f87b9b43640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f87a006ae40 0x7f87a006d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f87b40cf0e0 tx=0x7f87a8009190 comp rx=0 tx=0).stop 2023-12-08T08:38:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.316+0000 7f87b9b43640 1 -- 172.21.15.16:0/1439665707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40b0cf0 msgr2=0x7f87b40b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.316+0000 7f87b9b43640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40b0cf0 0x7f87b40b3de0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c400 tx=0x7f87a400c8c0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.316+0000 7f87b9b43640 1 -- 172.21.15.16:0/1439665707 shutdown_connections 2023-12-08T08:38:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.316+0000 7f87b9b43640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f87a006ae40 0x7f87a006d300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.316+0000 7f87b9b43640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f87b41608a0 0x7f87b40c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.316+0000 7f87b9b43640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f87b40b1d70 0x7f87b40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.316+0000 7f87b9b43640 1 --2- 172.21.15.16:0/1439665707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87b40b0cf0 0x7f87b40b3de0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.316+0000 7f87b9b43640 1 -- 172.21.15.16:0/1439665707 >> 172.21.15.16:0/1439665707 conn(0x7f87b40d3030 msgr2=0x7f87b415d620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.316+0000 7f87b9b43640 1 -- 172.21.15.16:0/1439665707 shutdown_connections 2023-12-08T08:38:52.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.316+0000 7f87b9b43640 1 -- 172.21.15.16:0/1439665707 wait complete. 2023-12-08T08:38:52.733 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-08T08:38:18.991090+0000', 'modified': '2023-12-08T08:38:41.075742+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4297}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4297': {'gid': 4297, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6837/1082241707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6835', 'nonce': 1082241707}, {'type': 'v1', 'addr': '172.21.15.112:6837', 'nonce': 1082241707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-08T08:38:52.733 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-08T08:38:52.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-08T08:38:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.846+0000 7f9cfb09b640 1 Processor -- start 2023-12-08T08:38:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.846+0000 7f9cfb09b640 1 -- start start 2023-12-08T08:38:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.847+0000 7f9cfb09b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf415d580 0x7f9cf40d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.847+0000 7f9cfb09b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9cf415f890 0x7f9cf415d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.847+0000 7f9cfb09b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf415ee90 con 0x7f9cf40d3ad0 2023-12-08T08:38:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.847+0000 7f9cfb09b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf40d0fe0 con 0x7f9cf415f890 2023-12-08T08:38:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.847+0000 7f9cfb09b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40d1280 con 0x7f9cf415d580 2023-12-08T08:38:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.847+0000 7f9cf9638640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9cf415f890 0x7f9cf415d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.847+0000 7f9cf3fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9cf40d3ad0 0x7f9cf415f180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37564/0 (socket says 172.21.15.16:37564) 2023-12-08T08:38:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.847+0000 7f9cf3fff640 1 -- 172.21.15.16:0/2102580844 learned_addr learned my addr 172.21.15.16:0/2102580844 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.847+0000 7f9cf8e37640 1 --2- 172.21.15.16:0/2102580844 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf415d580 0x7f9cf40d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.847+0000 7f9cf9638640 1 -- 172.21.15.16:0/2102580844 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf415d580 msgr2=0x7f9cf40d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.847+0000 7f9cf9638640 1 --2- 172.21.15.16:0/2102580844 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf415d580 0x7f9cf40d1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.847+0000 7f9cf9638640 1 -- 172.21.15.16:0/2102580844 >> v1:172.21.15.16:6789/0 conn(0x7f9cf40d3ad0 legacy=0x7f9cf415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.847+0000 7f9cf9638640 1 -- 172.21.15.16:0/2102580844 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf40d1560 con 0x7f9cf415f890 2023-12-08T08:38:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.848+0000 7f9cf9638640 1 --2- 172.21.15.16:0/2102580844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415f890 0x7f9cf415d960 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009d90 tx=0x7f9cec02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1664a290f8478989 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.848+0000 7f9cf37fe640 1 -- 172.21.15.16:0/2102580844 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec02cbc0 con 0x7f9cf415f890 2023-12-08T08:38:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.848+0000 7f9cf37fe640 1 -- 172.21.15.16:0/2102580844 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec02cd20 con 0x7f9cf415f890 2023-12-08T08:38:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.848+0000 7f9cf37fe640 1 -- 172.21.15.16:0/2102580844 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec034660 con 0x7f9cf415f890 2023-12-08T08:38:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.849+0000 7f9cfb09b640 1 -- 172.21.15.16:0/2102580844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415f890 msgr2=0x7f9cf415d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.849+0000 7f9cfb09b640 1 --2- 172.21.15.16:0/2102580844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415f890 0x7f9cf415d960 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009d90 tx=0x7f9cec02be10 comp rx=0 tx=0).stop 2023-12-08T08:38:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.849+0000 7f9cfb09b640 1 -- 172.21.15.16:0/2102580844 shutdown_connections 2023-12-08T08:38:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.849+0000 7f9cfb09b640 1 --2- 172.21.15.16:0/2102580844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415f890 0x7f9cf415d960 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.849+0000 7f9cfb09b640 1 --2- 172.21.15.16:0/2102580844 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf415d580 0x7f9cf40d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.849+0000 7f9cfb09b640 1 -- 172.21.15.16:0/2102580844 >> 172.21.15.16:0/2102580844 conn(0x7f9cf40d2fb0 msgr2=0x7f9cf40d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.849+0000 7f9cfb09b640 1 -- 172.21.15.16:0/2102580844 shutdown_connections 2023-12-08T08:38:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.849+0000 7f9cfb09b640 1 -- 172.21.15.16:0/2102580844 wait complete. 2023-12-08T08:38:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.850+0000 7f9cfb09b640 1 Processor -- start 2023-12-08T08:38:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.850+0000 7f9cfb09b640 1 -- start start 2023-12-08T08:38:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cfb09b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf40d3ad0 0x7f9cf40b7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cfb09b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf415d580 0x7f9cf40b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cfb09b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415f890 0x7f9cf40b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cfb09b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf41f5f50 con 0x7f9cf415f890 2023-12-08T08:38:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cfb09b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40b7ed0 con 0x7f9cf415d580 2023-12-08T08:38:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cfb09b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf40b8b40 con 0x7f9cf40d3ad0 2023-12-08T08:38:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cf9638640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415f890 0x7f9cf40b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cf9638640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415f890 0x7f9cf40b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53738/0 (socket says 172.21.15.16:53738) 2023-12-08T08:38:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cf9638640 1 -- 172.21.15.16:0/3731564360 learned_addr learned my addr 172.21.15.16:0/3731564360 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cf8e37640 1 --2- 172.21.15.16:0/3731564360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf40d3ad0 0x7f9cf40b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/3731564360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf415d580 0x7f9cf40b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cf9638640 1 -- 172.21.15.16:0/3731564360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf40d3ad0 msgr2=0x7f9cf40b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cf9638640 1 --2- 172.21.15.16:0/3731564360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf40d3ad0 0x7f9cf40b7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cf9638640 1 -- 172.21.15.16:0/3731564360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf415d580 msgr2=0x7f9cf40b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cf9638640 1 --2- 172.21.15.16:0/3731564360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf415d580 0x7f9cf40b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.851+0000 7f9cf9638640 1 -- 172.21.15.16:0/3731564360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec009a70 con 0x7f9cf415f890 2023-12-08T08:38:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.852+0000 7f9cf9638640 1 --2- 172.21.15.16:0/3731564360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415f890 0x7f9cf40b4aa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009d90 tx=0x7f9cec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.852+0000 7f9cf17fa640 1 -- 172.21.15.16:0/3731564360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec03e610 con 0x7f9cf415f890 2023-12-08T08:38:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.852+0000 7f9cf17fa640 1 -- 172.21.15.16:0/3731564360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec02ce90 con 0x7f9cf415f890 2023-12-08T08:38:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.852+0000 7f9cfb09b640 1 -- 172.21.15.16:0/3731564360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf41f6130 con 0x7f9cf415f890 2023-12-08T08:38:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.852+0000 7f9cf17fa640 1 -- 172.21.15.16:0/3731564360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec03e9d0 con 0x7f9cf415f890 2023-12-08T08:38:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.852+0000 7f9cfb09b640 1 -- 172.21.15.16:0/3731564360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf41f66f0 con 0x7f9cf415f890 2023-12-08T08:38:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.853+0000 7f9cf17fa640 1 -- 172.21.15.16:0/3731564360 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9cec03b050 con 0x7f9cf415f890 2023-12-08T08:38:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.854+0000 7f9cf17fa640 1 --2- 172.21.15.16:0/3731564360 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9cd806ae40 0x7f9cd806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.854+0000 7f9cf17fa640 1 -- 172.21.15.16:0/3731564360 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9cec0aba50 con 0x7f9cf415f890 2023-12-08T08:38:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.854+0000 7f9cf8e37640 1 --2- 172.21.15.16:0/3731564360 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9cd806ae40 0x7f9cd806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.855+0000 7f9cf8e37640 1 --2- 172.21.15.16:0/3731564360 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9cd806ae40 0x7f9cd806d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800aac0 tx=0x7f9ce8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.855+0000 7f9cfdc38740 1 Processor -- start 2023-12-08T08:38:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.855+0000 7f9cfdc38740 1 -- start start 2023-12-08T08:38:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.855+0000 7f9cfdc38740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdac50 0x557399cadbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.855+0000 7f9cfdc38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399cdd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.855+0000 7f9cfdc38740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399cae130 0x557399cddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cfdc38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557399c6e980 con 0x557399cdb030 2023-12-08T08:38:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cfdc38740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x557399c20b90 con 0x557399cae130 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cfdc38740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x557399c1e280 con 0x557399cdac50 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cf3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399cdd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cf8e37640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdac50 0x557399cadbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cf3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399cdd9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53742/0 (socket says 172.21.15.16:53742) 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cf3fff640 1 -- 172.21.15.16:0/3081755224 learned_addr learned my addr 172.21.15.16:0/3081755224 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cf9638640 1 --2- 172.21.15.16:0/3081755224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399cae130 0x557399cddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cf3fff640 1 -- 172.21.15.16:0/3081755224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdac50 msgr2=0x557399cadbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/3081755224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdac50 0x557399cadbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cf3fff640 1 -- 172.21.15.16:0/3081755224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399cae130 msgr2=0x557399cddf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/3081755224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399cae130 0x557399cddf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cf3fff640 1 -- 172.21.15.16:0/3081755224 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557399bdf500 con 0x557399cdb030 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cf8e37640 1 --2- 172.21.15.16:0/3081755224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdac50 0x557399cadbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/3081755224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399cdd9e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0001740 tx=0x7f9ce0007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a22670448e7d24 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cde7fc640 1 -- 172.21.15.16:0/3081755224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0005ac0 con 0x557399cdb030 2023-12-08T08:38:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cde7fc640 1 -- 172.21.15.16:0/3081755224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce0005c20 con 0x557399cdb030 2023-12-08T08:38:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.856+0000 7f9cde7fc640 1 -- 172.21.15.16:0/3081755224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00070d0 con 0x557399cdb030 2023-12-08T08:38:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.857+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3081755224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 msgr2=0x557399cdd9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.857+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3081755224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399cdd9e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9ce0001740 tx=0x7f9ce0007ec0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.857+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3081755224 shutdown_connections 2023-12-08T08:38:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.857+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3081755224 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399cae130 0x557399cddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.857+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3081755224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399cdd9e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.857+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3081755224 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdac50 0x557399cadbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.857+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3081755224 >> 172.21.15.16:0/3081755224 conn(0x557399cafe60 msgr2=0x557399cf89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.857+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3081755224 shutdown_connections 2023-12-08T08:38:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.857+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3081755224 wait complete. 2023-12-08T08:38:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.857+0000 7f9cfdc38740 1 -- start start 2023-12-08T08:38:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.858+0000 7f9cfdc38740 1 Processor -- start 2023-12-08T08:38:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.858+0000 7f9cfdc38740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdac50 0x557399ce05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.858+0000 7f9cfdc38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399d08d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.858+0000 7f9cfdc38740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399d092b0 0x557399cdca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.858+0000 7f9cfdc38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557399c19b70 con 0x557399cdb030 2023-12-08T08:38:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.858+0000 7f9cfdc38740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x557399bd3e00 con 0x557399d092b0 2023-12-08T08:38:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.858+0000 7f9cfdc38740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x557399bd3aa0 con 0x557399cdac50 2023-12-08T08:38:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cfdc38740 10 client.0 _ll_register_callbacks cb 0x55739992bac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-08T08:38:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cf3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399d08d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cf3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399d08d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53756/0 (socket says 172.21.15.16:53756) 2023-12-08T08:38:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cf3fff640 1 -- 172.21.15.16:0/3532800122 learned_addr learned my addr 172.21.15.16:0/3532800122 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cf8e37640 1 --2- 172.21.15.16:0/3532800122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdac50 0x557399ce05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cf9638640 1 --2- 172.21.15.16:0/3532800122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399d092b0 0x557399cdca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cf3fff640 1 -- 172.21.15.16:0/3532800122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdac50 msgr2=0x557399ce05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/3532800122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdac50 0x557399ce05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cf3fff640 1 -- 172.21.15.16:0/3532800122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399d092b0 msgr2=0x557399cdca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/3532800122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399d092b0 0x557399cdca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cf3fff640 1 -- 172.21.15.16:0/3532800122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf415ee90 con 0x557399cdb030 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/3532800122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399d08d70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00016e0 tx=0x7f9ce0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883335bd134f96e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cd37fe640 1 -- 172.21.15.16:0/3532800122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0005ac0 con 0x557399cdb030 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cd37fe640 1 -- 172.21.15.16:0/3532800122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9ce0028dd0 con 0x557399cdb030 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3532800122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557399c12b50 con 0x557399cdb030 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cd37fe640 1 -- 172.21.15.16:0/3532800122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ce000f7b0 con 0x557399cdb030 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cfdc38740 10 client.4863 fetch_fsmap learned FSMap version 9 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.859+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3532800122 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x557399c181f0 con 0x557399cdb030 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.860+0000 7f9cd37fe640 1 -- 172.21.15.16:0/3532800122 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557399c12b50 con 0x557399cdb030 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.860+0000 7f9cd37fe640 1 -- 172.21.15.16:0/3532800122 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f9ce0012070 con 0x557399cdb030 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.860+0000 7f9cfdc38740 10 client.4863 fetch_fsmap finished waiting for FSMap version 9 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.860+0000 7f9cfdc38740 10 client.4863 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.860+0000 7f9cfdc38740 20 client.4863 populate_metadata read hostname 'smithi016' 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.860+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3532800122 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x557399cf0150 0x557399d0c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.860+0000 7f9cfdc38740 4 client.4863 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.860+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3532800122 --> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557399c12870 con 0x557399cf0150 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.860+0000 7f9cf8e37640 1 --2- 172.21.15.16:0/3532800122 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x557399cf0150 0x557399d0c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.860+0000 7f9cd37fe640 10 client.4863 ms_handle_connect on v2:172.21.15.112:6835/1082241707 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.860+0000 7f9cf8e37640 1 --2- 172.21.15.16:0/3532800122 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x557399cf0150 0x557399d0c660 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a04de3ff32321655 server_cookie=5de6a52d2b336e92 in_seq=0 out_seq=0 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.861+0000 7f9cd37fe640 1 -- 172.21.15.16:0/3532800122 <== mds.0 v2:172.21.15.112:6835/1082241707 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x557399c12870 con 0x557399cf0150 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.861+0000 7f9cd37fe640 10 client.4863 handle_command_reply: tid=0 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.861+0000 7f9cfdc38740 1 client.4863 shutdown 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.861+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3532800122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 msgr2=0x557399d08d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.861+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3532800122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399d08d70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00016e0 tx=0x7f9ce0000ed0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.861+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3532800122 shutdown_connections 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.861+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3532800122 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399d092b0 0x557399cdca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.861+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3532800122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399d08d70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.861+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3532800122 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x557399cf0150 0x557399d0c660 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.861+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3532800122 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdac50 0x557399ce05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.861+0000 7f9cd37fe640 0 client.4863 ms_handle_reset on v2:172.21.15.112:6835/1082241707 2023-12-08T08:38:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.861+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3532800122 >> 172.21.15.16:0/3532800122 conn(0x557399cae130 msgr2=0x557399cb1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.861+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3532800122 shutdown_connections 2023-12-08T08:38:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.861+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3532800122 wait complete. 2023-12-08T08:38:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.862+0000 7f9cfdc38740 20 client.4863 trim_cache size 0 max 16384 2023-12-08T08:38:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.878+0000 7f9cfdc38740 1 Processor -- start 2023-12-08T08:38:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.878+0000 7f9cfdc38740 1 -- start start 2023-12-08T08:38:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cfdc38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdac50 0x557399cd9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cfdc38740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdb030 0x557399cb4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cfdc38740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399d092b0 0x557399cfbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cfdc38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557399b99270 con 0x557399cdac50 2023-12-08T08:38:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cfdc38740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x557399cada30 con 0x557399d092b0 2023-12-08T08:38:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cfdc38740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x557399cb5150 con 0x557399cdb030 2023-12-08T08:38:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cf8e37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdac50 0x557399cd9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cf8e37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdac50 0x557399cd9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53768/0 (socket says 172.21.15.16:53768) 2023-12-08T08:38:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cf8e37640 1 -- 172.21.15.16:0/1934519335 learned_addr learned my addr 172.21.15.16:0/1934519335 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/1934519335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdb030 0x557399cb4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cf9638640 1 --2- 172.21.15.16:0/1934519335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399d092b0 0x557399cfbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cf8e37640 1 -- 172.21.15.16:0/1934519335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdb030 msgr2=0x557399cb4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cf8e37640 1 --2- 172.21.15.16:0/1934519335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdb030 0x557399cb4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cf8e37640 1 -- 172.21.15.16:0/1934519335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399d092b0 msgr2=0x557399cfbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cf8e37640 1 --2- 172.21.15.16:0/1934519335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399d092b0 0x557399cfbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cf8e37640 1 -- 172.21.15.16:0/1934519335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557399bb5d00 con 0x557399cdac50 2023-12-08T08:38:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cf9638640 1 --2- 172.21.15.16:0/1934519335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399d092b0 0x557399cfbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cf8e37640 1 --2- 172.21.15.16:0/1934519335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdac50 0x557399cd9ac0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8029c70 tx=0x7f9ce8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0304fd986d612e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cdeffd640 1 -- 172.21.15.16:0/1934519335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8026040 con 0x557399cdac50 2023-12-08T08:38:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cdeffd640 1 -- 172.21.15.16:0/1934519335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce800c040 con 0x557399cdac50 2023-12-08T08:38:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.879+0000 7f9cdeffd640 1 -- 172.21.15.16:0/1934519335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8029ea0 con 0x557399cdac50 2023-12-08T08:38:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.880+0000 7f9cfdc38740 1 -- 172.21.15.16:0/1934519335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdac50 msgr2=0x557399cd9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.880+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/1934519335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdac50 0x557399cd9ac0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8029c70 tx=0x7f9ce8016000 comp rx=0 tx=0).stop 2023-12-08T08:38:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.880+0000 7f9cfdc38740 1 -- 172.21.15.16:0/1934519335 shutdown_connections 2023-12-08T08:38:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.880+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/1934519335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399d092b0 0x557399cfbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.880+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/1934519335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399cdb030 0x557399cb4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.880+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/1934519335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdac50 0x557399cd9ac0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.880+0000 7f9cfdc38740 1 -- 172.21.15.16:0/1934519335 >> 172.21.15.16:0/1934519335 conn(0x557399cae130 msgr2=0x557399cb1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.880+0000 7f9cfdc38740 1 -- 172.21.15.16:0/1934519335 shutdown_connections 2023-12-08T08:38:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.880+0000 7f9cfdc38740 1 -- 172.21.15.16:0/1934519335 wait complete. 2023-12-08T08:38:52.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.880+0000 7f9cfdc38740 1 -- start start 2023-12-08T08:38:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.881+0000 7f9cfdc38740 1 Processor -- start 2023-12-08T08:38:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.881+0000 7f9cfdc38740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399cdac50 0x557399d49d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.881+0000 7f9cfdc38740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399ce0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.881+0000 7f9cfdc38740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399d092b0 0x557399cfbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.881+0000 7f9cfdc38740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x557399ce1890 con 0x557399cdb030 2023-12-08T08:38:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.881+0000 7f9cfdc38740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x557399ce0b40 con 0x557399cdac50 2023-12-08T08:38:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.881+0000 7f9cfdc38740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x557399d651a0 con 0x557399d092b0 2023-12-08T08:38:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.881+0000 7f9cfdc38740 10 client.0 _ll_register_callbacks cb 0x7f9ce0001320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-08T08:38:52.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.881+0000 7f9cf3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399ce0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.881+0000 7f9cf3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399ce0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53778/0 (socket says 172.21.15.16:53778) 2023-12-08T08:38:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.881+0000 7f9cf3fff640 1 -- 172.21.15.16:0/3992479621 learned_addr learned my addr 172.21.15.16:0/3992479621 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cf9638640 1 --2- 172.21.15.16:0/3992479621 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399d092b0 0x557399cfbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cf8e37640 1 --2- 172.21.15.16:0/3992479621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399cdac50 0x557399d49d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cf3fff640 1 -- 172.21.15.16:0/3992479621 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399d092b0 msgr2=0x557399cfbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/3992479621 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399d092b0 0x557399cfbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cf3fff640 1 -- 172.21.15.16:0/3992479621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399cdac50 msgr2=0x557399d49d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/3992479621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399cdac50 0x557399d49d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cf3fff640 1 -- 172.21.15.16:0/3992479621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce80041a0 con 0x557399cdb030 2023-12-08T08:38:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cf3fff640 1 --2- 172.21.15.16:0/3992479621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399ce0d20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00016e0 tx=0x7f9ce0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f198dc048ec9b0c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-12-08T08:38:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cd37fe640 1 -- 172.21.15.16:0/3992479621 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0001510 con 0x557399cdb030 2023-12-08T08:38:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cd37fe640 1 -- 172.21.15.16:0/3992479621 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9ce000fb20 con 0x557399cdb030 2023-12-08T08:38:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3992479621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ce0005c20 con 0x557399cdb030 2023-12-08T08:38:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cd37fe640 1 -- 172.21.15.16:0/3992479621 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ce000f9c0 con 0x557399cdb030 2023-12-08T08:38:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cfdc38740 10 client.4869 fetch_fsmap learned FSMap version 9 2023-12-08T08:38:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3992479621 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x557399bdb4f0 con 0x557399cdb030 2023-12-08T08:38:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.882+0000 7f9cd37fe640 1 -- 172.21.15.16:0/3992479621 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce0005c20 con 0x557399cdb030 2023-12-08T08:38:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.883+0000 7f9cd37fe640 1 -- 172.21.15.16:0/3992479621 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f9ce0012040 con 0x557399cdb030 2023-12-08T08:38:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.883+0000 7f9cfdc38740 10 client.4869 fetch_fsmap finished waiting for FSMap version 9 2023-12-08T08:38:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.883+0000 7f9cfdc38740 10 client.4869 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-08T08:38:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.883+0000 7f9cfdc38740 20 client.4869 populate_metadata read hostname 'smithi016' 2023-12-08T08:38:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.883+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3992479621 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x557399d642b0 0x557399d64690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.883+0000 7f9cfdc38740 4 client.4869 mds_command: new command op to 4297 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-08T08:38:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.883+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3992479621 --> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55739991a6c0 con 0x557399d642b0 2023-12-08T08:38:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.883+0000 7f9cf8e37640 1 --2- 172.21.15.16:0/3992479621 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x557399d642b0 0x557399d64690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.883+0000 7f9cd37fe640 10 client.4869 ms_handle_connect on v2:172.21.15.112:6835/1082241707 2023-12-08T08:38:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.883+0000 7f9cf8e37640 1 --2- 172.21.15.16:0/3992479621 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x557399d642b0 0x557399d64690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46052992e1c54a44 server_cookie=3b6629b372dfa89d in_seq=0 out_seq=0 2023-12-08T08:38:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.883+0000 7f9cd37fe640 1 -- 172.21.15.16:0/3992479621 <== mds.0 v2:172.21.15.112:6835/1082241707 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55739991a6c0 con 0x557399d642b0 2023-12-08T08:38:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.883+0000 7f9cd37fe640 10 client.4869 handle_command_reply: tid=0 2023-12-08T08:38:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.883+0000 7f9cfdc38740 1 client.4869 shutdown 2023-12-08T08:38:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3992479621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 msgr2=0x557399ce0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3992479621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399ce0d20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9ce00016e0 tx=0x7f9ce0009040 comp rx=0 tx=0).stop 2023-12-08T08:38:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3992479621 shutdown_connections 2023-12-08T08:38:52.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3992479621 >> [v2:172.21.15.112:6835/1082241707,v1:172.21.15.112:6837/1082241707] conn(0x557399d642b0 0x557399d64690 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3992479621 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x557399d092b0 0x557399cfbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3992479621 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557399cdb030 0x557399ce0d20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cd37fe640 0 client.4869 ms_handle_reset on v2:172.21.15.112:6835/1082241707 2023-12-08T08:38:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfdc38740 1 --2- 172.21.15.16:0/3992479621 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x557399cdac50 0x557399d49d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3992479621 >> 172.21.15.16:0/3992479621 conn(0x557399cae130 msgr2=0x557399cf06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3992479621 shutdown_connections 2023-12-08T08:38:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfdc38740 1 -- 172.21.15.16:0/3992479621 wait complete. 2023-12-08T08:38:52.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfdc38740 20 client.4869 trim_cache size 0 max 16384 2023-12-08T08:38:52.893 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-12-08T08:38:52.893 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ead7ea87-fe4a-461b-916c-c6458a26d625", 2023-12-08T08:38:52.893 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-12-08T08:38:52.894 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4297, 2023-12-08T08:38:52.894 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-12-08T08:38:52.894 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-12-08T08:38:52.894 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-12-08T08:38:52.894 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 33.252206418999997, 2023-12-08T08:38:52.894 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 9, 2023-12-08T08:38:52.894 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 24, 2023-12-08T08:38:52.894 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 24, 2023-12-08T08:38:52.894 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 40.266388147999997 2023-12-08T08:38:52.894 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-12-08T08:38:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfb09b640 1 -- 172.21.15.16:0/3731564360 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9cd806ae40 msgr2=0x7f9cd806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfb09b640 1 --2- 172.21.15.16:0/3731564360 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9cd806ae40 0x7f9cd806d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800aac0 tx=0x7f9ce8008040 comp rx=0 tx=0).stop 2023-12-08T08:38:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfb09b640 1 -- 172.21.15.16:0/3731564360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415f890 msgr2=0x7f9cf40b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfb09b640 1 --2- 172.21.15.16:0/3731564360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415f890 0x7f9cf40b4aa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9cec009d90 tx=0x7f9cec004a40 comp rx=0 tx=0).stop 2023-12-08T08:38:52.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfb09b640 1 -- 172.21.15.16:0/3731564360 shutdown_connections 2023-12-08T08:38:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfb09b640 1 --2- 172.21.15.16:0/3731564360 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9cd806ae40 0x7f9cd806d300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfb09b640 1 --2- 172.21.15.16:0/3731564360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9cf415f890 0x7f9cf40b4aa0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfb09b640 1 --2- 172.21.15.16:0/3731564360 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9cf415d580 0x7f9cf40b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfb09b640 1 --2- 172.21.15.16:0/3731564360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9cf40d3ad0 0x7f9cf40b7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfb09b640 1 -- 172.21.15.16:0/3731564360 >> 172.21.15.16:0/3731564360 conn(0x7f9cf40d2fb0 msgr2=0x7f9cf40d3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfb09b640 1 -- 172.21.15.16:0/3731564360 shutdown_connections 2023-12-08T08:38:52.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:52.884+0000 7f9cfb09b640 1 -- 172.21.15.16:0/3731564360 wait complete. 2023-12-08T08:38:52.898 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-12-08T08:38:52.909 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-12-08T08:38:52.909 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-12-08T08:38:52.910 INFO:tasks.ceph_fuse:client.0 config is {} 2023-12-08T08:38:52.910 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-08T08:38:52.910 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-08T08:38:52.910 INFO:tasks.ceph_fuse:client.2 config is {} 2023-12-08T08:38:52.910 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-08T08:38:52.911 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-08T08:38:52.911 INFO:tasks.ceph_fuse:client.1 config is {} 2023-12-08T08:38:52.911 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-08T08:38:52.911 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-08T08:38:52.911 INFO:tasks.ceph_fuse:client.3 config is {} 2023-12-08T08:38:52.911 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-08T08:38:52.912 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-08T08:38:52.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:52.912 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-12-08T08:38:52.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:52.931 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link delete ceph-brx 2023-12-08T08:38:53.004 INFO:teuthology.orchestra.run.smithi112.stderr:Cannot find device "ceph-brx" 2023-12-08T08:38:53.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T08:38:53.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:53.007 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-12-08T08:38:53.023 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-12-08T08:38:53.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:53.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-08T08:38:53.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:53.094 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2023-12-08T08:38:53.206 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-12-08T08:38:53.206 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-08T08:38:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.310+0000 7f84abfa9640 1 Processor -- start 2023-12-08T08:38:53.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.310+0000 7f84abfa9640 1 -- start start 2023-12-08T08:38:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.311+0000 7f84abfa9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f84a4160ca0 0x7f84a4161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:53.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.311+0000 7f84abfa9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84a40ad290 0x7f84a415d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.311+0000 7f84abfa9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a406aee0 con 0x7f84a40ad710 2023-12-08T08:38:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.311+0000 7f84abfa9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a406bd40 con 0x7f84a40ad290 2023-12-08T08:38:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.311+0000 7f84abfa9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84a41615c0 con 0x7f84a4160ca0 2023-12-08T08:38:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.311+0000 7f84a9544640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84a40ad290 0x7f84a415d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.311+0000 7f84a9544640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84a40ad290 0x7f84a415d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53790/0 (socket says 172.21.15.16:53790) 2023-12-08T08:38:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.311+0000 7f84a9544640 1 -- 172.21.15.16:0/3137389731 learned_addr learned my addr 172.21.15.16:0/3137389731 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.311+0000 7f84a9d45640 1 --2- 172.21.15.16:0/3137389731 >> v2:172.21.15.112:3300/0 conn(0x7f84a4160ca0 0x7f84a4161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.311+0000 7f84a9544640 1 -- 172.21.15.16:0/3137389731 >> v2:172.21.15.112:3300/0 conn(0x7f84a4160ca0 msgr2=0x7f84a4161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.311+0000 7f84a9544640 1 --2- 172.21.15.16:0/3137389731 >> v2:172.21.15.112:3300/0 conn(0x7f84a4160ca0 0x7f84a4161080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.311+0000 7f84a9544640 1 -- 172.21.15.16:0/3137389731 >> v1:172.21.15.112:6789/0 conn(0x7f84a40ad710 legacy=0x7f84a415dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:53.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.311+0000 7f84a9544640 1 -- 172.21.15.16:0/3137389731 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a406b130 con 0x7f84a40ad290 2023-12-08T08:38:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.312+0000 7f84a9544640 1 --2- 172.21.15.16:0/3137389731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a40ad290 0x7f84a415d6b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f84a4072330 tx=0x7f849402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffdb37ff231bba6 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.312+0000 7f84a8d43640 1 -- 172.21.15.16:0/3137389731 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494030860 con 0x7f84a40ad290 2023-12-08T08:38:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.312+0000 7f84a8d43640 1 -- 172.21.15.16:0/3137389731 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84940309c0 con 0x7f84a40ad290 2023-12-08T08:38:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.312+0000 7f84a8d43640 1 -- 172.21.15.16:0/3137389731 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494030d50 con 0x7f84a40ad290 2023-12-08T08:38:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.313+0000 7f84abfa9640 1 -- 172.21.15.16:0/3137389731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a40ad290 msgr2=0x7f84a415d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.313+0000 7f84abfa9640 1 --2- 172.21.15.16:0/3137389731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a40ad290 0x7f84a415d6b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f84a4072330 tx=0x7f849402b9a0 comp rx=0 tx=0).stop 2023-12-08T08:38:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.313+0000 7f84abfa9640 1 -- 172.21.15.16:0/3137389731 shutdown_connections 2023-12-08T08:38:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.313+0000 7f84abfa9640 1 --2- 172.21.15.16:0/3137389731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a40ad290 0x7f84a415d6b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.313+0000 7f84abfa9640 1 --2- 172.21.15.16:0/3137389731 >> v2:172.21.15.112:3300/0 conn(0x7f84a4160ca0 0x7f84a4161080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.313+0000 7f84abfa9640 1 -- 172.21.15.16:0/3137389731 >> 172.21.15.16:0/3137389731 conn(0x7f84a40cf470 msgr2=0x7f84a40cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.313+0000 7f84abfa9640 1 -- 172.21.15.16:0/3137389731 shutdown_connections 2023-12-08T08:38:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.313+0000 7f84abfa9640 1 -- 172.21.15.16:0/3137389731 wait complete. 2023-12-08T08:38:53.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.314+0000 7f84abfa9640 1 Processor -- start 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.314+0000 7f84abfa9640 1 -- start start 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84abfa9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84a40ad290 0x7f84a413cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84abfa9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a40ad710 0x7f84a414f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84abfa9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84a4160ca0 0x7f84a40af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84abfa9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a414f5d0 con 0x7f84a40ad710 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84abfa9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a414e950 con 0x7f84a4160ca0 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84abfa9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a414ec30 con 0x7f84a40ad290 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84a9544640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a40ad710 0x7f84a414f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84a9544640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a40ad710 0x7f84a414f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53796/0 (socket says 172.21.15.16:53796) 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84a9544640 1 -- 172.21.15.16:0/2301838372 learned_addr learned my addr 172.21.15.16:0/2301838372 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84a9d45640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84a40ad290 0x7f84a413cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84aa546640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84a4160ca0 0x7f84a40af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84a9544640 1 -- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84a40ad290 msgr2=0x7f84a413cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84a9544640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84a40ad290 0x7f84a413cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84a9544640 1 -- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84a4160ca0 msgr2=0x7f84a40af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:53.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84a9544640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84a4160ca0 0x7f84a40af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84a9544640 1 -- 172.21.15.16:0/2301838372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84940094d0 con 0x7f84a40ad710 2023-12-08T08:38:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84aa546640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84a4160ca0 0x7f84a40af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f84a9544640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a40ad710 0x7f84a414f9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f849402b9e0 tx=0x7f8494004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.315+0000 7f849a7fc640 1 -- 172.21.15.16:0/2301838372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494039d60 con 0x7f84a40ad710 2023-12-08T08:38:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.316+0000 7f849a7fc640 1 -- 172.21.15.16:0/2301838372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8494043400 con 0x7f84a40ad710 2023-12-08T08:38:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.316+0000 7f849a7fc640 1 -- 172.21.15.16:0/2301838372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84940437b0 con 0x7f84a40ad710 2023-12-08T08:38:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.316+0000 7f84abfa9640 1 -- 172.21.15.16:0/2301838372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a414ef10 con 0x7f84a40ad710 2023-12-08T08:38:53.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.316+0000 7f84abfa9640 1 -- 172.21.15.16:0/2301838372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a413d280 con 0x7f84a40ad710 2023-12-08T08:38:53.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.316+0000 7f84abfa9640 1 -- 172.21.15.16:0/2301838372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a4067430 con 0x7f84a40ad710 2023-12-08T08:38:53.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.317+0000 7f849a7fc640 1 -- 172.21.15.16:0/2301838372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8494043910 con 0x7f84a40ad710 2023-12-08T08:38:53.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.317+0000 7f849a7fc640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f848806ae40 0x7f848806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:53.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.317+0000 7f849a7fc640 1 -- 172.21.15.16:0/2301838372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f849403e070 con 0x7f84a40ad710 2023-12-08T08:38:53.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.318+0000 7f84a9d45640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f848806ae40 0x7f848806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:53.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.318+0000 7f84a9d45640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f848806ae40 0x7f848806d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f84a00079a0 tx=0x7f84a0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:53.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.322+0000 7f849a7fc640 1 -- 172.21.15.16:0/2301838372 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84940779b0 con 0x7f84a40ad710 2023-12-08T08:38:53.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.533+0000 7f84abfa9640 1 -- 172.21.15.16:0/2301838372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f84a415ddd0 con 0x7f84a40ad710 2023-12-08T08:38:53.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.533+0000 7f849a7fc640 1 -- 172.21.15.16:0/2301838372 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f849407afd0 con 0x7f84a40ad710 2023-12-08T08:38:53.537 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-08T08:38:53.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.535+0000 7f84abfa9640 1 -- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f848806ae40 msgr2=0x7f848806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:53.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.535+0000 7f84abfa9640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f848806ae40 0x7f848806d300 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f84a00079a0 tx=0x7f84a0008040 comp rx=0 tx=0).stop 2023-12-08T08:38:53.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.535+0000 7f84abfa9640 1 -- 172.21.15.16:0/2301838372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a40ad710 msgr2=0x7f84a414f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:53.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.535+0000 7f84abfa9640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a40ad710 0x7f84a414f9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f849402b9e0 tx=0x7f8494004730 comp rx=0 tx=0).stop 2023-12-08T08:38:53.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.535+0000 7f84abfa9640 1 -- 172.21.15.16:0/2301838372 shutdown_connections 2023-12-08T08:38:53.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.535+0000 7f84abfa9640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f848806ae40 0x7f848806d300 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:53.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.535+0000 7f84abfa9640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84a4160ca0 0x7f84a40af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:53.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.535+0000 7f84abfa9640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84a40ad710 0x7f84a414f9f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:53.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.535+0000 7f84abfa9640 1 --2- 172.21.15.16:0/2301838372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84a40ad290 0x7f84a413cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:53.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.535+0000 7f84abfa9640 1 -- 172.21.15.16:0/2301838372 >> 172.21.15.16:0/2301838372 conn(0x7f84a40cf470 msgr2=0x7f84a40b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:53.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.535+0000 7f84abfa9640 1 -- 172.21.15.16:0/2301838372 shutdown_connections 2023-12-08T08:38:53.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:53.535+0000 7f84abfa9640 1 -- 172.21.15.16:0/2301838372 wait complete. 2023-12-08T08:38:53.548 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-08T08:38:53.549 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-08T08:38:53.549 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-12-08T08:38:53.549 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-08T08:38:53.549 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-08T08:38:53.549 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-08T08:38:53.549 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-08T08:38:53.549 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-08T08:38:53.549 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-08T08:38:53.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:53.549 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-12-08T08:38:53.565 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-08T08:38:53.565 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-08T08:38:53.565 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-12-08T08:38:53.565 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:53.565 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-12-08T08:38:53.566 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:53.566 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:53.566 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:53.566 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-12-08T08:38:53.566 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:53.566 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:53.566 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-12-08T08:38:53.566 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-08T08:38:53.566 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:53.566 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-12-08T08:38:53.566 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-08T08:38:53.566 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42579sec preferred_lft 42579sec 2023-12-08T08:38:53.566 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-12-08T08:38:53.567 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:53.567 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:53.567 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:53.567 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-12-08T08:38:53.567 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-08T08:38:53.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:53.567 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:38:53.567 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:38:53.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2023-12-08T08:38:53.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2023-12-08T08:38:53.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2023-12-08T08:38:53.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-08T08:38:53.568 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:38:53.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:53.688 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-08T08:38:53.716 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-12-08T08:38:53.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:53.718 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-12-08T08:38:53.775 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-12-08T08:38:53.775 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-08T08:38:53.775 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-08T08:38:53.775 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-08T08:38:53.775 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-08T08:38:53.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:53.775 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:38:53.775 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:38:53.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-08T08:38:53.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-08T08:38:53.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-08T08:38:53.776 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:38:53.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:53.882 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-12-08T08:38:53.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:53.897 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2023-12-08T08:38:53.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:53.953 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:38:53.953 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:38:53.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:38:53.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-08T08:38:53.953 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:38:54.042 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-08T08:38:54.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:54.043 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:38:54.043 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:38:54.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-08T08:38:54.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-08T08:38:54.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-08T08:38:54.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-08T08:38:54.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-08T08:38:54.043 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:38:54.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:54.146 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:38:54.146 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:38:54.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-12-08T08:38:54.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2023-12-08T08:38:54.146 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:38:54.194 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-08T08:38:54.195 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-08T08:38:54.195 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-08T08:38:54.209 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-08T08:38:54.209 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-08T08:38:54.210 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-08T08:38:54.264 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-12-08T08:38:54.336 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-12-08T08:38:54.392 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-12-08T08:38:54.392 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-12-08T08:38:54.392 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-12-08T08:38:54.392 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-12-08T08:38:54.392 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-12-08T08:38:54.392 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-12-08T08:38:54.392 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-12-08T08:38:54.392 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-12-08T08:38:54.393 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-12-08T08:38:54.393 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2023-12-08T08:38:54.393 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-12-08T08:38:54.393 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-12-08T08:38:54.393 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-12-08T08:38:54.393 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-12-08T08:38:54.393 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-12-08T08:38:54.393 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-12-08T08:38:54.393 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-12-08T08:38:54.393 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-12-08T08:38:54.393 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-12-08T08:38:54.393 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-12-08T08:38:54.394 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-08T08:38:54.394 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-08T08:38:54.394 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-12-08T08:38:54.394 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-12-08T08:38:54.394 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-12-08T08:38:54.394 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-12-08T08:38:54.394 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-12-08T08:38:54.394 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-12-08T08:38:54.394 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-12-08T08:38:54.394 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-12-08T08:38:54.394 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:38:54.394 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:38:54.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:54.395 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-12-08T08:38:54.447 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-08T08:38:54.447 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-08T08:38:54.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-12-08T08:38:54.521 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-12-08T08:38:54.578 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-12-08T08:38:54.578 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-12-08T08:38:54.578 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-12-08T08:38:54.578 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-12-08T08:38:54.578 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-12-08T08:38:54.578 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-12-08T08:38:54.578 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-12-08T08:38:54.578 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-12-08T08:38:54.578 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-12-08T08:38:54.578 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2023-12-08T08:38:54.579 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-12-08T08:38:54.579 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-12-08T08:38:54.579 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-12-08T08:38:54.579 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-12-08T08:38:54.579 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-12-08T08:38:54.579 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-12-08T08:38:54.579 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-12-08T08:38:54.579 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-12-08T08:38:54.579 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-12-08T08:38:54.579 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-12-08T08:38:54.579 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-08T08:38:54.579 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-08T08:38:54.579 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-12-08T08:38:54.579 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-12-08T08:38:54.580 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-12-08T08:38:54.580 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-12-08T08:38:54.580 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-12-08T08:38:54.580 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-12-08T08:38:54.580 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-12-08T08:38:54.580 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-12-08T08:38:54.580 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:38:54.580 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:38:54.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:54.580 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-12-08T08:38:54.585 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2023-12-08T08:38:54.582+0000 7f253d436480 -1 init, newargv = 0x55873bdd5f20 newargc=15 2023-12-08T08:38:54.585 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[99030]: starting ceph client 2023-12-08T08:38:54.589 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[99030]: starting fuse 2023-12-08T08:38:54.600 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-12-08T08:38:54.600 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-08T08:38:54.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-12-08T08:38:54.600 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-12-08T08:38:54.600 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-12-08T08:38:54.601 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-08T08:38:54.602 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-08T08:38:54.602 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-08T08:38:54.602 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-12-08T08:38:54.602 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-12-08T08:38:54.602 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-12-08T08:38:54.602 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-08T08:38:54.602 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:38:54.602 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2023-12-08T08:38:54.602 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-12-08T08:38:54.725 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.99030.asok 2023-12-08T08:38:54.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-08T08:38:54.736 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.99030.asok 2023-12-08T08:38:54.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:54.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.99030.asok status 2023-12-08T08:38:54.878 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-12-08T08:38:54.878 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-12-08T08:38:54.878 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "a2cd52c00149b5caff897ea7f79d0529b62620da", 2023-12-08T08:38:54.878 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.0.0-46-ga2cd52c0 (a2cd52c00149b5caff897ea7f79d0529b62620da) reef (dev)", 2023-12-08T08:38:54.878 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2023-12-08T08:38:54.878 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-12-08T08:38:54.879 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-08T08:38:54.879 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "99030", 2023-12-08T08:38:54.879 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-12-08T08:38:54.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:38:54.879 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-12-08T08:38:54.879 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-12-08T08:38:54.879 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4881, 2023-12-08T08:38:54.879 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-12-08T08:38:54.879 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-12-08T08:38:54.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-12-08T08:38:54.879 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4881 2023-12-08T08:38:54.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:38:54.879 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-12-08T08:38:54.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1798287694 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1798287694 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4881 192.168.0.1:0/1798287694", 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1798287694", 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 9, 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 24, 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-12-08T08:38:54.880 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-12-08T08:38:54.881 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-12-08T08:38:54.881 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-12-08T08:38:54.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-08T08:38:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.990+0000 7f1920a87640 1 Processor -- start 2023-12-08T08:38:54.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.990+0000 7f1920a87640 1 -- start start 2023-12-08T08:38:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.991+0000 7f1920a87640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c071e50 0x7f191c06c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.991+0000 7f1920a87640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f191c070db0 con 0x7f191c140a10 2023-12-08T08:38:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.991+0000 7f1920a87640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f191c071090 con 0x7f191c0ce3e0 2023-12-08T08:38:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.991+0000 7f1920a87640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c070ba0 con 0x7f191c071e50 2023-12-08T08:38:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.991+0000 7f191a59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f191c140a10 0x7f191c0cdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37574/0 (socket says 172.21.15.16:37574) 2023-12-08T08:38:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.991+0000 7f191a59c640 1 -- 172.21.15.16:0/2129848693 learned_addr learned my addr 172.21.15.16:0/2129848693 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.991+0000 7f191959a640 1 -- 172.21.15.16:0/2129848693 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312231575 0 0) 0x7f191c070db0 con 0x7f191c140a10 2023-12-08T08:38:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.991+0000 7f191959a640 1 -- 172.21.15.16:0/2129848693 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1908003520 con 0x7f191c140a10 2023-12-08T08:38:54.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.991+0000 7f191ad9d640 1 --2- 172.21.15.16:0/2129848693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c071e50 0x7f191c06c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:54.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191959a640 1 -- 172.21.15.16:0/2129848693 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3253146477 0 0) 0x7f1908003520 con 0x7f191c140a10 2023-12-08T08:38:54.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191959a640 1 -- 172.21.15.16:0/2129848693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c071e50 msgr2=0x7f191c06c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:54.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191959a640 1 --2- 172.21.15.16:0/2129848693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c071e50 0x7f191c06c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:54.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191959a640 1 -- 172.21.15.16:0/2129848693 >> v1:172.21.15.112:6789/0 conn(0x7f191c0ce3e0 legacy=0x7f191c0b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:54.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191959a640 1 -- 172.21.15.16:0/2129848693 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c071370 con 0x7f191c140a10 2023-12-08T08:38:54.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191959a640 1 -- 172.21.15.16:0/2129848693 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f1914001670 con 0x7f191c140a10 2023-12-08T08:38:54.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191959a640 1 -- 172.21.15.16:0/2129848693 >> v1:172.21.15.16:6789/0 conn(0x7f191c140a10 legacy=0x7f191c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:54.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191959a640 1 --2- 172.21.15.16:0/2129848693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080039a0 0x7f1908003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:54.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191959a640 1 --2- 172.21.15.16:0/2129848693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c071e50 0x7f19080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:54.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191959a640 1 --2- 172.21.15.16:0/2129848693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1908004d00 0x7f1908005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191959a640 1 -- 172.21.15.16:0/2129848693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c070ba0 con 0x7f19080039a0 2023-12-08T08:38:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191959a640 1 -- 172.21.15.16:0/2129848693 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190800baf0 con 0x7f1908004d00 2023-12-08T08:38:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191959a640 1 -- 172.21.15.16:0/2129848693 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190800bcb0 con 0x7f191c071e50 2023-12-08T08:38:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191a59c640 1 --2- 172.21.15.16:0/2129848693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080039a0 0x7f1908003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f1919d9b640 1 --2- 172.21.15.16:0/2129848693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c071e50 0x7f19080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191ad9d640 1 --2- 172.21.15.16:0/2129848693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1908004d00 0x7f1908005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191a59c640 1 -- 172.21.15.16:0/2129848693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c071e50 msgr2=0x7f19080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191a59c640 1 --2- 172.21.15.16:0/2129848693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c071e50 0x7f19080047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191a59c640 1 -- 172.21.15.16:0/2129848693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1908004d00 msgr2=0x7f1908005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191a59c640 1 --2- 172.21.15.16:0/2129848693 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1908004d00 0x7f1908005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f191a59c640 1 -- 172.21.15.16:0/2129848693 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1908003520 con 0x7f19080039a0 2023-12-08T08:38:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.992+0000 7f1919d9b640 1 --2- 172.21.15.16:0/2129848693 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c071e50 0x7f19080047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.993+0000 7f191a59c640 1 --2- 172.21.15.16:0/2129848693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080039a0 0x7f1908003dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1914007fd0 tx=0x7f19080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894a21889e98e353 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.993+0000 7f191959a640 1 -- 172.21.15.16:0/2129848693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1914005ad0 con 0x7f19080039a0 2023-12-08T08:38:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.993+0000 7f191959a640 1 -- 172.21.15.16:0/2129848693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1914005c50 con 0x7f19080039a0 2023-12-08T08:38:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.993+0000 7f191959a640 1 -- 172.21.15.16:0/2129848693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191402c870 con 0x7f19080039a0 2023-12-08T08:38:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.993+0000 7f1920a87640 1 -- 172.21.15.16:0/2129848693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080039a0 msgr2=0x7f1908003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.993+0000 7f1920a87640 1 --2- 172.21.15.16:0/2129848693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19080039a0 0x7f1908003dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1914007fd0 tx=0x7f19080057f0 comp rx=0 tx=0).stop 2023-12-08T08:38:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.993+0000 7f191ad9d640 1 -- 172.21.15.16:0/2129848693 reap_dead start 2023-12-08T08:38:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.994+0000 7f1920a87640 1 -- 172.21.15.16:0/2129848693 shutdown_connections 2023-12-08T08:38:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.994+0000 7f1920a87640 1 -- 172.21.15.16:0/2129848693 >> 172.21.15.16:0/2129848693 conn(0x7f191c0b09f0 msgr2=0x7f191c0b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.994+0000 7f1920a87640 1 -- 172.21.15.16:0/2129848693 shutdown_connections 2023-12-08T08:38:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.994+0000 7f1920a87640 1 -- 172.21.15.16:0/2129848693 wait complete. 2023-12-08T08:38:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.995+0000 7f1920a87640 1 Processor -- start 2023-12-08T08:38:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.995+0000 7f1920a87640 1 -- start start 2023-12-08T08:38:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1920a87640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19080039a0 0x7f191c15ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1920a87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c15b0c0 0x7f191c15a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1920a87640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c15ab50 0x7f191c09afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1920a87640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c09bf00 con 0x7f191c15b0c0 2023-12-08T08:38:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1920a87640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c09c1e0 con 0x7f19080039a0 2023-12-08T08:38:55.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1920a87640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c09c4c0 con 0x7f191c15ab50 2023-12-08T08:38:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1919d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c15b0c0 0x7f191c15a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1919d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c15b0c0 0x7f191c15a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53820/0 (socket says 172.21.15.16:53820) 2023-12-08T08:38:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1919d9b640 1 -- 172.21.15.16:0/2936261865 learned_addr learned my addr 172.21.15.16:0/2936261865 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f191ad9d640 1 --2- 172.21.15.16:0/2936261865 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c15ab50 0x7f191c09afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f191a59c640 1 --2- 172.21.15.16:0/2936261865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19080039a0 0x7f191c15ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1919d9b640 1 -- 172.21.15.16:0/2936261865 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c15ab50 msgr2=0x7f191c09afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1919d9b640 1 --2- 172.21.15.16:0/2936261865 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c15ab50 0x7f191c09afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1919d9b640 1 -- 172.21.15.16:0/2936261865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19080039a0 msgr2=0x7f191c15ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1919d9b640 1 --2- 172.21.15.16:0/2936261865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19080039a0 0x7f191c15ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1919d9b640 1 -- 172.21.15.16:0/2936261865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c09aac0 con 0x7f191c15b0c0 2023-12-08T08:38:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1919d9b640 1 --2- 172.21.15.16:0/2936261865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c15b0c0 0x7f191c15a610 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f190400e580 tx=0x7f190400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1912ffd640 1 -- 172.21.15.16:0/2936261865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19040052e0 con 0x7f191c15b0c0 2023-12-08T08:38:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1912ffd640 1 -- 172.21.15.16:0/2936261865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1904005440 con 0x7f191c15b0c0 2023-12-08T08:38:55.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1920a87640 1 -- 172.21.15.16:0/2936261865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f191c09c7a0 con 0x7f191c15b0c0 2023-12-08T08:38:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1912ffd640 1 -- 172.21.15.16:0/2936261865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1904003830 con 0x7f191c15b0c0 2023-12-08T08:38:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.996+0000 7f1920a87640 1 -- 172.21.15.16:0/2936261865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191c09b520 con 0x7f191c15b0c0 2023-12-08T08:38:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.998+0000 7f1920a87640 1 -- 172.21.15.16:0/2936261865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191c09ac50 con 0x7f191c15b0c0 2023-12-08T08:38:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.998+0000 7f1912ffd640 1 -- 172.21.15.16:0/2936261865 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1904003990 con 0x7f191c15b0c0 2023-12-08T08:38:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.998+0000 7f1912ffd640 1 --2- 172.21.15.16:0/2936261865 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f18fc06ae40 0x7f18fc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.998+0000 7f1912ffd640 1 -- 172.21.15.16:0/2936261865 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1904089740 con 0x7f191c15b0c0 2023-12-08T08:38:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.999+0000 7f191a59c640 1 --2- 172.21.15.16:0/2936261865 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f18fc06ae40 0x7f18fc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:54.999+0000 7f191a59c640 1 --2- 172.21.15.16:0/2936261865 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f18fc06ae40 0x7f18fc06d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1914034000 tx=0x7f1914009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.003+0000 7f1912ffd640 1 -- 172.21.15.16:0/2936261865 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f19040545d0 con 0x7f191c15b0c0 2023-12-08T08:38:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.216+0000 7f1920a87640 1 -- 172.21.15.16:0/2936261865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f191c136490 con 0x7f191c15b0c0 2023-12-08T08:38:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.217+0000 7f1912ffd640 1 -- 172.21.15.16:0/2936261865 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1904057bf0 con 0x7f191c15b0c0 2023-12-08T08:38:55.220 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-08T08:38:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.219+0000 7f1920a87640 1 -- 172.21.15.16:0/2936261865 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f18fc06ae40 msgr2=0x7f18fc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.219+0000 7f1920a87640 1 --2- 172.21.15.16:0/2936261865 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f18fc06ae40 0x7f18fc06d300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1914034000 tx=0x7f1914009030 comp rx=0 tx=0).stop 2023-12-08T08:38:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.219+0000 7f1920a87640 1 -- 172.21.15.16:0/2936261865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c15b0c0 msgr2=0x7f191c15a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.219+0000 7f1920a87640 1 --2- 172.21.15.16:0/2936261865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c15b0c0 0x7f191c15a610 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f190400e580 tx=0x7f190400ea40 comp rx=0 tx=0).stop 2023-12-08T08:38:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.219+0000 7f1920a87640 1 -- 172.21.15.16:0/2936261865 shutdown_connections 2023-12-08T08:38:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.219+0000 7f1920a87640 1 --2- 172.21.15.16:0/2936261865 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f18fc06ae40 0x7f18fc06d300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:55.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.219+0000 7f1920a87640 1 --2- 172.21.15.16:0/2936261865 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f191c15ab50 0x7f191c09afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.219+0000 7f1920a87640 1 --2- 172.21.15.16:0/2936261865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c15b0c0 0x7f191c15a610 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.219+0000 7f1920a87640 1 --2- 172.21.15.16:0/2936261865 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f19080039a0 0x7f191c15ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.219+0000 7f1920a87640 1 -- 172.21.15.16:0/2936261865 >> 172.21.15.16:0/2936261865 conn(0x7f191c0b09f0 msgr2=0x7f191c0b0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.219+0000 7f1920a87640 1 -- 172.21.15.16:0/2936261865 shutdown_connections 2023-12-08T08:38:55.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:55.219+0000 7f1920a87640 1 -- 172.21.15.16:0/2936261865 wait complete. 2023-12-08T08:38:55.232 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-08T08:38:55.232 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-08T08:38:55.232 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-12-08T08:38:55.232 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-12-08T08:38:55.232 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-08T08:38:55.232 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-08T08:38:55.233 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-08T08:38:55.233 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-08T08:38:55.233 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-12-08T08:38:55.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:55.233 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-12-08T08:38:55.249 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-08T08:38:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-08T08:38:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-12-08T08:38:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-12-08T08:38:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:55.249 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:55.249 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-12-08T08:38:55.250 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-12-08T08:38:55.250 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-08T08:38:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-12-08T08:38:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-08T08:38:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42577sec preferred_lft 42577sec 2023-12-08T08:38:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-12-08T08:38:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:55.250 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:55.250 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-12-08T08:38:55.250 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-08T08:38:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 26:12:f3:8e:79:c7 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-08T08:38:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::20b7:78ff:fec0:4f49/64 scope link tentative 2023-12-08T08:38:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:55.251 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-08T08:38:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 26:12:f3:8e:79:c7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:38:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::2412:f3ff:fe8e:79c7/64 scope link tentative 2023-12-08T08:38:55.251 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:55.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:55.252 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-08T08:38:55.320 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-12-08T08:38:55.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:55.322 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-12-08T08:38:55.379 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-12-08T08:38:55.379 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-08T08:38:55.379 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-08T08:38:55.379 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-08T08:38:55.379 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-08T08:38:55.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:55.379 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:38:55.379 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:38:55.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-08T08:38:55.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-08T08:38:55.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-08T08:38:55.380 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:38:55.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:55.488 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-12-08T08:38:55.503 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-08T08:38:55.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:55.505 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2023-12-08T08:38:55.559 INFO:teuthology.orchestra.run.smithi016.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-12-08T08:38:55.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:55.560 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:38:55.560 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:38:55.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-08T08:38:55.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-12-08T08:38:55.560 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:38:55.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:55.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-08T08:38:55.684 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-08T08:38:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-08T08:38:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-12-08T08:38:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-12-08T08:38:55.684 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:55.685 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-08T08:38:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 32:f7:96:88:cd:4d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-08T08:38:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-08T08:38:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::30f7:96ff:fe88:cd4d/64 scope link tentative 2023-12-08T08:38:55.685 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:55.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:55.686 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-08T08:38:55.758 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-08T08:38:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-08T08:38:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-12-08T08:38:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-12-08T08:38:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:55.758 INFO:teuthology.orchestra.run.smithi016.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-08T08:38:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 32:f7:96:88:cd:4d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-08T08:38:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-08T08:38:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::30f7:96ff:fe88:cd4d/64 scope link tentative 2023-12-08T08:38:55.758 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:55.759 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-12-08T08:38:55.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:55.760 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:38:55.760 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:38:55.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-12-08T08:38:55.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-08T08:38:55.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-12-08T08:38:55.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-12-08T08:38:55.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-12-08T08:38:55.760 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:38:55.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:55.906 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:38:55.906 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:38:55.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.1 up 2023-12-08T08:38:55.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.1 master ceph-brx 2023-12-08T08:38:55.906 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:38:55.956 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-12-08T08:38:55.956 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-08T08:38:55.956 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-12-08T08:38:55.971 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-12-08T08:38:55.971 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-08T08:38:55.972 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-12-08T08:38:56.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-12-08T08:38:56.098 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-12-08T08:38:56.153 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-12-08T08:38:56.153 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-12-08T08:38:56.153 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-12-08T08:38:56.154 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-12-08T08:38:56.154 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-12-08T08:38:56.154 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-12-08T08:38:56.154 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-12-08T08:38:56.154 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-12-08T08:38:56.154 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-12-08T08:38:56.154 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2023-12-08T08:38:56.154 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-12-08T08:38:56.154 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-12-08T08:38:56.154 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-12-08T08:38:56.154 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-12-08T08:38:56.154 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-12-08T08:38:56.154 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-12-08T08:38:56.155 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:38:56.156 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:38:56.156 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2023-12-08T08:38:56.156 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-08T08:38:56.156 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-08T08:38:56.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:56.156 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-12-08T08:38:56.208 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-12-08T08:38:56.209 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-12-08T08:38:56.209 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-12-08T08:38:56.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-12-08T08:38:56.282 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-12-08T08:38:56.338 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-12-08T08:38:56.338 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-12-08T08:38:56.338 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-12-08T08:38:56.338 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-12-08T08:38:56.338 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-12-08T08:38:56.338 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-12-08T08:38:56.338 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-12-08T08:38:56.338 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-12-08T08:38:56.338 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-12-08T08:38:56.339 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2023-12-08T08:38:56.339 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-12-08T08:38:56.339 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-12-08T08:38:56.339 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-12-08T08:38:56.339 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-12-08T08:38:56.339 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-12-08T08:38:56.339 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-12-08T08:38:56.339 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-12-08T08:38:56.339 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-12-08T08:38:56.339 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-12-08T08:38:56.339 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-12-08T08:38:56.340 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-08T08:38:56.340 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-08T08:38:56.340 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-12-08T08:38:56.341 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-12-08T08:38:56.341 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-12-08T08:38:56.341 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-12-08T08:38:56.341 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-12-08T08:38:56.341 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-12-08T08:38:56.341 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-12-08T08:38:56.341 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-12-08T08:38:56.341 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:38:56.341 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:38:56.341 INFO:teuthology.orchestra.run.smithi016.stdout:/home/ubuntu/cephtest/mnt.0 2023-12-08T08:38:56.341 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-08T08:38:56.341 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-08T08:38:56.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:56.342 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-12-08T08:38:56.347 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:2023-12-08T08:38:56.344+0000 7faaef83a480 -1 init, newargv = 0x5594d5ba3090 newargc=15 2023-12-08T08:38:56.347 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:ceph-fuse[99536]: starting ceph client 2023-12-08T08:38:56.352 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi016.stderr:ceph-fuse[99536]: starting fuse 2023-12-08T08:38:56.363 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-12-08T08:38:56.363 INFO:teuthology.orchestra.run.smithi016.stdout:44 2023-12-08T08:38:56.364 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2023-12-08T08:38:56.364 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-12-08T08:38:56.364 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-12-08T08:38:56.364 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-12-08T08:38:56.365 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-12-08T08:38:56.365 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-12-08T08:38:56.365 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:38:56.365 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-12-08T08:38:56.365 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-12-08T08:38:56.365 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-12-08T08:38:56.365 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-12-08T08:38:56.366 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:38:56.366 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-12-08T08:38:56.366 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-12-08T08:38:56.366 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-12-08T08:38:56.366 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-12-08T08:38:56.366 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:38:56.366 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-12-08T08:38:56.366 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-08T08:38:56.367 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-08T08:38:56.367 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-08T08:38:56.367 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-12-08T08:38:56.367 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-12-08T08:38:56.367 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-12-08T08:38:56.367 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-08T08:38:56.367 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:38:56.368 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.2")) 2023-12-08T08:38:56.368 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-12-08T08:38:56.492 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.2.99536.asok 2023-12-08T08:38:56.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-08T08:38:56.503 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.99536.asok 2023-12-08T08:38:56.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:56.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.99536.asok status 2023-12-08T08:38:56.645 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-12-08T08:38:56.645 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-12-08T08:38:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "a2cd52c00149b5caff897ea7f79d0529b62620da", 2023-12-08T08:38:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.0.0-46-ga2cd52c0 (a2cd52c00149b5caff897ea7f79d0529b62620da) reef (dev)", 2023-12-08T08:38:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "2", 2023-12-08T08:38:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-12-08T08:38:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-12-08T08:38:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "99536", 2023-12-08T08:38:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-12-08T08:38:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:38:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-12-08T08:38:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-12-08T08:38:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4890, 2023-12-08T08:38:56.646 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 4890 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.2:0", 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 709173264 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.2:0", 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 709173264 2023-12-08T08:38:56.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:38:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.4890 192.168.0.2:0/709173264", 2023-12-08T08:38:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.2:0/709173264", 2023-12-08T08:38:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-12-08T08:38:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 9, 2023-12-08T08:38:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 24, 2023-12-08T08:38:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-12-08T08:38:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-12-08T08:38:56.648 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-12-08T08:38:56.648 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-12-08T08:38:56.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-08T08:38:56.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.757+0000 7fc9c6a48640 1 Processor -- start 2023-12-08T08:38:56.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.757+0000 7fc9c6a48640 1 -- start start 2023-12-08T08:38:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.757+0000 7fc9c6a48640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fc9c00cf760 0x7fc9c006a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.757+0000 7fc9c6a48640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9c015e150 0x7fc9c0161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.757+0000 7fc9c6a48640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c006b070 con 0x7fc9c00cf380 2023-12-08T08:38:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.757+0000 7fc9c6a48640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c0072290 con 0x7fc9c00cf760 2023-12-08T08:38:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.757+0000 7fc9c6a48640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00d0d00 con 0x7fc9c015e150 2023-12-08T08:38:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.758+0000 7fc9bffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc9c00cf380 0x7fc9c0071b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38064/0 (socket says 172.21.15.16:38064) 2023-12-08T08:38:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.758+0000 7fc9bffff640 1 -- 172.21.15.16:0/2813601965 learned_addr learned my addr 172.21.15.16:0/2813601965 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.758+0000 7fc9bf7fe640 1 --2- 172.21.15.16:0/2813601965 >> v2:172.21.15.112:3300/0 conn(0x7fc9c00cf760 0x7fc9c006a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.758+0000 7fc9c4fe5640 1 --2- 172.21.15.16:0/2813601965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9c015e150 0x7fc9c0161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.758+0000 7fc9beffd640 1 -- 172.21.15.16:0/2813601965 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2027639849 0 0) 0x7fc9c006b070 con 0x7fc9c00cf380 2023-12-08T08:38:56.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.758+0000 7fc9beffd640 1 -- 172.21.15.16:0/2813601965 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9a8003520 con 0x7fc9c00cf380 2023-12-08T08:38:56.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.758+0000 7fc9c4fe5640 1 -- 172.21.15.16:0/2813601965 >> v2:172.21.15.112:3300/0 conn(0x7fc9c00cf760 msgr2=0x7fc9c006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:38:56.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.758+0000 7fc9c4fe5640 1 --2- 172.21.15.16:0/2813601965 >> v2:172.21.15.112:3300/0 conn(0x7fc9c00cf760 0x7fc9c006a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:56.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.758+0000 7fc9c4fe5640 1 -- 172.21.15.16:0/2813601965 >> v1:172.21.15.16:6789/0 conn(0x7fc9c00cf380 legacy=0x7fc9c0071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:56.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.758+0000 7fc9c4fe5640 1 -- 172.21.15.16:0/2813601965 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c006aab0 con 0x7fc9c015e150 2023-12-08T08:38:56.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.759+0000 7fc9c4fe5640 1 --2- 172.21.15.16:0/2813601965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9c015e150 0x7fc9c0161690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009d90 tx=0x7fc9b802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cbbde81b12c9fc5 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:56.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.759+0000 7fc9beffd640 1 -- 172.21.15.16:0/2813601965 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b802cbc0 con 0x7fc9c015e150 2023-12-08T08:38:56.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.759+0000 7fc9beffd640 1 -- 172.21.15.16:0/2813601965 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b802cd20 con 0x7fc9c015e150 2023-12-08T08:38:56.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.759+0000 7fc9beffd640 1 -- 172.21.15.16:0/2813601965 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8034640 con 0x7fc9c015e150 2023-12-08T08:38:56.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.760+0000 7fc9c6a48640 1 -- 172.21.15.16:0/2813601965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9c015e150 msgr2=0x7fc9c0161690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:56.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.760+0000 7fc9c6a48640 1 --2- 172.21.15.16:0/2813601965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9c015e150 0x7fc9c0161690 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8009d90 tx=0x7fc9b802be10 comp rx=0 tx=0).stop 2023-12-08T08:38:56.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.760+0000 7fc9c6a48640 1 -- 172.21.15.16:0/2813601965 shutdown_connections 2023-12-08T08:38:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.760+0000 7fc9c6a48640 1 --2- 172.21.15.16:0/2813601965 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9c015e150 0x7fc9c0161690 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.760+0000 7fc9c6a48640 1 --2- 172.21.15.16:0/2813601965 >> v2:172.21.15.112:3300/0 conn(0x7fc9c00cf760 0x7fc9c006a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.760+0000 7fc9c6a48640 1 -- 172.21.15.16:0/2813601965 >> 172.21.15.16:0/2813601965 conn(0x7fc9c00cfda0 msgr2=0x7fc9c00d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.760+0000 7fc9c6a48640 1 -- 172.21.15.16:0/2813601965 shutdown_connections 2023-12-08T08:38:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.760+0000 7fc9c6a48640 1 -- 172.21.15.16:0/2813601965 wait complete. 2023-12-08T08:38:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.761+0000 7fc9c6a48640 1 Processor -- start 2023-12-08T08:38:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.761+0000 7fc9c6a48640 1 -- start start 2023-12-08T08:38:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9c6a48640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9c00cf380 0x7fc9c00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9c6a48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c00cf760 0x7fc9c00b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9c6a48640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c015e150 0x7fc9c01f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9c6a48640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00b3710 con 0x7fc9c00cf760 2023-12-08T08:38:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9c6a48640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00b39f0 con 0x7fc9c015e150 2023-12-08T08:38:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9c6a48640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c00b3cd0 con 0x7fc9c00cf380 2023-12-08T08:38:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c00cf760 0x7fc9c00b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c00cf760 0x7fc9c00b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53836/0 (socket says 172.21.15.16:53836) 2023-12-08T08:38:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/6799685 learned_addr learned my addr 172.21.15.16:0/6799685 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9bffff640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9c00cf380 0x7fc9c00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9c4fe5640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c015e150 0x7fc9c01f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9c00cf380 msgr2=0x7fc9c00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9bf7fe640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9c00cf380 0x7fc9c00b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c015e150 msgr2=0x7fc9c01f5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9bf7fe640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c015e150 0x7fc9c01f5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9bf7fe640 1 -- 172.21.15.16:0/6799685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b8009a70 con 0x7fc9c00cf760 2023-12-08T08:38:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9bffff640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9c00cf380 0x7fc9c00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.762+0000 7fc9c4fe5640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c015e150 0x7fc9c01f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:38:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.763+0000 7fc9bf7fe640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c00cf760 0x7fc9c00b2dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400e870 tx=0x7fc9b400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.763+0000 7fc9bcff9640 1 -- 172.21.15.16:0/6799685 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b40094f0 con 0x7fc9c00cf760 2023-12-08T08:38:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.763+0000 7fc9bcff9640 1 -- 172.21.15.16:0/6799685 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b400c940 con 0x7fc9c00cf760 2023-12-08T08:38:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.763+0000 7fc9c6a48640 1 -- 172.21.15.16:0/6799685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c00b3fb0 con 0x7fc9c00cf760 2023-12-08T08:38:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.763+0000 7fc9bcff9640 1 -- 172.21.15.16:0/6799685 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b400ccf0 con 0x7fc9c00cf760 2023-12-08T08:38:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.763+0000 7fc9c6a48640 1 -- 172.21.15.16:0/6799685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c00b4cf0 con 0x7fc9c00cf760 2023-12-08T08:38:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.764+0000 7fc9c6a48640 1 -- 172.21.15.16:0/6799685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c0074270 con 0x7fc9c00cf760 2023-12-08T08:38:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.765+0000 7fc9bcff9640 1 -- 172.21.15.16:0/6799685 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc9b4013470 con 0x7fc9c00cf760 2023-12-08T08:38:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.765+0000 7fc9bcff9640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc9ac06ae40 0x7fc9ac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.765+0000 7fc9bcff9640 1 -- 172.21.15.16:0/6799685 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc9b4089a40 con 0x7fc9c00cf760 2023-12-08T08:38:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.765+0000 7fc9bffff640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc9ac06ae40 0x7fc9ac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.766+0000 7fc9bffff640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc9ac06ae40 0x7fc9ac06d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0005880 tx=0x7fc9b0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.770+0000 7fc9bcff9640 1 -- 172.21.15.16:0/6799685 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc9b408be60 con 0x7fc9c00cf760 2023-12-08T08:38:56.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.982+0000 7fc9c6a48640 1 -- 172.21.15.16:0/6799685 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc9c0067430 con 0x7fc9c00cf760 2023-12-08T08:38:56.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.983+0000 7fc9bcff9640 1 -- 172.21.15.16:0/6799685 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fc9b4021070 con 0x7fc9c00cf760 2023-12-08T08:38:56.986 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-08T08:38:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.985+0000 7fc9c6a48640 1 -- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc9ac06ae40 msgr2=0x7fc9ac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.985+0000 7fc9c6a48640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc9ac06ae40 0x7fc9ac06d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0005880 tx=0x7fc9b0009040 comp rx=0 tx=0).stop 2023-12-08T08:38:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.985+0000 7fc9c6a48640 1 -- 172.21.15.16:0/6799685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c00cf760 msgr2=0x7fc9c00b2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.985+0000 7fc9c6a48640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c00cf760 0x7fc9c00b2dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400e870 tx=0x7fc9b400ed30 comp rx=0 tx=0).stop 2023-12-08T08:38:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.985+0000 7fc9c6a48640 1 -- 172.21.15.16:0/6799685 shutdown_connections 2023-12-08T08:38:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.985+0000 7fc9c6a48640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc9ac06ae40 0x7fc9ac06d300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.985+0000 7fc9c6a48640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc9c015e150 0x7fc9c01f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.985+0000 7fc9c6a48640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9c00cf760 0x7fc9c00b2dc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.985+0000 7fc9c6a48640 1 --2- 172.21.15.16:0/6799685 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc9c00cf380 0x7fc9c00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.985+0000 7fc9c6a48640 1 -- 172.21.15.16:0/6799685 >> 172.21.15.16:0/6799685 conn(0x7fc9c00cfda0 msgr2=0x7fc9c00b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:56.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.985+0000 7fc9c6a48640 1 -- 172.21.15.16:0/6799685 shutdown_connections 2023-12-08T08:38:56.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:56.985+0000 7fc9c6a48640 1 -- 172.21.15.16:0/6799685 wait complete. 2023-12-08T08:38:56.998 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-08T08:38:56.998 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-08T08:38:56.998 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-12-08T08:38:56.998 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-08T08:38:56.998 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-08T08:38:56.998 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-08T08:38:56.998 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-08T08:38:56.998 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-08T08:38:56.998 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-08T08:38:56.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:56.999 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-12-08T08:38:57.018 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-08T08:38:57.018 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-08T08:38:57.018 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-12-08T08:38:57.018 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:57.018 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-12-08T08:38:57.018 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:57.018 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:57.018 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:57.018 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2023-12-08T08:38:57.018 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:57.019 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:57.019 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2023-12-08T08:38:57.019 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:57.019 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:57.019 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-08T08:38:57.019 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:57.019 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-08T08:38:57.019 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42574sec preferred_lft 42574sec 2023-12-08T08:38:57.019 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2023-12-08T08:38:57.019 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:57.019 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-08T08:38:57.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:57.020 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-12-08T08:38:57.020 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-12-08T08:38:57.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add name ceph-brx type bridge 2023-12-08T08:38:57.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr flush dev ceph-brx 2023-12-08T08:38:57.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set ceph-brx up 2023-12-08T08:38:57.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-08T08:38:57.020 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-12-08T08:38:57.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:57.186 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-08T08:38:57.214 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-12-08T08:38:57.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:57.215 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-12-08T08:38:57.288 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-12-08T08:38:57.288 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-08T08:38:57.288 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-08T08:38:57.289 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-08T08:38:57.289 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-08T08:38:57.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:57.289 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-12-08T08:38:57.289 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-12-08T08:38:57.289 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-08T08:38:57.289 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-08T08:38:57.289 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-08T08:38:57.289 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-12-08T08:38:57.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:57.568 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-12-08T08:38:57.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:57.583 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2023-12-08T08:38:57.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:57.677 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-12-08T08:38:57.677 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-12-08T08:38:57.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-08T08:38:57.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-12-08T08:38:57.677 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-12-08T08:38:57.722 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-12-08T08:38:57.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:57.722 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-12-08T08:38:57.722 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-12-08T08:38:57.722 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-12-08T08:38:57.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-08T08:38:57.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-12-08T08:38:57.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-12-08T08:38:57.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-12-08T08:38:57.723 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-12-08T08:38:57.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:57.866 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-12-08T08:38:57.866 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-12-08T08:38:57.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.0 up 2023-12-08T08:38:57.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.0 master ceph-brx 2023-12-08T08:38:57.866 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-12-08T08:38:57.908 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-08T08:38:57.908 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-08T08:38:57.908 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-08T08:38:57.963 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-08T08:38:57.964 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-08T08:38:57.964 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-08T08:38:58.018 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-12-08T08:38:58.086 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2023-12-08T08:38:58.141 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2023-12-08T08:38:58.142 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2023-12-08T08:38:58.142 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2023-12-08T08:38:58.142 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2023-12-08T08:38:58.142 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2023-12-08T08:38:58.142 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2023-12-08T08:38:58.142 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2023-12-08T08:38:58.142 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2023-12-08T08:38:58.142 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2023-12-08T08:38:58.142 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2023-12-08T08:38:58.142 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2023-12-08T08:38:58.142 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2023-12-08T08:38:58.142 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2023-12-08T08:38:58.143 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2023-12-08T08:38:58.143 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2023-12-08T08:38:58.143 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2023-12-08T08:38:58.143 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2023-12-08T08:38:58.143 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2023-12-08T08:38:58.143 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2023-12-08T08:38:58.143 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2023-12-08T08:38:58.143 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-08T08:38:58.143 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-08T08:38:58.143 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2023-12-08T08:38:58.143 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2023-12-08T08:38:58.143 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2023-12-08T08:38:58.143 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-4 2023-12-08T08:38:58.143 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-5 2023-12-08T08:38:58.144 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-6 2023-12-08T08:38:58.144 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-7 2023-12-08T08:38:58.144 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2023-12-08T08:38:58.144 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-08T08:38:58.144 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-08T08:38:58.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:58.144 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-12-08T08:38:58.197 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-08T08:38:58.197 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-08T08:38:58.239 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-12-08T08:38:58.265 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2023-12-08T08:38:58.320 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2023-12-08T08:38:58.321 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2023-12-08T08:38:58.321 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2023-12-08T08:38:58.321 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2023-12-08T08:38:58.321 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2023-12-08T08:38:58.321 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2023-12-08T08:38:58.321 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2023-12-08T08:38:58.321 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2023-12-08T08:38:58.321 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2023-12-08T08:38:58.321 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2023-12-08T08:38:58.321 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2023-12-08T08:38:58.321 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2023-12-08T08:38:58.321 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2023-12-08T08:38:58.321 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2023-12-08T08:38:58.321 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2023-12-08T08:38:58.322 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2023-12-08T08:38:58.322 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2023-12-08T08:38:58.322 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2023-12-08T08:38:58.322 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2023-12-08T08:38:58.322 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2023-12-08T08:38:58.322 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-08T08:38:58.322 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-08T08:38:58.322 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2023-12-08T08:38:58.322 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2023-12-08T08:38:58.322 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2023-12-08T08:38:58.322 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-4 2023-12-08T08:38:58.322 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-5 2023-12-08T08:38:58.322 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-6 2023-12-08T08:38:58.322 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-7 2023-12-08T08:38:58.323 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2023-12-08T08:38:58.323 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-08T08:38:58.323 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-08T08:38:58.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:58.324 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-12-08T08:38:58.342 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi112.stderr:2023-12-08T08:38:58.340+0000 7fe532252480 -1 init, newargv = 0x56006d8ac5f0 newargc=15 2023-12-08T08:38:58.342 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi112.stderr:ceph-fuse[95801]: starting ceph client 2023-12-08T08:38:58.347 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi112.stderr:ceph-fuse[95801]: starting fuse 2023-12-08T08:38:58.357 INFO:teuthology.orchestra.run.smithi112.stdout:41 2023-12-08T08:38:58.357 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-08T08:38:58.357 DEBUG:teuthology.orchestra.run.smithi112:> sudo stdin-killer -- python3 -c ' 2023-12-08T08:38:58.357 DEBUG:teuthology.orchestra.run.smithi112:> import glob 2023-12-08T08:38:58.357 DEBUG:teuthology.orchestra.run.smithi112:> import re 2023-12-08T08:38:58.357 DEBUG:teuthology.orchestra.run.smithi112:> import os 2023-12-08T08:38:58.357 DEBUG:teuthology.orchestra.run.smithi112:> import subprocess 2023-12-08T08:38:58.358 DEBUG:teuthology.orchestra.run.smithi112:> 2023-12-08T08:38:58.358 DEBUG:teuthology.orchestra.run.smithi112:> def _find_admin_socket(client_name): 2023-12-08T08:38:58.358 DEBUG:teuthology.orchestra.run.smithi112:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-08T08:38:58.358 DEBUG:teuthology.orchestra.run.smithi112:> files = glob.glob(asok_path) 2023-12-08T08:38:58.358 DEBUG:teuthology.orchestra.run.smithi112:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-08T08:38:58.358 DEBUG:teuthology.orchestra.run.smithi112:> 2023-12-08T08:38:58.358 DEBUG:teuthology.orchestra.run.smithi112:> # Given a non-glob path, it better be there 2023-12-08T08:38:58.358 DEBUG:teuthology.orchestra.run.smithi112:> if "*" not in asok_path: 2023-12-08T08:38:58.358 DEBUG:teuthology.orchestra.run.smithi112:> assert(len(files) == 1) 2023-12-08T08:38:58.358 DEBUG:teuthology.orchestra.run.smithi112:> return files[0] 2023-12-08T08:38:58.358 DEBUG:teuthology.orchestra.run.smithi112:> 2023-12-08T08:38:58.358 DEBUG:teuthology.orchestra.run.smithi112:> for f in files: 2023-12-08T08:38:58.358 DEBUG:teuthology.orchestra.run.smithi112:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-08T08:38:58.359 DEBUG:teuthology.orchestra.run.smithi112:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-08T08:38:58.359 DEBUG:teuthology.orchestra.run.smithi112:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-08T08:38:58.359 DEBUG:teuthology.orchestra.run.smithi112:> contents = proc_f.read() 2023-12-08T08:38:58.359 DEBUG:teuthology.orchestra.run.smithi112:> if mountpoint in contents: 2023-12-08T08:38:58.359 DEBUG:teuthology.orchestra.run.smithi112:> return f 2023-12-08T08:38:58.359 DEBUG:teuthology.orchestra.run.smithi112:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-08T08:38:58.359 DEBUG:teuthology.orchestra.run.smithi112:> 2023-12-08T08:38:58.359 DEBUG:teuthology.orchestra.run.smithi112:> print(_find_admin_socket("client.1")) 2023-12-08T08:38:58.359 DEBUG:teuthology.orchestra.run.smithi112:> ' 2023-12-08T08:38:58.477 INFO:teuthology.orchestra.run.smithi112.stdout:/var/run/ceph/ceph-client.1.95801.asok 2023-12-08T08:38:58.480 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:38:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-08T08:38:58.488 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.95801.asok 2023-12-08T08:38:58.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:58.489 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.95801.asok status 2023-12-08T08:38:58.624 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-12-08T08:38:58.625 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": { 2023-12-08T08:38:58.625 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_sha1": "a2cd52c00149b5caff897ea7f79d0529b62620da", 2023-12-08T08:38:58.625 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-46-ga2cd52c0 (a2cd52c00149b5caff897ea7f79d0529b62620da) reef (dev)", 2023-12-08T08:38:58.625 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "1", 2023-12-08T08:38:58.625 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2023-12-08T08:38:58.625 INFO:teuthology.orchestra.run.smithi112.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-08T08:38:58.625 INFO:teuthology.orchestra.run.smithi112.stdout: "pid": "95801", 2023-12-08T08:38:58.625 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2023-12-08T08:38:58.625 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-12-08T08:38:58.625 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_count": 0, 2023-12-08T08:38:58.625 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_pinned_count": 0, 2023-12-08T08:38:58.625 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4768, 2023-12-08T08:38:58.625 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": { 2023-12-08T08:38:58.625 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2023-12-08T08:38:58.626 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2023-12-08T08:38:58.626 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4768 2023-12-08T08:38:58.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-12-08T08:38:58.626 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2023-12-08T08:38:58.626 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2023-12-08T08:38:58.626 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2023-12-08T08:38:58.626 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 801408442 2023-12-08T08:38:58.626 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-12-08T08:38:58.626 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-12-08T08:38:58.626 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2023-12-08T08:38:58.626 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2023-12-08T08:38:58.626 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.1:0", 2023-12-08T08:38:58.627 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 801408442 2023-12-08T08:38:58.627 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-12-08T08:38:58.627 INFO:teuthology.orchestra.run.smithi112.stdout: "inst_str": "client.4768 192.168.0.1:0/801408442", 2023-12-08T08:38:58.627 INFO:teuthology.orchestra.run.smithi112.stdout: "addr_str": "192.168.0.1:0/801408442", 2023-12-08T08:38:58.627 INFO:teuthology.orchestra.run.smithi112.stdout: "inode_count": 1, 2023-12-08T08:38:58.627 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_epoch": 9, 2023-12-08T08:38:58.627 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch": 24, 2023-12-08T08:38:58.627 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch_barrier": 0, 2023-12-08T08:38:58.627 INFO:teuthology.orchestra.run.smithi112.stdout: "blocklisted": false, 2023-12-08T08:38:58.627 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs" 2023-12-08T08:38:58.627 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-12-08T08:38:58.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-08T08:38:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.748+0000 7febc5c6b640 1 Processor -- start 2023-12-08T08:38:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.748+0000 7febc5c6b640 1 -- start start 2023-12-08T08:38:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.749+0000 7febc5c6b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7febc006c650 0x7febc015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.749+0000 7febc5c6b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc00ce6f0 con 0x7febc00ce270 2023-12-08T08:38:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.749+0000 7febc5c6b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc00cf9c0 con 0x7febc00ccd70 2023-12-08T08:38:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.749+0000 7febc5c6b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc00d0a60 con 0x7febc006c650 2023-12-08T08:38:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.749+0000 7febbffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7febc006c650 0x7febc015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.749+0000 7febbeffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7febc00ce270 0x7febc006bf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38066/0 (socket says 172.21.15.16:38066) 2023-12-08T08:38:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.749+0000 7febbffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7febc006c650 0x7febc015ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53840/0 (socket says 172.21.15.16:53840) 2023-12-08T08:38:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.749+0000 7febbeffd640 1 -- 172.21.15.16:0/1328976230 learned_addr learned my addr 172.21.15.16:0/1328976230 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:38:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.750+0000 7febbffff640 1 -- 172.21.15.16:0/1328976230 >> v1:172.21.15.112:6789/0 conn(0x7febc00ccd70 legacy=0x7febc00cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.750+0000 7febbffff640 1 -- 172.21.15.16:0/1328976230 >> v1:172.21.15.16:6789/0 conn(0x7febc00ce270 legacy=0x7febc006bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.750+0000 7febbffff640 1 -- 172.21.15.16:0/1328976230 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc00cfca0 con 0x7febc006c650 2023-12-08T08:38:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.750+0000 7febbe7fc640 1 -- 172.21.15.16:0/1328976230 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926036752 0 0) 0x7febc00ce6f0 con 0x7febc00ce270 2023-12-08T08:38:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.750+0000 7febbffff640 1 --2- 172.21.15.16:0/1328976230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc006c650 0x7febc015ca50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7febb8009d90 tx=0x7febb802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7078a1653d7711ce server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.751+0000 7febbe7fc640 1 -- 172.21.15.16:0/1328976230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb802cd00 con 0x7febc006c650 2023-12-08T08:38:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.751+0000 7febbe7fc640 1 -- 172.21.15.16:0/1328976230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb802ce60 con 0x7febc006c650 2023-12-08T08:38:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.751+0000 7febbe7fc640 1 -- 172.21.15.16:0/1328976230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8034660 con 0x7febc006c650 2023-12-08T08:38:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.751+0000 7febc5c6b640 1 -- 172.21.15.16:0/1328976230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc006c650 msgr2=0x7febc015ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.751+0000 7febc5c6b640 1 --2- 172.21.15.16:0/1328976230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc006c650 0x7febc015ca50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7febb8009d90 tx=0x7febb802bf50 comp rx=0 tx=0).stop 2023-12-08T08:38:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.751+0000 7febc5c6b640 1 -- 172.21.15.16:0/1328976230 shutdown_connections 2023-12-08T08:38:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.751+0000 7febc5c6b640 1 --2- 172.21.15.16:0/1328976230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc006c650 0x7febc015ca50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.751+0000 7febc5c6b640 1 -- 172.21.15.16:0/1328976230 >> 172.21.15.16:0/1328976230 conn(0x7febc00d0de0 msgr2=0x7febc00d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.751+0000 7febc5c6b640 1 -- 172.21.15.16:0/1328976230 shutdown_connections 2023-12-08T08:38:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.751+0000 7febc5c6b640 1 -- 172.21.15.16:0/1328976230 wait complete. 2023-12-08T08:38:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.753+0000 7febc5c6b640 1 Processor -- start 2023-12-08T08:38:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.753+0000 7febc5c6b640 1 -- start start 2023-12-08T08:38:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.753+0000 7febc5c6b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febc006c650 0x7febc01f2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.753+0000 7febc5c6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc00ccd70 0x7febc00ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febc5c6b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febc00ce270 0x7febc01f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febc5c6b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc00b07c0 con 0x7febc00ccd70 2023-12-08T08:38:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febc5c6b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc00b0190 con 0x7febc00ce270 2023-12-08T08:38:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febc5c6b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc00cade0 con 0x7febc006c650 2023-12-08T08:38:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febbeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc00ccd70 0x7febc00ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febbeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc00ccd70 0x7febc00ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53848/0 (socket says 172.21.15.16:53848) 2023-12-08T08:38:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febbeffd640 1 -- 172.21.15.16:0/1365416054 learned_addr learned my addr 172.21.15.16:0/1365416054 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:38:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febbffff640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febc00ce270 0x7febc01f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febbf7fe640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febc006c650 0x7febc01f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febbeffd640 1 -- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febc006c650 msgr2=0x7febc01f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febbeffd640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febc006c650 0x7febc01f2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febbeffd640 1 -- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febc00ce270 msgr2=0x7febc01f6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febbeffd640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febc00ce270 0x7febc01f6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febbeffd640 1 -- 172.21.15.16:0/1365416054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb8009a70 con 0x7febc00ccd70 2023-12-08T08:38:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febbf7fe640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febc006c650 0x7febc01f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:38:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febbeffd640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc00ccd70 0x7febc00ca490 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7febb400c4a0 tx=0x7febb400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7feba7fff640 1 -- 172.21.15.16:0/1365416054 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb40074e0 con 0x7febc00ccd70 2023-12-08T08:38:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7feba7fff640 1 -- 172.21.15.16:0/1365416054 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb4007640 con 0x7febc00ccd70 2023-12-08T08:38:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7febc5c6b640 1 -- 172.21.15.16:0/1365416054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc00cb0c0 con 0x7febc00ccd70 2023-12-08T08:38:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.754+0000 7feba7fff640 1 -- 172.21.15.16:0/1365416054 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb4004a00 con 0x7febc00ccd70 2023-12-08T08:38:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.755+0000 7febc5c6b640 1 -- 172.21.15.16:0/1365416054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc01f28c0 con 0x7febc00ccd70 2023-12-08T08:38:58.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.756+0000 7febc5c6b640 1 -- 172.21.15.16:0/1365416054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc0067430 con 0x7febc00ccd70 2023-12-08T08:38:58.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.756+0000 7feba7fff640 1 -- 172.21.15.16:0/1365416054 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7febb4004b60 con 0x7febc00ccd70 2023-12-08T08:38:58.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.757+0000 7feba7fff640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7febac06ae40 0x7febac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:38:58.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.757+0000 7feba7fff640 1 -- 172.21.15.16:0/1365416054 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7febb4088a20 con 0x7febc00ccd70 2023-12-08T08:38:58.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.757+0000 7febbf7fe640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7febac06ae40 0x7febac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:38:58.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.757+0000 7febbf7fe640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7febac06ae40 0x7febac06d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7febb0007fd0 tx=0x7febb0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:38:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.761+0000 7feba7fff640 1 -- 172.21.15.16:0/1365416054 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7febb4053810 con 0x7febc00ccd70 2023-12-08T08:38:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.974+0000 7febc5c6b640 1 -- 172.21.15.16:0/1365416054 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7febc0072520 con 0x7febc00ccd70 2023-12-08T08:38:58.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.974+0000 7feba7fff640 1 -- 172.21.15.16:0/1365416054 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7febb4018070 con 0x7febc00ccd70 2023-12-08T08:38:58.977 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-08T08:38:58.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.976+0000 7febc5c6b640 1 -- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7febac06ae40 msgr2=0x7febac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:58.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.976+0000 7febc5c6b640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7febac06ae40 0x7febac06d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7febb0007fd0 tx=0x7febb0009040 comp rx=0 tx=0).stop 2023-12-08T08:38:58.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.976+0000 7febc5c6b640 1 -- 172.21.15.16:0/1365416054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc00ccd70 msgr2=0x7febc00ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:38:58.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.976+0000 7febc5c6b640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc00ccd70 0x7febc00ca490 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7febb400c4a0 tx=0x7febb400c960 comp rx=0 tx=0).stop 2023-12-08T08:38:58.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.976+0000 7febc5c6b640 1 -- 172.21.15.16:0/1365416054 shutdown_connections 2023-12-08T08:38:58.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.976+0000 7febc5c6b640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7febac06ae40 0x7febac06d300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:58.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.976+0000 7febc5c6b640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7febc00ce270 0x7febc01f6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:58.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.976+0000 7febc5c6b640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc00ccd70 0x7febc00ca490 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:58.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.976+0000 7febc5c6b640 1 --2- 172.21.15.16:0/1365416054 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7febc006c650 0x7febc01f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:38:58.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.976+0000 7febc5c6b640 1 -- 172.21.15.16:0/1365416054 >> 172.21.15.16:0/1365416054 conn(0x7febc00d0de0 msgr2=0x7febc00cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:38:58.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.976+0000 7febc5c6b640 1 -- 172.21.15.16:0/1365416054 shutdown_connections 2023-12-08T08:38:58.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:38:58.976+0000 7febc5c6b640 1 -- 172.21.15.16:0/1365416054 wait complete. 2023-12-08T08:38:58.990 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-08T08:38:58.990 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-08T08:38:58.990 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi112.front.sepia.ceph.com 2023-12-08T08:38:58.990 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-12-08T08:38:58.990 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-08T08:38:58.990 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-08T08:38:58.990 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-08T08:38:58.990 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-08T08:38:58.990 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-12-08T08:38:58.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:58.991 DEBUG:teuthology.orchestra.run.smithi112:> ip addr 2023-12-08T08:38:59.006 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-08T08:38:59.006 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-08T08:38:59.006 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-12-08T08:38:59.006 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:59.006 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-12-08T08:38:59.007 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:59.007 INFO:teuthology.orchestra.run.smithi112.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:59.007 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2c brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:59.007 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f0 2023-12-08T08:38:59.007 INFO:teuthology.orchestra.run.smithi112.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:59.007 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:d8:a5:2d brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:59.007 INFO:teuthology.orchestra.run.smithi112.stdout: altname enp7s0f1 2023-12-08T08:38:59.008 INFO:teuthology.orchestra.run.smithi112.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:38:59.008 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:68 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:59.008 INFO:teuthology.orchestra.run.smithi112.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-08T08:38:59.008 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 0c:c4:7a:8f:cb:69 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:59.008 INFO:teuthology.orchestra.run.smithi112.stdout: inet 172.21.15.112/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-08T08:38:59.008 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft 42572sec preferred_lft 42572sec 2023-12-08T08:38:59.008 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec4:7aff:fe8f:cb69/64 scope link 2023-12-08T08:38:59.009 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:59.009 INFO:teuthology.orchestra.run.smithi112.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-08T08:38:59.009 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 26:60:ad:74:1a:f4 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:38:59.009 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-08T08:38:59.009 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:59.009 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::eca9:e3ff:fe0e:a774/64 scope link 2023-12-08T08:38:59.009 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:59.009 INFO:teuthology.orchestra.run.smithi112.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-08T08:38:59.009 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether 26:60:ad:74:1a:f4 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-08T08:38:59.009 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::2460:adff:fe74:1af4/64 scope link tentative 2023-12-08T08:38:59.009 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:59.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:59.010 DEBUG:teuthology.orchestra.run.smithi112:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-08T08:38:59.072 INFO:teuthology.orchestra.run.smithi112.stdout:1 2023-12-08T08:38:59.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:59.073 DEBUG:teuthology.orchestra.run.smithi112:> route 2023-12-08T08:38:59.129 INFO:teuthology.orchestra.run.smithi112.stdout:Kernel IP routing table 2023-12-08T08:38:59.129 INFO:teuthology.orchestra.run.smithi112.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-08T08:38:59.129 INFO:teuthology.orchestra.run.smithi112.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-08T08:38:59.129 INFO:teuthology.orchestra.run.smithi112.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-08T08:38:59.129 INFO:teuthology.orchestra.run.smithi112.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-08T08:38:59.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:59.130 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-12-08T08:38:59.130 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-12-08T08:38:59.130 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-08T08:38:59.130 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-08T08:38:59.130 DEBUG:teuthology.orchestra.run.smithi112:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-08T08:38:59.130 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-12-08T08:38:59.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:59.225 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list 2023-12-08T08:38:59.239 INFO:teuthology.orchestra.run.smithi112.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-08T08:38:59.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:59.239 DEBUG:teuthology.orchestra.run.smithi112:> ip netns list-id 2023-12-08T08:38:59.293 INFO:teuthology.orchestra.run.smithi112.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-12-08T08:38:59.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:59.294 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-12-08T08:38:59.294 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-12-08T08:38:59.294 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-08T08:38:59.294 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-12-08T08:38:59.294 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-12-08T08:38:59.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:59.376 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-08T08:38:59.442 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-08T08:38:59.442 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-08T08:38:59.442 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-12-08T08:38:59.443 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:59.443 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-12-08T08:38:59.443 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:59.443 INFO:teuthology.orchestra.run.smithi112.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-08T08:38:59.443 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ee:2f:4d:3c:61:a1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-08T08:38:59.443 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-08T08:38:59.443 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:59.443 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec2f:4dff:fe3c:61a1/64 scope link tentative 2023-12-08T08:38:59.443 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:59.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:59.444 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-08T08:38:59.511 INFO:teuthology.orchestra.run.smithi112.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-08T08:38:59.511 INFO:teuthology.orchestra.run.smithi112.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-08T08:38:59.511 INFO:teuthology.orchestra.run.smithi112.stdout: inet 127.0.0.1/8 scope host lo 2023-12-08T08:38:59.511 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:59.511 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 ::1/128 scope host 2023-12-08T08:38:59.511 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:59.511 INFO:teuthology.orchestra.run.smithi112.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-08T08:38:59.511 INFO:teuthology.orchestra.run.smithi112.stdout: link/ether ee:2f:4d:3c:61:a1 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-08T08:38:59.511 INFO:teuthology.orchestra.run.smithi112.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-08T08:38:59.511 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:59.511 INFO:teuthology.orchestra.run.smithi112.stdout: inet6 fe80::ec2f:4dff:fe3c:61a1/64 scope link tentative 2023-12-08T08:38:59.511 INFO:teuthology.orchestra.run.smithi112.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:38:59.512 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-12-08T08:38:59.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:59.513 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-12-08T08:38:59.513 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-12-08T08:38:59.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-12-08T08:38:59.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-08T08:38:59.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-12-08T08:38:59.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-12-08T08:38:59.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-12-08T08:38:59.514 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-12-08T08:38:59.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:59.640 DEBUG:teuthology.orchestra.run.smithi112:> (cd / && exec bash -c ' 2023-12-08T08:38:59.640 DEBUG:teuthology.orchestra.run.smithi112:> set -e 2023-12-08T08:38:59.640 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set brx.1 up 2023-12-08T08:38:59.640 DEBUG:teuthology.orchestra.run.smithi112:> sudo ip link set dev brx.1 master ceph-brx 2023-12-08T08:38:59.640 DEBUG:teuthology.orchestra.run.smithi112:> ') 2023-12-08T08:38:59.682 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-12-08T08:38:59.682 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-08T08:38:59.682 DEBUG:teuthology.orchestra.run.smithi112:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-12-08T08:38:59.737 INFO:teuthology.orchestra.run.smithi112.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-12-08T08:38:59.737 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-08T08:38:59.737 DEBUG:teuthology.orchestra.run.smithi112:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-12-08T08:38:59.794 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-12-08T08:38:59.862 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2023-12-08T08:38:59.917 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2023-12-08T08:38:59.918 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2023-12-08T08:38:59.918 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2023-12-08T08:38:59.918 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2023-12-08T08:38:59.918 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2023-12-08T08:38:59.918 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2023-12-08T08:38:59.918 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2023-12-08T08:38:59.918 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2023-12-08T08:38:59.918 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2023-12-08T08:38:59.918 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2023-12-08T08:38:59.918 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2023-12-08T08:38:59.918 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2023-12-08T08:38:59.918 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2023-12-08T08:38:59.918 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2023-12-08T08:38:59.918 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2023-12-08T08:38:59.919 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2023-12-08T08:38:59.919 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2023-12-08T08:38:59.919 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2023-12-08T08:38:59.919 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2023-12-08T08:38:59.919 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2023-12-08T08:38:59.919 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-08T08:38:59.919 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-08T08:38:59.919 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2023-12-08T08:38:59.919 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2023-12-08T08:38:59.919 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2023-12-08T08:38:59.919 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-4 2023-12-08T08:38:59.919 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-5 2023-12-08T08:38:59.919 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-6 2023-12-08T08:38:59.919 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-7 2023-12-08T08:38:59.920 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2023-12-08T08:38:59.920 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-08T08:38:59.920 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-08T08:38:59.920 INFO:teuthology.orchestra.run.smithi112.stdout:/home/ubuntu/cephtest/mnt.1 2023-12-08T08:38:59.920 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-08T08:38:59.920 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-08T08:38:59.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:38:59.920 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-12-08T08:38:59.973 INFO:teuthology.orchestra.run.smithi112.stdout:41 2023-12-08T08:38:59.973 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-12-08T08:38:59.973 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-12-08T08:39:00.015 DEBUG:teuthology.orchestra.run.smithi112:> sudo modprobe fuse 2023-12-08T08:39:00.042 DEBUG:teuthology.orchestra.run.smithi112:> cat /proc/self/mounts | awk '{print $2}' 2023-12-08T08:39:00.098 INFO:teuthology.orchestra.run.smithi112.stdout:/proc 2023-12-08T08:39:00.098 INFO:teuthology.orchestra.run.smithi112.stdout:/sys 2023-12-08T08:39:00.098 INFO:teuthology.orchestra.run.smithi112.stdout:/dev 2023-12-08T08:39:00.098 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/security 2023-12-08T08:39:00.098 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/shm 2023-12-08T08:39:00.098 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/pts 2023-12-08T08:39:00.098 INFO:teuthology.orchestra.run.smithi112.stdout:/run 2023-12-08T08:39:00.098 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/cgroup 2023-12-08T08:39:00.098 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/pstore 2023-12-08T08:39:00.098 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/bpf 2023-12-08T08:39:00.098 INFO:teuthology.orchestra.run.smithi112.stdout:/ 2023-12-08T08:39:00.098 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/selinux 2023-12-08T08:39:00.099 INFO:teuthology.orchestra.run.smithi112.stdout:/proc/sys/fs/binfmt_misc 2023-12-08T08:39:00.099 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/mqueue 2023-12-08T08:39:00.099 INFO:teuthology.orchestra.run.smithi112.stdout:/dev/hugepages 2023-12-08T08:39:00.099 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/debug 2023-12-08T08:39:00.099 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/tracing 2023-12-08T08:39:00.099 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/fs/fuse/connections 2023-12-08T08:39:00.099 INFO:teuthology.orchestra.run.smithi112.stdout:/sys/kernel/config 2023-12-08T08:39:00.099 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-sysctl.service 2023-12-08T08:39:00.099 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-08T08:39:00.099 INFO:teuthology.orchestra.run.smithi112.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-08T08:39:00.099 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/nfs/rpc_pipefs 2023-12-08T08:39:00.099 INFO:teuthology.orchestra.run.smithi112.stdout:/run/user/1000 2023-12-08T08:39:00.099 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph 2023-12-08T08:39:00.100 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-4 2023-12-08T08:39:00.100 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-5 2023-12-08T08:39:00.100 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-6 2023-12-08T08:39:00.100 INFO:teuthology.orchestra.run.smithi112.stdout:/var/lib/ceph/osd/ceph-7 2023-12-08T08:39:00.100 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns 2023-12-08T08:39:00.100 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-08T08:39:00.100 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-08T08:39:00.100 INFO:teuthology.orchestra.run.smithi112.stdout:/home/ubuntu/cephtest/mnt.1 2023-12-08T08:39:00.100 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-08T08:39:00.100 INFO:teuthology.orchestra.run.smithi112.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-08T08:39:00.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:00.100 DEBUG:teuthology.orchestra.run.smithi112:> ls /sys/fs/fuse/connections 2023-12-08T08:39:00.101 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi112.stderr:2023-12-08T08:39:00.097+0000 7f3b56845480 -1 init, newargv = 0x559e1e92c050 newargc=15 2023-12-08T08:39:00.102 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi112.stderr:ceph-fuse[96236]: starting ceph client 2023-12-08T08:39:00.104 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi112.stderr:ceph-fuse[96236]: starting fuse 2023-12-08T08:39:00.115 INFO:teuthology.orchestra.run.smithi112.stdout:41 2023-12-08T08:39:00.115 INFO:teuthology.orchestra.run.smithi112.stdout:44 2023-12-08T08:39:00.116 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2023-12-08T08:39:00.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo stdin-killer -- python3 -c ' 2023-12-08T08:39:00.116 DEBUG:teuthology.orchestra.run.smithi112:> import glob 2023-12-08T08:39:00.116 DEBUG:teuthology.orchestra.run.smithi112:> import re 2023-12-08T08:39:00.116 DEBUG:teuthology.orchestra.run.smithi112:> import os 2023-12-08T08:39:00.116 DEBUG:teuthology.orchestra.run.smithi112:> import subprocess 2023-12-08T08:39:00.116 DEBUG:teuthology.orchestra.run.smithi112:> 2023-12-08T08:39:00.116 DEBUG:teuthology.orchestra.run.smithi112:> def _find_admin_socket(client_name): 2023-12-08T08:39:00.116 DEBUG:teuthology.orchestra.run.smithi112:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-12-08T08:39:00.116 DEBUG:teuthology.orchestra.run.smithi112:> files = glob.glob(asok_path) 2023-12-08T08:39:00.116 DEBUG:teuthology.orchestra.run.smithi112:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> # Given a non-glob path, it better be there 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> if "*" not in asok_path: 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> assert(len(files) == 1) 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> return files[0] 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> for f in files: 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> contents = proc_f.read() 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> if mountpoint in contents: 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> return f 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-08T08:39:00.117 DEBUG:teuthology.orchestra.run.smithi112:> 2023-12-08T08:39:00.118 DEBUG:teuthology.orchestra.run.smithi112:> print(_find_admin_socket("client.3")) 2023-12-08T08:39:00.118 DEBUG:teuthology.orchestra.run.smithi112:> ' 2023-12-08T08:39:00.235 INFO:teuthology.orchestra.run.smithi112.stdout:/var/run/ceph/ceph-client.3.96236.asok 2023-12-08T08:39:00.238 INFO:teuthology.orchestra.run.smithi112.stderr:2023-12-08T08:39:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-08T08:39:00.245 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.96236.asok 2023-12-08T08:39:00.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:00.245 DEBUG:teuthology.orchestra.run.smithi112:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.96236.asok status 2023-12-08T08:39:00.381 INFO:teuthology.orchestra.run.smithi112.stdout:{ 2023-12-08T08:39:00.381 INFO:teuthology.orchestra.run.smithi112.stdout: "metadata": { 2023-12-08T08:39:00.381 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_sha1": "a2cd52c00149b5caff897ea7f79d0529b62620da", 2023-12-08T08:39:00.381 INFO:teuthology.orchestra.run.smithi112.stdout: "ceph_version": "ceph version 19.0.0-46-ga2cd52c0 (a2cd52c00149b5caff897ea7f79d0529b62620da) reef (dev)", 2023-12-08T08:39:00.382 INFO:teuthology.orchestra.run.smithi112.stdout: "entity_id": "3", 2023-12-08T08:39:00.382 INFO:teuthology.orchestra.run.smithi112.stdout: "hostname": "smithi112", 2023-12-08T08:39:00.382 INFO:teuthology.orchestra.run.smithi112.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-12-08T08:39:00.382 INFO:teuthology.orchestra.run.smithi112.stdout: "pid": "96236", 2023-12-08T08:39:00.382 INFO:teuthology.orchestra.run.smithi112.stdout: "root": "/" 2023-12-08T08:39:00.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-12-08T08:39:00.382 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_count": 0, 2023-12-08T08:39:00.382 INFO:teuthology.orchestra.run.smithi112.stdout: "dentry_pinned_count": 0, 2023-12-08T08:39:00.382 INFO:teuthology.orchestra.run.smithi112.stdout: "id": 4783, 2023-12-08T08:39:00.382 INFO:teuthology.orchestra.run.smithi112.stdout: "inst": { 2023-12-08T08:39:00.382 INFO:teuthology.orchestra.run.smithi112.stdout: "name": { 2023-12-08T08:39:00.382 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "client", 2023-12-08T08:39:00.382 INFO:teuthology.orchestra.run.smithi112.stdout: "num": 4783 2023-12-08T08:39:00.382 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-12-08T08:39:00.383 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2023-12-08T08:39:00.383 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2023-12-08T08:39:00.383 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.2:0", 2023-12-08T08:39:00.383 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 712074891 2023-12-08T08:39:00.383 INFO:teuthology.orchestra.run.smithi112.stdout: } 2023-12-08T08:39:00.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-12-08T08:39:00.383 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": { 2023-12-08T08:39:00.383 INFO:teuthology.orchestra.run.smithi112.stdout: "type": "v1", 2023-12-08T08:39:00.383 INFO:teuthology.orchestra.run.smithi112.stdout: "addr": "192.168.0.2:0", 2023-12-08T08:39:00.383 INFO:teuthology.orchestra.run.smithi112.stdout: "nonce": 712074891 2023-12-08T08:39:00.383 INFO:teuthology.orchestra.run.smithi112.stdout: }, 2023-12-08T08:39:00.383 INFO:teuthology.orchestra.run.smithi112.stdout: "inst_str": "client.4783 192.168.0.2:0/712074891", 2023-12-08T08:39:00.383 INFO:teuthology.orchestra.run.smithi112.stdout: "addr_str": "192.168.0.2:0/712074891", 2023-12-08T08:39:00.383 INFO:teuthology.orchestra.run.smithi112.stdout: "inode_count": 1, 2023-12-08T08:39:00.384 INFO:teuthology.orchestra.run.smithi112.stdout: "mds_epoch": 9, 2023-12-08T08:39:00.384 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch": 25, 2023-12-08T08:39:00.384 INFO:teuthology.orchestra.run.smithi112.stdout: "osd_epoch_barrier": 0, 2023-12-08T08:39:00.384 INFO:teuthology.orchestra.run.smithi112.stdout: "blocklisted": false, 2023-12-08T08:39:00.384 INFO:teuthology.orchestra.run.smithi112.stdout: "fs_name": "cephfs" 2023-12-08T08:39:00.384 INFO:teuthology.orchestra.run.smithi112.stdout:} 2023-12-08T08:39:00.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:00.390 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-12-08T08:39:00.390 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-08T08:39:00.410 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-12-08T08:39:00.410 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-08T08:39:00.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:00.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-08T08:39:00.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:00.488 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-12-08T08:39:00.488 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.2 2023-12-08T08:39:00.546 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-12-08T08:39:00.546 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-12-08T08:39:00.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:00.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-12-08T08:39:00.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:00.621 DEBUG:teuthology.orchestra.run.smithi112:> stat --file-system '--printf=%T 2023-12-08T08:39:00.621 DEBUG:teuthology.orchestra.run.smithi112:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-08T08:39:00.637 INFO:teuthology.orchestra.run.smithi112.stdout:fuseblk 2023-12-08T08:39:00.637 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-08T08:39:00.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:00.637 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-08T08:39:00.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:00.704 DEBUG:teuthology.orchestra.run.smithi112:> stat --file-system '--printf=%T 2023-12-08T08:39:00.705 DEBUG:teuthology.orchestra.run.smithi112:> ' -- /home/ubuntu/cephtest/mnt.3 2023-12-08T08:39:00.759 INFO:teuthology.orchestra.run.smithi112.stdout:fuseblk 2023-12-08T08:39:00.759 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-12-08T08:39:00.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:00.759 DEBUG:teuthology.orchestra.run.smithi112:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-12-08T08:39:00.827 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-12-08T08:39:00.843 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.959+0000 7f84c31d7640 1 Processor -- start 2023-12-08T08:39:00.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.959+0000 7f84c31d7640 1 -- start start 2023-12-08T08:39:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.960+0000 7f84c31d7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f84bc0ce270 0x7f84bc06bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.960+0000 7f84c31d7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84bc06c480 0x7f84bc15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.960+0000 7f84c31d7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84bc0ce6f0 con 0x7f84bc0ccd70 2023-12-08T08:39:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.960+0000 7f84c31d7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84bc0d0a60 con 0x7f84bc06c480 2023-12-08T08:39:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.960+0000 7f84c31d7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84bc06c910 con 0x7f84bc0ce270 2023-12-08T08:39:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.960+0000 7f84c1774640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84bc06c480 0x7f84bc15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:00.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.960+0000 7f84c0f73640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f84bc0ccd70 0x7f84bc0cf1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38082/0 (socket says 172.21.15.16:38082) 2023-12-08T08:39:00.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.960+0000 7f84c0f73640 1 -- 172.21.15.16:0/168889294 learned_addr learned my addr 172.21.15.16:0/168889294 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:00.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.960+0000 7f84c1774640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84bc06c480 0x7f84bc15ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53862/0 (socket says 172.21.15.16:53862) 2023-12-08T08:39:00.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.960+0000 7f84bbfff640 1 --2- 172.21.15.16:0/168889294 >> v2:172.21.15.112:3300/0 conn(0x7f84bc0ce270 0x7f84bc06bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:00.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.960+0000 7f84c1774640 1 -- 172.21.15.16:0/168889294 >> v2:172.21.15.112:3300/0 conn(0x7f84bc0ce270 msgr2=0x7f84bc06bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:00.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.960+0000 7f84c1774640 1 --2- 172.21.15.16:0/168889294 >> v2:172.21.15.112:3300/0 conn(0x7f84bc0ce270 0x7f84bc06bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:00.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.960+0000 7f84c1774640 1 -- 172.21.15.16:0/168889294 >> v1:172.21.15.16:6789/0 conn(0x7f84bc0ccd70 legacy=0x7f84bc0cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:00.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.960+0000 7f84c1774640 1 -- 172.21.15.16:0/168889294 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84bc0cf9c0 con 0x7f84bc06c480 2023-12-08T08:39:00.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.961+0000 7f84c1774640 1 --2- 172.21.15.16:0/168889294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc06c480 0x7f84bc15ca50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f84bc070cb0 tx=0x7f84b402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dca98716893de2c server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:00.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.961+0000 7f84bb7fe640 1 -- 172.21.15.16:0/168889294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b402cc60 con 0x7f84bc06c480 2023-12-08T08:39:00.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.961+0000 7f84bb7fe640 1 -- 172.21.15.16:0/168889294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b402cdc0 con 0x7f84bc06c480 2023-12-08T08:39:00.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.961+0000 7f84bb7fe640 1 -- 172.21.15.16:0/168889294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4034660 con 0x7f84bc06c480 2023-12-08T08:39:00.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.962+0000 7f84c31d7640 1 -- 172.21.15.16:0/168889294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc06c480 msgr2=0x7f84bc15ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:00.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.962+0000 7f84c31d7640 1 --2- 172.21.15.16:0/168889294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc06c480 0x7f84bc15ca50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f84bc070cb0 tx=0x7f84b402beb0 comp rx=0 tx=0).stop 2023-12-08T08:39:00.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.962+0000 7f84c31d7640 1 -- 172.21.15.16:0/168889294 shutdown_connections 2023-12-08T08:39:00.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.962+0000 7f84c31d7640 1 --2- 172.21.15.16:0/168889294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc06c480 0x7f84bc15ca50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:00.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.962+0000 7f84c31d7640 1 --2- 172.21.15.16:0/168889294 >> v2:172.21.15.112:3300/0 conn(0x7f84bc0ce270 0x7f84bc06bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:00.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.962+0000 7f84c31d7640 1 -- 172.21.15.16:0/168889294 >> 172.21.15.16:0/168889294 conn(0x7f84bc0d0de0 msgr2=0x7f84bc0d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:00.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.962+0000 7f84c31d7640 1 -- 172.21.15.16:0/168889294 shutdown_connections 2023-12-08T08:39:00.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.962+0000 7f84c31d7640 1 -- 172.21.15.16:0/168889294 wait complete. 2023-12-08T08:39:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.963+0000 7f84c31d7640 1 Processor -- start 2023-12-08T08:39:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.963+0000 7f84c31d7640 1 -- start start 2023-12-08T08:39:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c31d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc06c480 0x7f84bc1f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c31d7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84bc0ccd70 0x7f84bc0c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c31d7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc0ce270 0x7f84bc1f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c31d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc1f6670 con 0x7f84bc06c480 2023-12-08T08:39:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c31d7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc1f3b50 con 0x7f84bc0ce270 2023-12-08T08:39:00.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c31d7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc1f3e30 con 0x7f84bc0ccd70 2023-12-08T08:39:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c0f73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc06c480 0x7f84bc1f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c0f73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc06c480 0x7f84bc1f70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53878/0 (socket says 172.21.15.16:53878) 2023-12-08T08:39:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c0f73640 1 -- 172.21.15.16:0/4073671362 learned_addr learned my addr 172.21.15.16:0/4073671362 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c1774640 1 --2- 172.21.15.16:0/4073671362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc0ce270 0x7f84bc1f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84bbfff640 1 --2- 172.21.15.16:0/4073671362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84bc0ccd70 0x7f84bc0c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c0f73640 1 -- 172.21.15.16:0/4073671362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84bc0ccd70 msgr2=0x7f84bc0c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c0f73640 1 --2- 172.21.15.16:0/4073671362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84bc0ccd70 0x7f84bc0c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c0f73640 1 -- 172.21.15.16:0/4073671362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc0ce270 msgr2=0x7f84bc1f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c0f73640 1 --2- 172.21.15.16:0/4073671362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc0ce270 0x7f84bc1f3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.964+0000 7f84c0f73640 1 -- 172.21.15.16:0/4073671362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b4009a70 con 0x7f84bc06c480 2023-12-08T08:39:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.965+0000 7f84c0f73640 1 --2- 172.21.15.16:0/4073671362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc06c480 0x7f84bc1f70c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c4a0 tx=0x7f84b000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.965+0000 7f84b97fa640 1 -- 172.21.15.16:0/4073671362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0005ce0 con 0x7f84bc06c480 2023-12-08T08:39:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.965+0000 7f84b97fa640 1 -- 172.21.15.16:0/4073671362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b0005e40 con 0x7f84bc06c480 2023-12-08T08:39:00.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.965+0000 7f84c31d7640 1 -- 172.21.15.16:0/4073671362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84bc1f68c0 con 0x7f84bc06c480 2023-12-08T08:39:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.965+0000 7f84b97fa640 1 -- 172.21.15.16:0/4073671362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0007270 con 0x7f84bc06c480 2023-12-08T08:39:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.965+0000 7f84c31d7640 1 -- 172.21.15.16:0/4073671362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc15a080 con 0x7f84bc06c480 2023-12-08T08:39:00.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.966+0000 7f84c31d7640 1 -- 172.21.15.16:0/4073671362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84bc1f6de0 con 0x7f84bc06c480 2023-12-08T08:39:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.966+0000 7f84b97fa640 1 -- 172.21.15.16:0/4073671362 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f84b0007490 con 0x7f84bc06c480 2023-12-08T08:39:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.967+0000 7f84b97fa640 1 --2- 172.21.15.16:0/4073671362 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f84a006ae40 0x7f84a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.967+0000 7f84b97fa640 1 -- 172.21.15.16:0/4073671362 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f84b0088e00 con 0x7f84bc06c480 2023-12-08T08:39:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.967+0000 7f84bbfff640 1 --2- 172.21.15.16:0/4073671362 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f84a006ae40 0x7f84a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:00.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.967+0000 7f84bbfff640 1 --2- 172.21.15.16:0/4073671362 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f84a006ae40 0x7f84a006d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0af4c0 tx=0x7f84a8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:00.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:00.971+0000 7f84b97fa640 1 -- 172.21.15.16:0/4073671362 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84bc1f6de0 con 0x7f84bc06c480 2023-12-08T08:39:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.151+0000 7f84c31d7640 1 -- 172.21.15.16:0/4073671362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84bc1f6de0 con 0x7f84bc06c480 2023-12-08T08:39:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.152+0000 7f84b97fa640 1 -- 172.21.15.16:0/4073671362 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7f84bc1f6de0 con 0x7f84bc06c480 2023-12-08T08:39:01.156 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:01.156 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:01.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.154+0000 7f84c31d7640 1 -- 172.21.15.16:0/4073671362 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f84a006ae40 msgr2=0x7f84a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.154+0000 7f84c31d7640 1 --2- 172.21.15.16:0/4073671362 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f84a006ae40 0x7f84a006d300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f84bc0af4c0 tx=0x7f84a8008040 comp rx=0 tx=0).stop 2023-12-08T08:39:01.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.154+0000 7f84c31d7640 1 -- 172.21.15.16:0/4073671362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc06c480 msgr2=0x7f84bc1f70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.154+0000 7f84c31d7640 1 --2- 172.21.15.16:0/4073671362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc06c480 0x7f84bc1f70c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f84b000c4a0 tx=0x7f84b000c960 comp rx=0 tx=0).stop 2023-12-08T08:39:01.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.154+0000 7f84c31d7640 1 -- 172.21.15.16:0/4073671362 shutdown_connections 2023-12-08T08:39:01.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.154+0000 7f84c31d7640 1 --2- 172.21.15.16:0/4073671362 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f84a006ae40 0x7f84a006d300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.154+0000 7f84c31d7640 1 --2- 172.21.15.16:0/4073671362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f84bc0ce270 0x7f84bc1f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.154+0000 7f84c31d7640 1 --2- 172.21.15.16:0/4073671362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f84bc0ccd70 0x7f84bc0c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.154+0000 7f84c31d7640 1 --2- 172.21.15.16:0/4073671362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84bc06c480 0x7f84bc1f70c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.154+0000 7f84c31d7640 1 -- 172.21.15.16:0/4073671362 >> 172.21.15.16:0/4073671362 conn(0x7f84bc0d0de0 msgr2=0x7f84bc0afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:01.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.154+0000 7f84c31d7640 1 -- 172.21.15.16:0/4073671362 shutdown_connections 2023-12-08T08:39:01.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.154+0000 7f84c31d7640 1 -- 172.21.15.16:0/4073671362 wait complete. 2023-12-08T08:39:01.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:39:01.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.275+0000 7f349301c640 1 Processor -- start 2023-12-08T08:39:01.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.275+0000 7f349301c640 1 -- start start 2023-12-08T08:39:01.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.275+0000 7f349301c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348c0cfa30 0x7f348c0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.275+0000 7f349301c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f348c0cfe10 0x7f348c0cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.275+0000 7f349301c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348c0c6be0 con 0x7f348c0b4150 2023-12-08T08:39:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.275+0000 7f349301c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c0d3ff0 con 0x7f348c0cfe10 2023-12-08T08:39:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.275+0000 7f349301c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0b4860 con 0x7f348c0cfa30 2023-12-08T08:39:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.276+0000 7f34915b9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f348c0cfe10 0x7f348c0cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.276+0000 7f34915b9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f348c0cfe10 0x7f348c0cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53892/0 (socket says 172.21.15.16:53892) 2023-12-08T08:39:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.276+0000 7f348bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f348c0b4150 0x7f348c0b3460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38098/0 (socket says 172.21.15.16:38098) 2023-12-08T08:39:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.276+0000 7f34915b9640 1 -- 172.21.15.16:0/3863896482 learned_addr learned my addr 172.21.15.16:0/3863896482 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.276+0000 7f3490db8640 1 --2- 172.21.15.16:0/3863896482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348c0cfa30 0x7f348c0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.276+0000 7f34915b9640 1 -- 172.21.15.16:0/3863896482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348c0cfa30 msgr2=0x7f348c0b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.276+0000 7f34915b9640 1 --2- 172.21.15.16:0/3863896482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348c0cfa30 0x7f348c0b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.276+0000 7f34915b9640 1 -- 172.21.15.16:0/3863896482 >> v1:172.21.15.16:6789/0 conn(0x7f348c0b4150 legacy=0x7f348c0b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.276+0000 7f34915b9640 1 -- 172.21.15.16:0/3863896482 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c0d3d50 con 0x7f348c0cfe10 2023-12-08T08:39:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.277+0000 7f34915b9640 1 --2- 172.21.15.16:0/3863896482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f348c0cfe10 0x7f348c0cdf50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3484009e50 tx=0x7f348402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868af87201507c81 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.277+0000 7f348b7fe640 1 -- 172.21.15.16:0/3863896482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348402ca70 con 0x7f348c0cfe10 2023-12-08T08:39:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.277+0000 7f348b7fe640 1 -- 172.21.15.16:0/3863896482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348402cbd0 con 0x7f348c0cfe10 2023-12-08T08:39:01.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.277+0000 7f348b7fe640 1 -- 172.21.15.16:0/3863896482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34840343f0 con 0x7f348c0cfe10 2023-12-08T08:39:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.277+0000 7f349301c640 1 -- 172.21.15.16:0/3863896482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f348c0cfe10 msgr2=0x7f348c0cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.277+0000 7f349301c640 1 --2- 172.21.15.16:0/3863896482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f348c0cfe10 0x7f348c0cdf50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3484009e50 tx=0x7f348402bed0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.277+0000 7f349301c640 1 -- 172.21.15.16:0/3863896482 shutdown_connections 2023-12-08T08:39:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.277+0000 7f349301c640 1 --2- 172.21.15.16:0/3863896482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f348c0cfe10 0x7f348c0cdf50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.277+0000 7f349301c640 1 --2- 172.21.15.16:0/3863896482 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348c0cfa30 0x7f348c0b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.277+0000 7f349301c640 1 -- 172.21.15.16:0/3863896482 >> 172.21.15.16:0/3863896482 conn(0x7f348c0d3090 msgr2=0x7f348c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.277+0000 7f349301c640 1 -- 172.21.15.16:0/3863896482 shutdown_connections 2023-12-08T08:39:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.277+0000 7f349301c640 1 -- 172.21.15.16:0/3863896482 wait complete. 2023-12-08T08:39:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.279+0000 7f349301c640 1 Processor -- start 2023-12-08T08:39:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.279+0000 7f349301c640 1 -- start start 2023-12-08T08:39:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.279+0000 7f349301c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f348c0b4150 0x7f348c159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.279+0000 7f349301c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348c0cfa30 0x7f348c161580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.279+0000 7f349301c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f348c0cfe10 0x7f348c1f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.279+0000 7f349301c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0cced0 con 0x7f348c0cfe10 2023-12-08T08:39:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.279+0000 7f349301c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0cb190 con 0x7f348c0b4150 2023-12-08T08:39:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.279+0000 7f349301c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0cb470 con 0x7f348c0cfa30 2023-12-08T08:39:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f34915b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f348c0cfe10 0x7f348c1f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f34915b9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f348c0cfe10 0x7f348c1f39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53902/0 (socket says 172.21.15.16:53902) 2023-12-08T08:39:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f34915b9640 1 -- 172.21.15.16:0/306234472 learned_addr learned my addr 172.21.15.16:0/306234472 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f348bfff640 1 --2- 172.21.15.16:0/306234472 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348c0cfa30 0x7f348c161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f3490db8640 1 --2- 172.21.15.16:0/306234472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f348c0b4150 0x7f348c159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f34915b9640 1 -- 172.21.15.16:0/306234472 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348c0cfa30 msgr2=0x7f348c161580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f34915b9640 1 --2- 172.21.15.16:0/306234472 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348c0cfa30 0x7f348c161580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f34915b9640 1 -- 172.21.15.16:0/306234472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f348c0b4150 msgr2=0x7f348c159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f34915b9640 1 --2- 172.21.15.16:0/306234472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f348c0b4150 0x7f348c159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f34915b9640 1 -- 172.21.15.16:0/306234472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3484009b30 con 0x7f348c0cfe10 2023-12-08T08:39:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f34915b9640 1 --2- 172.21.15.16:0/306234472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f348c0cfe10 0x7f348c1f39a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3484009e50 tx=0x7f3484004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f34897fa640 1 -- 172.21.15.16:0/306234472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484009510 con 0x7f348c0cfe10 2023-12-08T08:39:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f34897fa640 1 -- 172.21.15.16:0/306234472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348403e450 con 0x7f348c0cfe10 2023-12-08T08:39:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f349301c640 1 -- 172.21.15.16:0/306234472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348c0b19b0 con 0x7f348c0cfe10 2023-12-08T08:39:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f34897fa640 1 -- 172.21.15.16:0/306234472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348403e800 con 0x7f348c0cfe10 2023-12-08T08:39:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.280+0000 7f349301c640 1 -- 172.21.15.16:0/306234472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c159d60 con 0x7f348c0cfe10 2023-12-08T08:39:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.281+0000 7f349301c640 1 -- 172.21.15.16:0/306234472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348c067430 con 0x7f348c0cfe10 2023-12-08T08:39:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.282+0000 7f34897fa640 1 -- 172.21.15.16:0/306234472 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f348403e960 con 0x7f348c0cfe10 2023-12-08T08:39:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.282+0000 7f34897fa640 1 --2- 172.21.15.16:0/306234472 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f347006ae40 0x7f347006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.282+0000 7f34897fa640 1 -- 172.21.15.16:0/306234472 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f34840ac7b0 con 0x7f348c0cfe10 2023-12-08T08:39:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.283+0000 7f3490db8640 1 --2- 172.21.15.16:0/306234472 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f347006ae40 0x7f347006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.283+0000 7f3490db8640 1 --2- 172.21.15.16:0/306234472 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f347006ae40 0x7f347006d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3480009a30 tx=0x7f3480008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.287+0000 7f34897fa640 1 -- 172.21.15.16:0/306234472 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f34840773e0 con 0x7f348c0cfe10 2023-12-08T08:39:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.460+0000 7f349301c640 1 -- 172.21.15.16:0/306234472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f348c0c69c0 con 0x7f348c0cfe10 2023-12-08T08:39:01.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.460+0000 7f34897fa640 1 -- 172.21.15.16:0/306234472 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f348407aa00 con 0x7f348c0cfe10 2023-12-08T08:39:01.464 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.462+0000 7f349301c640 1 -- 172.21.15.16:0/306234472 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f347006ae40 msgr2=0x7f347006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.462+0000 7f349301c640 1 --2- 172.21.15.16:0/306234472 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f347006ae40 0x7f347006d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3480009a30 tx=0x7f3480008040 comp rx=0 tx=0).stop 2023-12-08T08:39:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.462+0000 7f349301c640 1 -- 172.21.15.16:0/306234472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f348c0cfe10 msgr2=0x7f348c1f39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.462+0000 7f349301c640 1 --2- 172.21.15.16:0/306234472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f348c0cfe10 0x7f348c1f39a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3484009e50 tx=0x7f3484004a40 comp rx=0 tx=0).stop 2023-12-08T08:39:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.462+0000 7f349301c640 1 -- 172.21.15.16:0/306234472 shutdown_connections 2023-12-08T08:39:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.462+0000 7f349301c640 1 --2- 172.21.15.16:0/306234472 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f347006ae40 0x7f347006d300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.462+0000 7f349301c640 1 --2- 172.21.15.16:0/306234472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f348c0cfe10 0x7f348c1f39a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.462+0000 7f349301c640 1 --2- 172.21.15.16:0/306234472 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f348c0cfa30 0x7f348c161580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.462+0000 7f349301c640 1 --2- 172.21.15.16:0/306234472 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f348c0b4150 0x7f348c159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.462+0000 7f349301c640 1 -- 172.21.15.16:0/306234472 >> 172.21.15.16:0/306234472 conn(0x7f348c0d3090 msgr2=0x7f348c15cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.462+0000 7f349301c640 1 -- 172.21.15.16:0/306234472 shutdown_connections 2023-12-08T08:39:01.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.462+0000 7f349301c640 1 -- 172.21.15.16:0/306234472 wait complete. 2023-12-08T08:39:01.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:39:01.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.582+0000 7f3d32b25640 1 Processor -- start 2023-12-08T08:39:01.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.582+0000 7f3d32b25640 1 -- start start 2023-12-08T08:39:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.583+0000 7f3d32b25640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2c0cfa30 0x7f3d2c0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.583+0000 7f3d32b25640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3d2c0cfe10 0x7f3d2c0cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.583+0000 7f3d32b25640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d2c0c6be0 con 0x7f3d2c0b4150 2023-12-08T08:39:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.583+0000 7f3d32b25640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d2c0d3ff0 con 0x7f3d2c0cfe10 2023-12-08T08:39:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.583+0000 7f3d32b25640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c0b4860 con 0x7f3d2c0cfa30 2023-12-08T08:39:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.584+0000 7f3d310c2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3d2c0cfe10 0x7f3d2c0cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.584+0000 7f3d308c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2c0cfa30 0x7f3d2c0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.584+0000 7f3d308c1640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2c0cfa30 0x7f3d2c0b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:56660/0 (socket says 172.21.15.16:56660) 2023-12-08T08:39:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.584+0000 7f3d310c2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3d2c0cfe10 0x7f3d2c0cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:54388/0 (socket says 172.21.15.16:54388) 2023-12-08T08:39:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.584+0000 7f3d2bfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f3d2c0b4150 0x7f3d2c0b3460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:33914/0 (socket says 172.21.15.16:33914) 2023-12-08T08:39:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.584+0000 7f3d308c1640 1 -- 172.21.15.16:0/2966977644 learned_addr learned my addr 172.21.15.16:0/2966977644 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.584+0000 7f3d308c1640 1 -- 172.21.15.16:0/2966977644 >> v2:172.21.15.112:3300/0 conn(0x7f3d2c0cfe10 msgr2=0x7f3d2c0cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.584+0000 7f3d308c1640 1 --2- 172.21.15.16:0/2966977644 >> v2:172.21.15.112:3300/0 conn(0x7f3d2c0cfe10 0x7f3d2c0cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.584+0000 7f3d308c1640 1 -- 172.21.15.16:0/2966977644 >> v1:172.21.15.112:6789/0 conn(0x7f3d2c0b4150 legacy=0x7f3d2c0b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.584+0000 7f3d2b7fe640 1 -- 172.21.15.16:0/2966977644 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594028920 0 0) 0x7f3d2c0c6be0 con 0x7f3d2c0b4150 2023-12-08T08:39:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.584+0000 7f3d308c1640 1 -- 172.21.15.16:0/2966977644 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c0d3d50 con 0x7f3d2c0cfa30 2023-12-08T08:39:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.585+0000 7f3d308c1640 1 --2- 172.21.15.16:0/2966977644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2c0cfa30 0x7f3d2c0b2f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009e50 tx=0x7f3d2402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aff0c8fb92758cdc server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.585+0000 7f3d2b7fe640 1 -- 172.21.15.16:0/2966977644 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2402ca70 con 0x7f3d2c0cfa30 2023-12-08T08:39:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.585+0000 7f3d2b7fe640 1 -- 172.21.15.16:0/2966977644 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2402cbd0 con 0x7f3d2c0cfa30 2023-12-08T08:39:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.585+0000 7f3d2b7fe640 1 -- 172.21.15.16:0/2966977644 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d240343f0 con 0x7f3d2c0cfa30 2023-12-08T08:39:01.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.585+0000 7f3d32b25640 1 -- 172.21.15.16:0/2966977644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2c0cfa30 msgr2=0x7f3d2c0b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.585+0000 7f3d32b25640 1 --2- 172.21.15.16:0/2966977644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2c0cfa30 0x7f3d2c0b2f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009e50 tx=0x7f3d2402bed0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.586+0000 7f3d32b25640 1 -- 172.21.15.16:0/2966977644 shutdown_connections 2023-12-08T08:39:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.586+0000 7f3d32b25640 1 --2- 172.21.15.16:0/2966977644 >> v2:172.21.15.112:3300/0 conn(0x7f3d2c0cfe10 0x7f3d2c0cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.586+0000 7f3d32b25640 1 --2- 172.21.15.16:0/2966977644 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2c0cfa30 0x7f3d2c0b2f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.586+0000 7f3d32b25640 1 -- 172.21.15.16:0/2966977644 >> 172.21.15.16:0/2966977644 conn(0x7f3d2c0d3090 msgr2=0x7f3d2c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.586+0000 7f3d32b25640 1 -- 172.21.15.16:0/2966977644 shutdown_connections 2023-12-08T08:39:01.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.586+0000 7f3d32b25640 1 -- 172.21.15.16:0/2966977644 wait complete. 2023-12-08T08:39:01.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.587+0000 7f3d32b25640 1 Processor -- start 2023-12-08T08:39:01.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.587+0000 7f3d32b25640 1 -- start start 2023-12-08T08:39:01.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d32b25640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d2c0b4150 0x7f3d2c0a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d32b25640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2c0cfa30 0x7f3d2c15a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d32b25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d2c0cfe10 0x7f3d2c09dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d32b25640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c09f860 con 0x7f3d2c0cfe10 2023-12-08T08:39:01.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d32b25640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c09fb40 con 0x7f3d2c0b4150 2023-12-08T08:39:01.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d32b25640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2c09fe20 con 0x7f3d2c0cfa30 2023-12-08T08:39:01.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d310c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d2c0cfe10 0x7f3d2c09dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d310c2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d2c0cfe10 0x7f3d2c09dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53908/0 (socket says 172.21.15.16:53908) 2023-12-08T08:39:01.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d310c2640 1 -- 172.21.15.16:0/2279445373 learned_addr learned my addr 172.21.15.16:0/2279445373 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:01.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d308c1640 1 --2- 172.21.15.16:0/2279445373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d2c0b4150 0x7f3d2c0a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d2bfff640 1 --2- 172.21.15.16:0/2279445373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2c0cfa30 0x7f3d2c15a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d310c2640 1 -- 172.21.15.16:0/2279445373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2c0cfa30 msgr2=0x7f3d2c15a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d310c2640 1 --2- 172.21.15.16:0/2279445373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2c0cfa30 0x7f3d2c15a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d310c2640 1 -- 172.21.15.16:0/2279445373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d2c0b4150 msgr2=0x7f3d2c0a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d310c2640 1 --2- 172.21.15.16:0/2279445373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d2c0b4150 0x7f3d2c0a0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d310c2640 1 -- 172.21.15.16:0/2279445373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d24009b30 con 0x7f3d2c0cfe10 2023-12-08T08:39:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d310c2640 1 --2- 172.21.15.16:0/2279445373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d2c0cfe10 0x7f3d2c09dad0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00e7f0 tx=0x7f3d1c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d297fa640 1 -- 172.21.15.16:0/2279445373 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c009510 con 0x7f3d2c0cfe10 2023-12-08T08:39:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d297fa640 1 -- 172.21.15.16:0/2279445373 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1c00ee40 con 0x7f3d2c0cfe10 2023-12-08T08:39:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d32b25640 1 -- 172.21.15.16:0/2279445373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2c0a0100 con 0x7f3d2c0cfe10 2023-12-08T08:39:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d297fa640 1 -- 172.21.15.16:0/2279445373 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c00cb80 con 0x7f3d2c0cfe10 2023-12-08T08:39:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.588+0000 7f3d32b25640 1 -- 172.21.15.16:0/2279445373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2c15ab50 con 0x7f3d2c0cfe10 2023-12-08T08:39:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.589+0000 7f3d32b25640 1 -- 172.21.15.16:0/2279445373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2c0a0620 con 0x7f3d2c0cfe10 2023-12-08T08:39:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.590+0000 7f3d297fa640 1 -- 172.21.15.16:0/2279445373 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3d1c00cce0 con 0x7f3d2c0cfe10 2023-12-08T08:39:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.590+0000 7f3d297fa640 1 --2- 172.21.15.16:0/2279445373 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d0c06ae70 0x7f3d0c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.590+0000 7f3d297fa640 1 -- 172.21.15.16:0/2279445373 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f3d1c089af0 con 0x7f3d2c0cfe10 2023-12-08T08:39:01.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.591+0000 7f3d308c1640 1 --2- 172.21.15.16:0/2279445373 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d0c06ae70 0x7f3d0c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.591+0000 7f3d308c1640 1 --2- 172.21.15.16:0/2279445373 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d0c06ae70 0x7f3d0c06d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3d24035000 tx=0x7f3d24030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:01.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.594+0000 7f3d297fa640 1 -- 172.21.15.16:0/2279445373 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d1c0547b0 con 0x7f3d2c0cfe10 2023-12-08T08:39:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.769+0000 7f3d32b25640 1 -- 172.21.15.16:0/2279445373 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3d2c067430 con 0x7f3d2c0cfe10 2023-12-08T08:39:01.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.769+0000 7f3d297fa640 1 -- 172.21.15.16:0/2279445373 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3d1c057dd0 con 0x7f3d2c0cfe10 2023-12-08T08:39:01.772 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.771+0000 7f3d32b25640 1 -- 172.21.15.16:0/2279445373 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d0c06ae70 msgr2=0x7f3d0c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.771+0000 7f3d32b25640 1 --2- 172.21.15.16:0/2279445373 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d0c06ae70 0x7f3d0c06d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3d24035000 tx=0x7f3d24030000 comp rx=0 tx=0).stop 2023-12-08T08:39:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.771+0000 7f3d32b25640 1 -- 172.21.15.16:0/2279445373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d2c0cfe10 msgr2=0x7f3d2c09dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.771+0000 7f3d32b25640 1 --2- 172.21.15.16:0/2279445373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d2c0cfe10 0x7f3d2c09dad0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00e7f0 tx=0x7f3d1c00ebf0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.771+0000 7f3d32b25640 1 -- 172.21.15.16:0/2279445373 shutdown_connections 2023-12-08T08:39:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.771+0000 7f3d32b25640 1 --2- 172.21.15.16:0/2279445373 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d0c06ae70 0x7f3d0c06d330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.771+0000 7f3d32b25640 1 --2- 172.21.15.16:0/2279445373 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d2c0cfe10 0x7f3d2c09dad0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.771+0000 7f3d32b25640 1 --2- 172.21.15.16:0/2279445373 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d2c0cfa30 0x7f3d2c15a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.771+0000 7f3d32b25640 1 --2- 172.21.15.16:0/2279445373 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d2c0b4150 0x7f3d2c0a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.771+0000 7f3d32b25640 1 -- 172.21.15.16:0/2279445373 >> 172.21.15.16:0/2279445373 conn(0x7f3d2c0d3090 msgr2=0x7f3d2c0cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.771+0000 7f3d32b25640 1 -- 172.21.15.16:0/2279445373 shutdown_connections 2023-12-08T08:39:01.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.771+0000 7f3d32b25640 1 -- 172.21.15.16:0/2279445373 wait complete. 2023-12-08T08:39:01.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:39:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.892+0000 7f9b923e5640 1 Processor -- start 2023-12-08T08:39:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.892+0000 7f9b923e5640 1 -- start start 2023-12-08T08:39:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.893+0000 7f9b923e5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b8c15d590 0x7f9b8c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.893+0000 7f9b923e5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8c06d650 con 0x7f9b8c0ae350 2023-12-08T08:39:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.893+0000 7f9b923e5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8c0adbe0 con 0x7f9b8c0afeb0 2023-12-08T08:39:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.893+0000 7f9b923e5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8c15d970 con 0x7f9b8c15d590 2023-12-08T08:39:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.893+0000 7f9b8bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b8c15d590 0x7f9b8c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.893+0000 7f9b8bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9b8c15d590 0x7f9b8c0ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53910/0 (socket says 172.21.15.16:53910) 2023-12-08T08:39:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.893+0000 7f9b8bfff640 1 -- 172.21.15.16:0/675884451 learned_addr learned my addr 172.21.15.16:0/675884451 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.893+0000 7f9b90982640 1 --1- 172.21.15.16:0/675884451 >> v1:172.21.15.16:6789/0 conn(0x7f9b8c0ae350 0x7f9b8c0ae7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38104/0 (socket says 172.21.15.16:38104) 2023-12-08T08:39:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.893+0000 7f9b8bfff640 1 -- 172.21.15.16:0/675884451 >> v1:172.21.15.112:6789/0 conn(0x7f9b8c0afeb0 legacy=0x7f9b8c0cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.893+0000 7f9b8bfff640 1 -- 172.21.15.16:0/675884451 >> v1:172.21.15.16:6789/0 conn(0x7f9b8c0ae350 legacy=0x7f9b8c0ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.893+0000 7f9b8bfff640 1 -- 172.21.15.16:0/675884451 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c06c810 con 0x7f9b8c15d590 2023-12-08T08:39:01.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.893+0000 7f9b8affd640 1 -- 172.21.15.16:0/675884451 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962708560 0 0) 0x7f9b84001490 con 0x7f9b8c0ae350 2023-12-08T08:39:01.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.894+0000 7f9b8bfff640 1 --2- 172.21.15.16:0/675884451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c15d590 0x7f9b8c0ced10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009870 tx=0x7f9b7402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4a6e2bafe7cd1f server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:01.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.894+0000 7f9b8affd640 1 -- 172.21.15.16:0/675884451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74030860 con 0x7f9b8c15d590 2023-12-08T08:39:01.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.894+0000 7f9b8affd640 1 -- 172.21.15.16:0/675884451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b740309c0 con 0x7f9b8c15d590 2023-12-08T08:39:01.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.894+0000 7f9b8affd640 1 -- 172.21.15.16:0/675884451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74030d50 con 0x7f9b8c15d590 2023-12-08T08:39:01.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.895+0000 7f9b923e5640 1 -- 172.21.15.16:0/675884451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c15d590 msgr2=0x7f9b8c0ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.895+0000 7f9b923e5640 1 --2- 172.21.15.16:0/675884451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c15d590 0x7f9b8c0ced10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009870 tx=0x7f9b7402b9a0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.895+0000 7f9b923e5640 1 -- 172.21.15.16:0/675884451 shutdown_connections 2023-12-08T08:39:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.895+0000 7f9b923e5640 1 --2- 172.21.15.16:0/675884451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c15d590 0x7f9b8c0ced10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.895+0000 7f9b923e5640 1 -- 172.21.15.16:0/675884451 >> 172.21.15.16:0/675884451 conn(0x7f9b8c0d0e10 msgr2=0x7f9b8c0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.895+0000 7f9b923e5640 1 -- 172.21.15.16:0/675884451 shutdown_connections 2023-12-08T08:39:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.895+0000 7f9b923e5640 1 -- 172.21.15.16:0/675884451 wait complete. 2023-12-08T08:39:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.896+0000 7f9b923e5640 1 Processor -- start 2023-12-08T08:39:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.896+0000 7f9b923e5640 1 -- start start 2023-12-08T08:39:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b923e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b8c0ae350 0x7f9b8c0cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b923e5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b8c0afeb0 0x7f9b8c1580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b923e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c15d590 0x7f9b8c1605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b923e5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c0cc6a0 con 0x7f9b8c15d590 2023-12-08T08:39:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b923e5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c160b00 con 0x7f9b8c0ae350 2023-12-08T08:39:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b923e5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c0cba90 con 0x7f9b8c0afeb0 2023-12-08T08:39:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b90982640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c15d590 0x7f9b8c1605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b90982640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c15d590 0x7f9b8c1605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53916/0 (socket says 172.21.15.16:53916) 2023-12-08T08:39:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b90982640 1 -- 172.21.15.16:0/3310558570 learned_addr learned my addr 172.21.15.16:0/3310558570 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b8bfff640 1 --2- 172.21.15.16:0/3310558570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b8c0ae350 0x7f9b8c0cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b8b7fe640 1 --2- 172.21.15.16:0/3310558570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b8c0afeb0 0x7f9b8c1580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b90982640 1 -- 172.21.15.16:0/3310558570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b8c0afeb0 msgr2=0x7f9b8c1580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b90982640 1 --2- 172.21.15.16:0/3310558570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b8c0afeb0 0x7f9b8c1580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b90982640 1 -- 172.21.15.16:0/3310558570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b8c0ae350 msgr2=0x7f9b8c0cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b90982640 1 --2- 172.21.15.16:0/3310558570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b8c0ae350 0x7f9b8c0cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b90982640 1 -- 172.21.15.16:0/3310558570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b740094d0 con 0x7f9b8c15d590 2023-12-08T08:39:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.897+0000 7f9b90982640 1 --2- 172.21.15.16:0/3310558570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c15d590 0x7f9b8c1605c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400c7d0 tx=0x7f9b8400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.898+0000 7f9b88ff9640 1 -- 172.21.15.16:0/3310558570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8400a640 con 0x7f9b8c15d590 2023-12-08T08:39:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.898+0000 7f9b88ff9640 1 -- 172.21.15.16:0/3310558570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8400a7c0 con 0x7f9b8c15d590 2023-12-08T08:39:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.898+0000 7f9b88ff9640 1 -- 172.21.15.16:0/3310558570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8400abb0 con 0x7f9b8c15d590 2023-12-08T08:39:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.898+0000 7f9b923e5640 1 -- 172.21.15.16:0/3310558570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c0cbd70 con 0x7f9b8c15d590 2023-12-08T08:39:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.898+0000 7f9b923e5640 1 -- 172.21.15.16:0/3310558570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c0cc330 con 0x7f9b8c15d590 2023-12-08T08:39:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.899+0000 7f9b923e5640 1 -- 172.21.15.16:0/3310558570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8c0cc810 con 0x7f9b8c15d590 2023-12-08T08:39:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.899+0000 7f9b88ff9640 1 -- 172.21.15.16:0/3310558570 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9b8400ad10 con 0x7f9b8c15d590 2023-12-08T08:39:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.900+0000 7f9b88ff9640 1 --2- 172.21.15.16:0/3310558570 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b6c06ae40 0x7f9b6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.900+0000 7f9b88ff9640 1 -- 172.21.15.16:0/3310558570 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f9b84088dd0 con 0x7f9b8c15d590 2023-12-08T08:39:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.900+0000 7f9b8bfff640 1 --2- 172.21.15.16:0/3310558570 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b6c06ae40 0x7f9b6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.900+0000 7f9b8bfff640 1 --2- 172.21.15.16:0/3310558570 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b6c06ae40 0x7f9b6c06d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9b74036000 tx=0x7f9b74008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:01.904+0000 7f9b88ff9640 1 -- 172.21.15.16:0/3310558570 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b84053ab0 con 0x7f9b8c15d590 2023-12-08T08:39:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.080+0000 7f9b923e5640 1 -- 172.21.15.16:0/3310558570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9b8c06d650 con 0x7f9b8c15d590 2023-12-08T08:39:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.080+0000 7f9b88ff9640 1 -- 172.21.15.16:0/3310558570 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9b8c06d650 con 0x7f9b8c15d590 2023-12-08T08:39:02.084 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:39:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.083+0000 7f9b923e5640 1 -- 172.21.15.16:0/3310558570 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b6c06ae40 msgr2=0x7f9b6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.083+0000 7f9b923e5640 1 --2- 172.21.15.16:0/3310558570 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b6c06ae40 0x7f9b6c06d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9b74036000 tx=0x7f9b74008030 comp rx=0 tx=0).stop 2023-12-08T08:39:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.083+0000 7f9b923e5640 1 -- 172.21.15.16:0/3310558570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c15d590 msgr2=0x7f9b8c1605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.083+0000 7f9b923e5640 1 --2- 172.21.15.16:0/3310558570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c15d590 0x7f9b8c1605c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9b8400c7d0 tx=0x7f9b8400cc90 comp rx=0 tx=0).stop 2023-12-08T08:39:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.083+0000 7f9b923e5640 1 -- 172.21.15.16:0/3310558570 shutdown_connections 2023-12-08T08:39:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.083+0000 7f9b923e5640 1 --2- 172.21.15.16:0/3310558570 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b6c06ae40 0x7f9b6c06d300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.083+0000 7f9b923e5640 1 --2- 172.21.15.16:0/3310558570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b8c15d590 0x7f9b8c1605c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.083+0000 7f9b923e5640 1 --2- 172.21.15.16:0/3310558570 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b8c0afeb0 0x7f9b8c1580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.083+0000 7f9b923e5640 1 --2- 172.21.15.16:0/3310558570 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b8c0ae350 0x7f9b8c0cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.083+0000 7f9b923e5640 1 -- 172.21.15.16:0/3310558570 >> 172.21.15.16:0/3310558570 conn(0x7f9b8c0d0e10 msgr2=0x7f9b8c0b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.083+0000 7f9b923e5640 1 -- 172.21.15.16:0/3310558570 shutdown_connections 2023-12-08T08:39:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.083+0000 7f9b923e5640 1 -- 172.21.15.16:0/3310558570 wait complete. 2023-12-08T08:39:02.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:39:02.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.203+0000 7f176207c640 1 Processor -- start 2023-12-08T08:39:02.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.203+0000 7f176207c640 1 -- start start 2023-12-08T08:39:02.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.203+0000 7f176207c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f175c157a20 0x7f175c15d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.203+0000 7f176207c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f175c157e00 con 0x7f175c0cb370 2023-12-08T08:39:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.203+0000 7f176207c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f175c06dd20 con 0x7f175c06d7d0 2023-12-08T08:39:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.203+0000 7f176207c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175c070cd0 con 0x7f175c157a20 2023-12-08T08:39:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f175c0cb370 0x7f175c15df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38120/0 (socket says 172.21.15.16:38120) 2023-12-08T08:39:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175affd640 1 -- 172.21.15.16:0/2132530362 learned_addr learned my addr 172.21.15.16:0/2132530362 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175b7fe640 1 --2- 172.21.15.16:0/2132530362 >> v2:172.21.15.112:3300/0 conn(0x7f175c157a20 0x7f175c15d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 -- 172.21.15.16:0/2132530362 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901515405 0 0) 0x7f175c157e00 con 0x7f175c0cb370 2023-12-08T08:39:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 -- 172.21.15.16:0/2132530362 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1744003520 con 0x7f175c0cb370 2023-12-08T08:39:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 -- 172.21.15.16:0/2132530362 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4046746931 0 0) 0x7f1744003520 con 0x7f175c0cb370 2023-12-08T08:39:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 -- 172.21.15.16:0/2132530362 >> v2:172.21.15.112:3300/0 conn(0x7f175c157a20 msgr2=0x7f175c15d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 --2- 172.21.15.16:0/2132530362 >> v2:172.21.15.112:3300/0 conn(0x7f175c157a20 0x7f175c15d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 -- 172.21.15.16:0/2132530362 >> v1:172.21.15.112:6789/0 conn(0x7f175c06d7d0 legacy=0x7f175c0b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 -- 172.21.15.16:0/2132530362 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c070fb0 con 0x7f175c0cb370 2023-12-08T08:39:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 -- 172.21.15.16:0/2132530362 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f1750001670 con 0x7f175c0cb370 2023-12-08T08:39:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 -- 172.21.15.16:0/2132530362 >> v1:172.21.15.16:6789/0 conn(0x7f175c0cb370 legacy=0x7f175c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 --2- 172.21.15.16:0/2132530362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1744003970 0x7f1744003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 --2- 172.21.15.16:0/2132530362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17440047c0 0x7f1744004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 --2- 172.21.15.16:0/2132530362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1744005180 0x7f174400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 -- 172.21.15.16:0/2132530362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c070cd0 con 0x7f17440047c0 2023-12-08T08:39:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 -- 172.21.15.16:0/2132530362 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174400c020 con 0x7f1744003970 2023-12-08T08:39:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175a7fc640 1 -- 172.21.15.16:0/2132530362 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174400c1e0 con 0x7f1744005180 2023-12-08T08:39:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175affd640 1 --2- 172.21.15.16:0/2132530362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17440047c0 0x7f1744004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.204+0000 7f175b7fe640 1 --2- 172.21.15.16:0/2132530362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1744003970 0x7f1744003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.205+0000 7f175bfff640 1 --2- 172.21.15.16:0/2132530362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1744005180 0x7f174400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.205+0000 7f175affd640 1 -- 172.21.15.16:0/2132530362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1744005180 msgr2=0x7f174400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.205+0000 7f175affd640 1 --2- 172.21.15.16:0/2132530362 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1744005180 0x7f174400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.205+0000 7f175affd640 1 -- 172.21.15.16:0/2132530362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1744003970 msgr2=0x7f1744003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.205+0000 7f175affd640 1 --2- 172.21.15.16:0/2132530362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1744003970 0x7f1744003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.205+0000 7f175bfff640 1 -- 172.21.15.16:0/2132530362 reap_dead start 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.205+0000 7f175affd640 1 -- 172.21.15.16:0/2132530362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1744003520 con 0x7f17440047c0 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.205+0000 7f175b7fe640 1 --2- 172.21.15.16:0/2132530362 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1744003970 0x7f1744003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.206+0000 7f175affd640 1 --2- 172.21.15.16:0/2132530362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17440047c0 0x7f1744004c40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f175000c190 tx=0x7f17440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52122d9606964a24 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.206+0000 7f175a7fc640 1 -- 172.21.15.16:0/2132530362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750005ad0 con 0x7f17440047c0 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.206+0000 7f175a7fc640 1 -- 172.21.15.16:0/2132530362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1750005c50 con 0x7f17440047c0 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.206+0000 7f175a7fc640 1 -- 172.21.15.16:0/2132530362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175002c870 con 0x7f17440047c0 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.206+0000 7f176207c640 1 -- 172.21.15.16:0/2132530362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17440047c0 msgr2=0x7f1744004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.206+0000 7f176207c640 1 --2- 172.21.15.16:0/2132530362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17440047c0 0x7f1744004c40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f175000c190 tx=0x7f17440057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.206+0000 7f176207c640 1 -- 172.21.15.16:0/2132530362 shutdown_connections 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.206+0000 7f176207c640 1 --2- 172.21.15.16:0/2132530362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17440047c0 0x7f1744004c40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.206+0000 7f176207c640 1 -- 172.21.15.16:0/2132530362 >> 172.21.15.16:0/2132530362 conn(0x7f175c0b0850 msgr2=0x7f175c0b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:02.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.206+0000 7f176207c640 1 -- 172.21.15.16:0/2132530362 shutdown_connections 2023-12-08T08:39:02.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.206+0000 7f176207c640 1 -- 172.21.15.16:0/2132530362 wait complete. 2023-12-08T08:39:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.208+0000 7f176207c640 1 Processor -- start 2023-12-08T08:39:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.208+0000 7f176207c640 1 -- start start 2023-12-08T08:39:02.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.208+0000 7f176207c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17440047c0 0x7f175c14c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.208+0000 7f176207c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c1529e0 0x7f175c0b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.208+0000 7f176207c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f175c0adc30 0x7f175c0ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.208+0000 7f176207c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c158ca0 con 0x7f175c1529e0 2023-12-08T08:39:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.208+0000 7f176207c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c0ca9a0 con 0x7f175c0adc30 2023-12-08T08:39:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.208+0000 7f176207c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175c14c7d0 con 0x7f17440047c0 2023-12-08T08:39:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f175affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c1529e0 0x7f175c0b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f175affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c1529e0 0x7f175c0b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53932/0 (socket says 172.21.15.16:53932) 2023-12-08T08:39:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f175affd640 1 -- 172.21.15.16:0/3728179601 learned_addr learned my addr 172.21.15.16:0/3728179601 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f175affd640 1 -- 172.21.15.16:0/3728179601 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17440047c0 msgr2=0x7f175c14c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f175bfff640 1 --2- 172.21.15.16:0/3728179601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f175c0adc30 0x7f175c0ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f175affd640 1 --2- 172.21.15.16:0/3728179601 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17440047c0 0x7f175c14c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f175affd640 1 -- 172.21.15.16:0/3728179601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f175c0adc30 msgr2=0x7f175c0ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f175affd640 1 --2- 172.21.15.16:0/3728179601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f175c0adc30 0x7f175c0ae0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f175affd640 1 -- 172.21.15.16:0/3728179601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c0ad7a0 con 0x7f175c1529e0 2023-12-08T08:39:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f175affd640 1 --2- 172.21.15.16:0/3728179601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c1529e0 0x7f175c0b2350 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1750001930 tx=0x7f175002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f1743fff640 1 -- 172.21.15.16:0/3728179601 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175003c040 con 0x7f175c1529e0 2023-12-08T08:39:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f1743fff640 1 -- 172.21.15.16:0/3728179601 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175002cd40 con 0x7f175c1529e0 2023-12-08T08:39:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f176207c640 1 -- 172.21.15.16:0/3728179601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175c1586e0 con 0x7f175c1529e0 2023-12-08T08:39:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f1743fff640 1 -- 172.21.15.16:0/3728179601 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750007270 con 0x7f175c1529e0 2023-12-08T08:39:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.209+0000 7f176207c640 1 -- 172.21.15.16:0/3728179601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175c0ad430 con 0x7f175c1529e0 2023-12-08T08:39:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.211+0000 7f176207c640 1 -- 172.21.15.16:0/3728179601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175c0ad290 con 0x7f175c1529e0 2023-12-08T08:39:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.211+0000 7f1743fff640 1 -- 172.21.15.16:0/3728179601 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1750032070 con 0x7f175c1529e0 2023-12-08T08:39:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.211+0000 7f1743fff640 1 --2- 172.21.15.16:0/3728179601 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f174806ae70 0x7f174806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.211+0000 7f1743fff640 1 -- 172.21.15.16:0/3728179601 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f17500aaea0 con 0x7f175c1529e0 2023-12-08T08:39:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.212+0000 7f175b7fe640 1 --2- 172.21.15.16:0/3728179601 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f174806ae70 0x7f174806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.212+0000 7f175b7fe640 1 --2- 172.21.15.16:0/3728179601 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f174806ae70 0x7f174806d330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f174c006a90 tx=0x7f174c006a20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:02.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.215+0000 7f1743fff640 1 -- 172.21.15.16:0/3728179601 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f175c0ad290 con 0x7f175c1529e0 2023-12-08T08:39:02.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.390+0000 7f176207c640 1 -- 172.21.15.16:0/3728179601 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f175c0ad290 con 0x7f175c1529e0 2023-12-08T08:39:02.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.390+0000 7f1743fff640 1 -- 172.21.15.16:0/3728179601 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f175c0ad290 con 0x7f175c1529e0 2023-12-08T08:39:02.393 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:02.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.392+0000 7f176207c640 1 -- 172.21.15.16:0/3728179601 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f174806ae70 msgr2=0x7f174806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.392+0000 7f176207c640 1 --2- 172.21.15.16:0/3728179601 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f174806ae70 0x7f174806d330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f174c006a90 tx=0x7f174c006a20 comp rx=0 tx=0).stop 2023-12-08T08:39:02.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.392+0000 7f176207c640 1 -- 172.21.15.16:0/3728179601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c1529e0 msgr2=0x7f175c0b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.392+0000 7f176207c640 1 --2- 172.21.15.16:0/3728179601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c1529e0 0x7f175c0b2350 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1750001930 tx=0x7f175002e070 comp rx=0 tx=0).stop 2023-12-08T08:39:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.392+0000 7f176207c640 1 -- 172.21.15.16:0/3728179601 shutdown_connections 2023-12-08T08:39:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.392+0000 7f176207c640 1 --2- 172.21.15.16:0/3728179601 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f174806ae70 0x7f174806d330 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.392+0000 7f176207c640 1 --2- 172.21.15.16:0/3728179601 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f175c0adc30 0x7f175c0ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.392+0000 7f176207c640 1 --2- 172.21.15.16:0/3728179601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f175c1529e0 0x7f175c0b2350 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.392+0000 7f176207c640 1 --2- 172.21.15.16:0/3728179601 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f17440047c0 0x7f175c14c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.392+0000 7f176207c640 1 -- 172.21.15.16:0/3728179601 >> 172.21.15.16:0/3728179601 conn(0x7f175c0b0850 msgr2=0x7f175c06c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.392+0000 7f176207c640 1 -- 172.21.15.16:0/3728179601 shutdown_connections 2023-12-08T08:39:02.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.392+0000 7f176207c640 1 -- 172.21.15.16:0/3728179601 wait complete. 2023-12-08T08:39:02.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:02.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.510+0000 7f062cfc5640 1 Processor -- start 2023-12-08T08:39:02.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.510+0000 7f062cfc5640 1 -- start start 2023-12-08T08:39:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.511+0000 7f062cfc5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06280cbd10 0x7f0628070e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.511+0000 7f062cfc5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f062815b750 0x7f062806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.511+0000 7f062cfc5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06280720d0 con 0x7f0628158ee0 2023-12-08T08:39:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.511+0000 7f062cfc5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06280cc890 con 0x7f062815b750 2023-12-08T08:39:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.511+0000 7f062cfc5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06280714c0 con 0x7f06280cbd10 2023-12-08T08:39:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.511+0000 7f0626d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f062815b750 0x7f062806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.511+0000 7f0626d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f062815b750 0x7f062806c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53936/0 (socket says 172.21.15.16:53936) 2023-12-08T08:39:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.511+0000 7f0626d9d640 1 -- 172.21.15.16:0/1025208061 learned_addr learned my addr 172.21.15.16:0/1025208061 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.511+0000 7f061dd9b640 1 --2- 172.21.15.16:0/1025208061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06280cbd10 0x7f0628070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.511+0000 7f0626d9d640 1 -- 172.21.15.16:0/1025208061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06280cbd10 msgr2=0x7f0628070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.511+0000 7f0626d9d640 1 --2- 172.21.15.16:0/1025208061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06280cbd10 0x7f0628070e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.511+0000 7f0626d9d640 1 -- 172.21.15.16:0/1025208061 >> v1:172.21.15.112:6789/0 conn(0x7f0628158ee0 legacy=0x7f06280cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.511+0000 7f0626d9d640 1 -- 172.21.15.16:0/1025208061 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06280717a0 con 0x7f062815b750 2023-12-08T08:39:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.512+0000 7f0626d9d640 1 --2- 172.21.15.16:0/1025208061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f062815b750 0x7f062806c480 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0620009e50 tx=0x7f062002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394fbc8ade885f44 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.512+0000 7f0625d9b640 1 -- 172.21.15.16:0/1025208061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062002ca70 con 0x7f062815b750 2023-12-08T08:39:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.512+0000 7f0625d9b640 1 -- 172.21.15.16:0/1025208061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062002cbd0 con 0x7f062815b750 2023-12-08T08:39:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.512+0000 7f0625d9b640 1 -- 172.21.15.16:0/1025208061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06200343f0 con 0x7f062815b750 2023-12-08T08:39:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.512+0000 7f062cfc5640 1 -- 172.21.15.16:0/1025208061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f062815b750 msgr2=0x7f062806c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.512+0000 7f062cfc5640 1 --2- 172.21.15.16:0/1025208061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f062815b750 0x7f062806c480 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0620009e50 tx=0x7f062002bed0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.513+0000 7f062cfc5640 1 -- 172.21.15.16:0/1025208061 shutdown_connections 2023-12-08T08:39:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.513+0000 7f062cfc5640 1 --2- 172.21.15.16:0/1025208061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f062815b750 0x7f062806c480 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.513+0000 7f062cfc5640 1 --2- 172.21.15.16:0/1025208061 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06280cbd10 0x7f0628070e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.513+0000 7f062cfc5640 1 -- 172.21.15.16:0/1025208061 >> 172.21.15.16:0/1025208061 conn(0x7f06280b09f0 msgr2=0x7f06280b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.513+0000 7f062cfc5640 1 -- 172.21.15.16:0/1025208061 shutdown_connections 2023-12-08T08:39:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.513+0000 7f062cfc5640 1 -- 172.21.15.16:0/1025208061 wait complete. 2023-12-08T08:39:02.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.514+0000 7f062cfc5640 1 Processor -- start 2023-12-08T08:39:02.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.514+0000 7f062cfc5640 1 -- start start 2023-12-08T08:39:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f062cfc5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06280cbd10 0x7f062814d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f062cfc5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0628158ee0 0x7f06280b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f062cfc5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f062815b750 0x7f062815da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f062cfc5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062815b600 con 0x7f0628158ee0 2023-12-08T08:39:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f062cfc5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06280b2f20 con 0x7f062815b750 2023-12-08T08:39:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f062cfc5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06281f2660 con 0x7f06280cbd10 2023-12-08T08:39:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f061dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0628158ee0 0x7f06280b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f061dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0628158ee0 0x7f06280b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53944/0 (socket says 172.21.15.16:53944) 2023-12-08T08:39:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f061dd9b640 1 -- 172.21.15.16:0/1584069055 learned_addr learned my addr 172.21.15.16:0/1584069055 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f0626d9d640 1 --2- 172.21.15.16:0/1584069055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f062815b750 0x7f062815da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f062659c640 1 --2- 172.21.15.16:0/1584069055 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06280cbd10 0x7f062814d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f061dd9b640 1 -- 172.21.15.16:0/1584069055 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06280cbd10 msgr2=0x7f062814d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f061dd9b640 1 --2- 172.21.15.16:0/1584069055 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06280cbd10 0x7f062814d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f061dd9b640 1 -- 172.21.15.16:0/1584069055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f062815b750 msgr2=0x7f062815da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f061dd9b640 1 --2- 172.21.15.16:0/1584069055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f062815b750 0x7f062815da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.515+0000 7f061dd9b640 1 -- 172.21.15.16:0/1584069055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0620009b30 con 0x7f0628158ee0 2023-12-08T08:39:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.516+0000 7f061dd9b640 1 --2- 172.21.15.16:0/1584069055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0628158ee0 0x7f06280b26f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f061000d800 tx=0x7f061000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.516+0000 7f061f7fe640 1 -- 172.21.15.16:0/1584069055 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061000b6c0 con 0x7f0628158ee0 2023-12-08T08:39:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.516+0000 7f061f7fe640 1 -- 172.21.15.16:0/1584069055 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061000b820 con 0x7f0628158ee0 2023-12-08T08:39:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.516+0000 7f062cfc5640 1 -- 172.21.15.16:0/1584069055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062815ad60 con 0x7f0628158ee0 2023-12-08T08:39:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.516+0000 7f061f7fe640 1 -- 172.21.15.16:0/1584069055 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061000bbb0 con 0x7f0628158ee0 2023-12-08T08:39:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.516+0000 7f062cfc5640 1 -- 172.21.15.16:0/1584069055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062815b320 con 0x7f0628158ee0 2023-12-08T08:39:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.517+0000 7f062cfc5640 1 -- 172.21.15.16:0/1584069055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0628067430 con 0x7f0628158ee0 2023-12-08T08:39:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.517+0000 7f061f7fe640 1 -- 172.21.15.16:0/1584069055 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f061000bd10 con 0x7f0628158ee0 2023-12-08T08:39:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.518+0000 7f061f7fe640 1 --2- 172.21.15.16:0/1584069055 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f060806ae40 0x7f060806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.518+0000 7f061f7fe640 1 -- 172.21.15.16:0/1584069055 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f0610089e90 con 0x7f0628158ee0 2023-12-08T08:39:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.518+0000 7f062659c640 1 --2- 172.21.15.16:0/1584069055 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f060806ae40 0x7f060806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.518+0000 7f062659c640 1 --2- 172.21.15.16:0/1584069055 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f060806ae40 0x7f060806d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0618000f60 tx=0x7f0618009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.522+0000 7f061f7fe640 1 -- 172.21.15.16:0/1584069055 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0610054b70 con 0x7f0628158ee0 2023-12-08T08:39:02.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.700+0000 7f062cfc5640 1 -- 172.21.15.16:0/1584069055 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f062814dfe0 con 0x7f0628158ee0 2023-12-08T08:39:02.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.701+0000 7f061f7fe640 1 -- 172.21.15.16:0/1584069055 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7f0610081e60 con 0x7f0628158ee0 2023-12-08T08:39:02.704 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:02.704 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:02.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.703+0000 7f062cfc5640 1 -- 172.21.15.16:0/1584069055 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f060806ae40 msgr2=0x7f060806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.703+0000 7f062cfc5640 1 --2- 172.21.15.16:0/1584069055 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f060806ae40 0x7f060806d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0618000f60 tx=0x7f0618009040 comp rx=0 tx=0).stop 2023-12-08T08:39:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.703+0000 7f062cfc5640 1 -- 172.21.15.16:0/1584069055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0628158ee0 msgr2=0x7f06280b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.703+0000 7f062cfc5640 1 --2- 172.21.15.16:0/1584069055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0628158ee0 0x7f06280b26f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f061000d800 tx=0x7f061000dcc0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.703+0000 7f062cfc5640 1 -- 172.21.15.16:0/1584069055 shutdown_connections 2023-12-08T08:39:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.703+0000 7f062cfc5640 1 --2- 172.21.15.16:0/1584069055 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f060806ae40 0x7f060806d300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.703+0000 7f062cfc5640 1 --2- 172.21.15.16:0/1584069055 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f062815b750 0x7f062815da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.703+0000 7f062cfc5640 1 --2- 172.21.15.16:0/1584069055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0628158ee0 0x7f06280b26f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.703+0000 7f062cfc5640 1 --2- 172.21.15.16:0/1584069055 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06280cbd10 0x7f062814d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.703+0000 7f062cfc5640 1 -- 172.21.15.16:0/1584069055 >> 172.21.15.16:0/1584069055 conn(0x7f06280b09f0 msgr2=0x7f062806dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.703+0000 7f062cfc5640 1 -- 172.21.15.16:0/1584069055 shutdown_connections 2023-12-08T08:39:02.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.703+0000 7f062cfc5640 1 -- 172.21.15.16:0/1584069055 wait complete. 2023-12-08T08:39:02.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:39:02.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:02.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.824+0000 7fbcc61a9640 1 Processor -- start 2023-12-08T08:39:02.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.824+0000 7fbcc61a9640 1 -- start start 2023-12-08T08:39:02.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.825+0000 7fbcc61a9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbcc00cb370 0x7fbcc015e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.825+0000 7fbcc61a9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc0157e00 con 0x7fbcc0157a20 2023-12-08T08:39:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.825+0000 7fbcc61a9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc006dd20 con 0x7fbcc006d7d0 2023-12-08T08:39:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.825+0000 7fbcc61a9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc0070cd0 con 0x7fbcc00cb370 2023-12-08T08:39:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.825+0000 7fbcbeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbcc00cb370 0x7fbcc015e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.825+0000 7fbcbf7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbcc0157a20 0x7fbcc015d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38130/0 (socket says 172.21.15.16:38130) 2023-12-08T08:39:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.825+0000 7fbcbf7fe640 1 -- 172.21.15.16:0/771422856 learned_addr learned my addr 172.21.15.16:0/771422856 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.826+0000 7fbcbeffd640 1 -- 172.21.15.16:0/771422856 >> v1:172.21.15.112:6789/0 conn(0x7fbcc006d7d0 legacy=0x7fbcc00b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.826+0000 7fbcbeffd640 1 -- 172.21.15.16:0/771422856 >> v1:172.21.15.16:6789/0 conn(0x7fbcc0157a20 legacy=0x7fbcc015d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.826+0000 7fbcbe7fc640 1 -- 172.21.15.16:0/771422856 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422107618 0 0) 0x7fbcc0157e00 con 0x7fbcc0157a20 2023-12-08T08:39:02.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.826+0000 7fbcbeffd640 1 -- 172.21.15.16:0/771422856 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc0070fb0 con 0x7fbcc00cb370 2023-12-08T08:39:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.826+0000 7fbcbeffd640 1 --2- 172.21.15.16:0/771422856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc00cb370 0x7fbcc015e0e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbcb40097a0 tx=0x7fbcb402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11c5dfd164f5b2d server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.827+0000 7fbcbe7fc640 1 -- 172.21.15.16:0/771422856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4031830 con 0x7fbcc00cb370 2023-12-08T08:39:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.827+0000 7fbcbe7fc640 1 -- 172.21.15.16:0/771422856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb4031990 con 0x7fbcc00cb370 2023-12-08T08:39:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.827+0000 7fbcbe7fc640 1 -- 172.21.15.16:0/771422856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4031d20 con 0x7fbcc00cb370 2023-12-08T08:39:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.827+0000 7fbcc61a9640 1 -- 172.21.15.16:0/771422856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc00cb370 msgr2=0x7fbcc015e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.827+0000 7fbcc61a9640 1 --2- 172.21.15.16:0/771422856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc00cb370 0x7fbcc015e0e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbcb40097a0 tx=0x7fbcb402b8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.827+0000 7fbcc61a9640 1 -- 172.21.15.16:0/771422856 shutdown_connections 2023-12-08T08:39:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.827+0000 7fbcc61a9640 1 --2- 172.21.15.16:0/771422856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc00cb370 0x7fbcc015e0e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.827+0000 7fbcc61a9640 1 -- 172.21.15.16:0/771422856 >> 172.21.15.16:0/771422856 conn(0x7fbcc00b0850 msgr2=0x7fbcc00b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.827+0000 7fbcc61a9640 1 -- 172.21.15.16:0/771422856 shutdown_connections 2023-12-08T08:39:02.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.827+0000 7fbcc61a9640 1 -- 172.21.15.16:0/771422856 wait complete. 2023-12-08T08:39:02.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.828+0000 7fbcc61a9640 1 Processor -- start 2023-12-08T08:39:02.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.829+0000 7fbcc61a9640 1 -- start start 2023-12-08T08:39:02.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.829+0000 7fbcc61a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc006d7d0 0x7fbcc013f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.829+0000 7fbcc61a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc00cb370 0x7fbcc01f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.829+0000 7fbcc61a9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcc0157a20 0x7fbcc0151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.829+0000 7fbcc61a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc013d760 con 0x7fbcc006d7d0 2023-12-08T08:39:02.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.829+0000 7fbcc61a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc013fce0 con 0x7fbcc00cb370 2023-12-08T08:39:02.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.829+0000 7fbcc61a9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc013ffc0 con 0x7fbcc0157a20 2023-12-08T08:39:02.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.829+0000 7fbcbf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc006d7d0 0x7fbcc013f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.829+0000 7fbcbf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc006d7d0 0x7fbcc013f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53956/0 (socket says 172.21.15.16:53956) 2023-12-08T08:39:02.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.829+0000 7fbcbf7fe640 1 -- 172.21.15.16:0/1838342261 learned_addr learned my addr 172.21.15.16:0/1838342261 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:02.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.829+0000 7fbcbeffd640 1 --2- 172.21.15.16:0/1838342261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc00cb370 0x7fbcc01f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.830+0000 7fbcbffff640 1 --2- 172.21.15.16:0/1838342261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcc0157a20 0x7fbcc0151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.830+0000 7fbcbf7fe640 1 -- 172.21.15.16:0/1838342261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcc0157a20 msgr2=0x7fbcc0151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.830+0000 7fbcbf7fe640 1 --2- 172.21.15.16:0/1838342261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcc0157a20 0x7fbcc0151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.830+0000 7fbcbf7fe640 1 -- 172.21.15.16:0/1838342261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc00cb370 msgr2=0x7fbcc01f2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:02.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.830+0000 7fbcbf7fe640 1 --2- 172.21.15.16:0/1838342261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc00cb370 0x7fbcc01f2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:02.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.830+0000 7fbcbf7fe640 1 -- 172.21.15.16:0/1838342261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb4009400 con 0x7fbcc006d7d0 2023-12-08T08:39:02.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.830+0000 7fbcbf7fe640 1 --2- 172.21.15.16:0/1838342261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc006d7d0 0x7fbcc013f290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000c430 tx=0x7fbcb000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:02.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.830+0000 7fbca7fff640 1 -- 172.21.15.16:0/1838342261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00074e0 con 0x7fbcc006d7d0 2023-12-08T08:39:02.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.830+0000 7fbca7fff640 1 -- 172.21.15.16:0/1838342261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb0007660 con 0x7fbcc006d7d0 2023-12-08T08:39:02.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.830+0000 7fbcc61a9640 1 -- 172.21.15.16:0/1838342261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc013d380 con 0x7fbcc006d7d0 2023-12-08T08:39:02.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.830+0000 7fbca7fff640 1 -- 172.21.15.16:0/1838342261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0004a20 con 0x7fbcc006d7d0 2023-12-08T08:39:02.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.830+0000 7fbcc61a9640 1 -- 172.21.15.16:0/1838342261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc013f7d0 con 0x7fbcc006d7d0 2023-12-08T08:39:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.831+0000 7fbcc61a9640 1 -- 172.21.15.16:0/1838342261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc013d8a0 con 0x7fbcc006d7d0 2023-12-08T08:39:02.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.832+0000 7fbca7fff640 1 -- 172.21.15.16:0/1838342261 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbcb0004b80 con 0x7fbcc006d7d0 2023-12-08T08:39:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.832+0000 7fbca7fff640 1 --2- 172.21.15.16:0/1838342261 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbcac06ae40 0x7fbcac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.832+0000 7fbca7fff640 1 -- 172.21.15.16:0/1838342261 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fbcb0089640 con 0x7fbcc006d7d0 2023-12-08T08:39:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.832+0000 7fbcbeffd640 1 --2- 172.21.15.16:0/1838342261 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbcac06ae40 0x7fbcac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:02.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.833+0000 7fbcbeffd640 1 --2- 172.21.15.16:0/1838342261 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbcac06ae40 0x7fbcac06d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4036000 tx=0x7fbcb403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:02.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:02.836+0000 7fbca7fff640 1 -- 172.21.15.16:0/1838342261 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbcc013d8a0 con 0x7fbcc006d7d0 2023-12-08T08:39:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.011+0000 7fbcc61a9640 1 -- 172.21.15.16:0/1838342261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbcc0067430 con 0x7fbcc006d7d0 2023-12-08T08:39:03.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.011+0000 7fbca7fff640 1 -- 172.21.15.16:0/1838342261 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbcb0054450 con 0x7fbcc006d7d0 2023-12-08T08:39:03.014 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:03.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.013+0000 7fbcc61a9640 1 -- 172.21.15.16:0/1838342261 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbcac06ae40 msgr2=0x7fbcac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.013+0000 7fbcc61a9640 1 --2- 172.21.15.16:0/1838342261 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbcac06ae40 0x7fbcac06d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4036000 tx=0x7fbcb403d040 comp rx=0 tx=0).stop 2023-12-08T08:39:03.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.013+0000 7fbcc61a9640 1 -- 172.21.15.16:0/1838342261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc006d7d0 msgr2=0x7fbcc013f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.013+0000 7fbcc61a9640 1 --2- 172.21.15.16:0/1838342261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc006d7d0 0x7fbcc013f290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000c430 tx=0x7fbcb000c8f0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.013+0000 7fbcc61a9640 1 -- 172.21.15.16:0/1838342261 shutdown_connections 2023-12-08T08:39:03.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.013+0000 7fbcc61a9640 1 --2- 172.21.15.16:0/1838342261 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbcac06ae40 0x7fbcac06d300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.013+0000 7fbcc61a9640 1 --2- 172.21.15.16:0/1838342261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbcc0157a20 0x7fbcc0151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.013+0000 7fbcc61a9640 1 --2- 172.21.15.16:0/1838342261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbcc00cb370 0x7fbcc01f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.013+0000 7fbcc61a9640 1 --2- 172.21.15.16:0/1838342261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcc006d7d0 0x7fbcc013f290 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.013+0000 7fbcc61a9640 1 -- 172.21.15.16:0/1838342261 >> 172.21.15.16:0/1838342261 conn(0x7fbcc00b0850 msgr2=0x7fbcc0153850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.013+0000 7fbcc61a9640 1 -- 172.21.15.16:0/1838342261 shutdown_connections 2023-12-08T08:39:03.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.013+0000 7fbcc61a9640 1 -- 172.21.15.16:0/1838342261 wait complete. 2023-12-08T08:39:03.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:39:03.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.133+0000 7ff4fd84b640 1 Processor -- start 2023-12-08T08:39:03.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.133+0000 7ff4fd84b640 1 -- start start 2023-12-08T08:39:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.134+0000 7ff4fd84b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4f8160980 0x7ff4f80cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.134+0000 7ff4fd84b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4f806b6e0 0x7ff4f806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.134+0000 7ff4fd84b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f80b1260 con 0x7ff4f80d0000 2023-12-08T08:39:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.134+0000 7ff4fd84b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f80d04f0 con 0x7ff4f8160980 2023-12-08T08:39:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.134+0000 7ff4fd84b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80b0930 con 0x7ff4f806b6e0 2023-12-08T08:39:03.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.135+0000 7ff4f6ffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4f8160980 0x7ff4f80cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.135+0000 7ff4f77fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff4f80d0000 0x7ff4f8160ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38144/0 (socket says 172.21.15.16:38144) 2023-12-08T08:39:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.135+0000 7ff4f77fe640 1 -- 172.21.15.16:0/2251826952 learned_addr learned my addr 172.21.15.16:0/2251826952 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.135+0000 7ff4f67fc640 1 --2- 172.21.15.16:0/2251826952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4f806b6e0 0x7ff4f806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.135+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2251826952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4f806b6e0 msgr2=0x7ff4f806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.135+0000 7ff4f6ffd640 1 --2- 172.21.15.16:0/2251826952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4f806b6e0 0x7ff4f806bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.135+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2251826952 >> v1:172.21.15.16:6789/0 conn(0x7ff4f80d0000 legacy=0x7ff4f8160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.135+0000 7ff4f6ffd640 1 -- 172.21.15.16:0/2251826952 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f80b0c10 con 0x7ff4f8160980 2023-12-08T08:39:03.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.135+0000 7ff4f5ffb640 1 -- 172.21.15.16:0/2251826952 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162303966 0 0) 0x7ff4f80b1260 con 0x7ff4f80d0000 2023-12-08T08:39:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.136+0000 7ff4f6ffd640 1 --2- 172.21.15.16:0/2251826952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f8160980 0x7ff4f80cfac0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00097a0 tx=0x7ff4f002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea07e49fb04c075 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.136+0000 7ff4f5ffb640 1 -- 172.21.15.16:0/2251826952 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0030610 con 0x7ff4f8160980 2023-12-08T08:39:03.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.136+0000 7ff4f5ffb640 1 -- 172.21.15.16:0/2251826952 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f0030770 con 0x7ff4f8160980 2023-12-08T08:39:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.136+0000 7ff4f5ffb640 1 -- 172.21.15.16:0/2251826952 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0030b00 con 0x7ff4f8160980 2023-12-08T08:39:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.136+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2251826952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f8160980 msgr2=0x7ff4f80cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.136+0000 7ff4fd84b640 1 --2- 172.21.15.16:0/2251826952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f8160980 0x7ff4f80cfac0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff4f00097a0 tx=0x7ff4f002b960 comp rx=0 tx=0).stop 2023-12-08T08:39:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.136+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2251826952 shutdown_connections 2023-12-08T08:39:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.136+0000 7ff4fd84b640 1 --2- 172.21.15.16:0/2251826952 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4f806b6e0 0x7ff4f806bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.136+0000 7ff4fd84b640 1 --2- 172.21.15.16:0/2251826952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f8160980 0x7ff4f80cfac0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.136+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2251826952 >> 172.21.15.16:0/2251826952 conn(0x7ff4f80d0e30 msgr2=0x7ff4f80d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.136+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2251826952 shutdown_connections 2023-12-08T08:39:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.136+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2251826952 wait complete. 2023-12-08T08:39:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.138+0000 7ff4fd84b640 1 Processor -- start 2023-12-08T08:39:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.138+0000 7ff4fd84b640 1 -- start start 2023-12-08T08:39:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.138+0000 7ff4fd84b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f806b6e0 0x7ff4f81f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.138+0000 7ff4fd84b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f80d0000 0x7ff4f8153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.138+0000 7ff4fd84b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4f8160980 0x7ff4f80c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.138+0000 7ff4fd84b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81541c0 con 0x7ff4f80d0000 2023-12-08T08:39:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.138+0000 7ff4fd84b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80c6cb0 con 0x7ff4f806b6e0 2023-12-08T08:39:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.138+0000 7ff4fd84b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81f63b0 con 0x7ff4f8160980 2023-12-08T08:39:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f80d0000 0x7ff4f8153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f80d0000 0x7ff4f8153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53974/0 (socket says 172.21.15.16:53974) 2023-12-08T08:39:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4f67fc640 1 -- 172.21.15.16:0/2800710315 learned_addr learned my addr 172.21.15.16:0/2800710315 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4f77fe640 1 --2- 172.21.15.16:0/2800710315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4f8160980 0x7ff4f80c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4f67fc640 1 -- 172.21.15.16:0/2800710315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4f8160980 msgr2=0x7ff4f80c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4f6ffd640 1 --2- 172.21.15.16:0/2800710315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f806b6e0 0x7ff4f81f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4f67fc640 1 --2- 172.21.15.16:0/2800710315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4f8160980 0x7ff4f80c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4f67fc640 1 -- 172.21.15.16:0/2800710315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f806b6e0 msgr2=0x7ff4f81f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4f67fc640 1 --2- 172.21.15.16:0/2800710315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f806b6e0 0x7ff4f81f3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4f67fc640 1 -- 172.21.15.16:0/2800710315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f0009400 con 0x7ff4f80d0000 2023-12-08T08:39:03.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4f67fc640 1 --2- 172.21.15.16:0/2800710315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f80d0000 0x7ff4f8153c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000c820 tx=0x7ff4e000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4effff640 1 -- 172.21.15.16:0/2800710315 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0004220 con 0x7ff4f80d0000 2023-12-08T08:39:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4effff640 1 -- 172.21.15.16:0/2800710315 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e00043a0 con 0x7ff4f80d0000 2023-12-08T08:39:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2800710315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f80c69e0 con 0x7ff4f80d0000 2023-12-08T08:39:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4effff640 1 -- 172.21.15.16:0/2800710315 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0007a70 con 0x7ff4f80d0000 2023-12-08T08:39:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.139+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2800710315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f8155050 con 0x7ff4f80d0000 2023-12-08T08:39:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.140+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2800710315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f8067ad0 con 0x7ff4f80d0000 2023-12-08T08:39:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.141+0000 7ff4effff640 1 -- 172.21.15.16:0/2800710315 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff4e0007bd0 con 0x7ff4f80d0000 2023-12-08T08:39:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.141+0000 7ff4effff640 1 --2- 172.21.15.16:0/2800710315 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4d806ae70 0x7ff4d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.141+0000 7ff4effff640 1 -- 172.21.15.16:0/2800710315 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7ff4e00896a0 con 0x7ff4f80d0000 2023-12-08T08:39:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.141+0000 7ff4f6ffd640 1 --2- 172.21.15.16:0/2800710315 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4d806ae70 0x7ff4d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.142+0000 7ff4f6ffd640 1 --2- 172.21.15.16:0/2800710315 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4d806ae70 0x7ff4d806d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0036000 tx=0x7ff4f0008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:03.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.145+0000 7ff4effff640 1 -- 172.21.15.16:0/2800710315 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4e0054400 con 0x7ff4f80d0000 2023-12-08T08:39:03.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.325+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2800710315 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4f806dbe0 con 0x7ff4f80d0000 2023-12-08T08:39:03.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.326+0000 7ff4effff640 1 -- 172.21.15.16:0/2800710315 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff4e0057a20 con 0x7ff4f80d0000 2023-12-08T08:39:03.329 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.328+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2800710315 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4d806ae70 msgr2=0x7ff4d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.328+0000 7ff4fd84b640 1 --2- 172.21.15.16:0/2800710315 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4d806ae70 0x7ff4d806d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0036000 tx=0x7ff4f0008030 comp rx=0 tx=0).stop 2023-12-08T08:39:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.328+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2800710315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f80d0000 msgr2=0x7ff4f8153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.328+0000 7ff4fd84b640 1 --2- 172.21.15.16:0/2800710315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f80d0000 0x7ff4f8153c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000c820 tx=0x7ff4e000cce0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.328+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2800710315 shutdown_connections 2023-12-08T08:39:03.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.328+0000 7ff4fd84b640 1 --2- 172.21.15.16:0/2800710315 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4d806ae70 0x7ff4d806d330 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.328+0000 7ff4fd84b640 1 --2- 172.21.15.16:0/2800710315 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4f8160980 0x7ff4f80c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.328+0000 7ff4fd84b640 1 --2- 172.21.15.16:0/2800710315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4f80d0000 0x7ff4f8153c80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.328+0000 7ff4fd84b640 1 --2- 172.21.15.16:0/2800710315 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4f806b6e0 0x7ff4f81f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.328+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2800710315 >> 172.21.15.16:0/2800710315 conn(0x7ff4f80d0e30 msgr2=0x7ff4f80ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.328+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2800710315 shutdown_connections 2023-12-08T08:39:03.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.328+0000 7ff4fd84b640 1 -- 172.21.15.16:0/2800710315 wait complete. 2023-12-08T08:39:03.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:39:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.448+0000 7f3d64d43640 1 Processor -- start 2023-12-08T08:39:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.448+0000 7f3d64d43640 1 -- start start 2023-12-08T08:39:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.449+0000 7f3d64d43640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3d6015d120 0x7f3d600ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.449+0000 7f3d64d43640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6015d500 con 0x7f3d6006bbf0 2023-12-08T08:39:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.449+0000 7f3d64d43640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d600ad7c0 con 0x7f3d600ad290 2023-12-08T08:39:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.449+0000 7f3d64d43640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d600b2f80 con 0x7f3d6015d120 2023-12-08T08:39:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.449+0000 7f3d5ed9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3d6006bbf0 0x7f3d600b3a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38146/0 (socket says 172.21.15.16:38146) 2023-12-08T08:39:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.449+0000 7f3d5ed9d640 1 -- 172.21.15.16:0/3275835497 learned_addr learned my addr 172.21.15.16:0/3275835497 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.449+0000 7f3d5e59c640 1 --2- 172.21.15.16:0/3275835497 >> v2:172.21.15.112:3300/0 conn(0x7f3d6015d120 0x7f3d600ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.449+0000 7f3d5d59a640 1 -- 172.21.15.16:0/3275835497 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724167426 0 0) 0x7f3d6015d500 con 0x7f3d6006bbf0 2023-12-08T08:39:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.449+0000 7f3d5d59a640 1 -- 172.21.15.16:0/3275835497 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d4c003520 con 0x7f3d6006bbf0 2023-12-08T08:39:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5d59a640 1 -- 172.21.15.16:0/3275835497 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1516578313 0 0) 0x7f3d4c003520 con 0x7f3d6006bbf0 2023-12-08T08:39:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5d59a640 1 -- 172.21.15.16:0/3275835497 >> v2:172.21.15.112:3300/0 conn(0x7f3d6015d120 msgr2=0x7f3d600ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5d59a640 1 --2- 172.21.15.16:0/3275835497 >> v2:172.21.15.112:3300/0 conn(0x7f3d6015d120 0x7f3d600ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5d59a640 1 -- 172.21.15.16:0/3275835497 >> v1:172.21.15.112:6789/0 conn(0x7f3d600ad290 legacy=0x7f3d6006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5d59a640 1 -- 172.21.15.16:0/3275835497 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d600b3260 con 0x7f3d6006bbf0 2023-12-08T08:39:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5d59a640 1 -- 172.21.15.16:0/3275835497 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f3d500019f0 con 0x7f3d6006bbf0 2023-12-08T08:39:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5e59c640 1 --2- 172.21.15.16:0/3275835497 >> v2:172.21.15.112:3300/0 conn(0x7f3d6015d120 0x7f3d600ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:39:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5d59a640 1 -- 172.21.15.16:0/3275835497 >> v1:172.21.15.16:6789/0 conn(0x7f3d6006bbf0 legacy=0x7f3d600b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5d59a640 1 --2- 172.21.15.16:0/3275835497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d4c003970 0x7f3d4c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5d59a640 1 --2- 172.21.15.16:0/3275835497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d4c0047c0 0x7f3d4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5d59a640 1 --2- 172.21.15.16:0/3275835497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d4c005180 0x7f3d4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5d59a640 1 -- 172.21.15.16:0/3275835497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d600b2f80 con 0x7f3d4c005180 2023-12-08T08:39:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5d59a640 1 -- 172.21.15.16:0/3275835497 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c00c020 con 0x7f3d4c003970 2023-12-08T08:39:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5d59a640 1 -- 172.21.15.16:0/3275835497 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c00c1e0 con 0x7f3d4c0047c0 2023-12-08T08:39:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5ed9d640 1 --2- 172.21.15.16:0/3275835497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d4c005180 0x7f3d4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5e59c640 1 --2- 172.21.15.16:0/3275835497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d4c003970 0x7f3d4c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5dd9b640 1 --2- 172.21.15.16:0/3275835497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d4c0047c0 0x7f3d4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5ed9d640 1 -- 172.21.15.16:0/3275835497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d4c0047c0 msgr2=0x7f3d4c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5ed9d640 1 --2- 172.21.15.16:0/3275835497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d4c0047c0 0x7f3d4c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5ed9d640 1 -- 172.21.15.16:0/3275835497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d4c003970 msgr2=0x7f3d4c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5ed9d640 1 --2- 172.21.15.16:0/3275835497 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d4c003970 0x7f3d4c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5ed9d640 1 -- 172.21.15.16:0/3275835497 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c003520 con 0x7f3d4c005180 2023-12-08T08:39:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.450+0000 7f3d5dd9b640 1 --2- 172.21.15.16:0/3275835497 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d4c0047c0 0x7f3d4c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:03.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.451+0000 7f3d5ed9d640 1 -- 172.21.15.16:0/3275835497 reap_dead start 2023-12-08T08:39:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.451+0000 7f3d5ed9d640 1 --2- 172.21.15.16:0/3275835497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d4c005180 0x7f3d4c00b980 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3d60070610 tx=0x7f3d4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a733d401da91986 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.451+0000 7f3d5d59a640 1 -- 172.21.15.16:0/3275835497 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d500072d0 con 0x7f3d4c005180 2023-12-08T08:39:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.451+0000 7f3d5d59a640 1 -- 172.21.15.16:0/3275835497 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5002f070 con 0x7f3d4c005180 2023-12-08T08:39:03.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.451+0000 7f3d5d59a640 1 -- 172.21.15.16:0/3275835497 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d50007670 con 0x7f3d4c005180 2023-12-08T08:39:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.452+0000 7f3d64d43640 1 -- 172.21.15.16:0/3275835497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d4c005180 msgr2=0x7f3d4c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.452+0000 7f3d64d43640 1 --2- 172.21.15.16:0/3275835497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d4c005180 0x7f3d4c00b980 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3d60070610 tx=0x7f3d4c0057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.452+0000 7f3d64d43640 1 -- 172.21.15.16:0/3275835497 shutdown_connections 2023-12-08T08:39:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.452+0000 7f3d64d43640 1 --2- 172.21.15.16:0/3275835497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d4c005180 0x7f3d4c00b980 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.452+0000 7f3d64d43640 1 -- 172.21.15.16:0/3275835497 >> 172.21.15.16:0/3275835497 conn(0x7f3d60070960 msgr2=0x7f3d60070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.452+0000 7f3d64d43640 1 -- 172.21.15.16:0/3275835497 shutdown_connections 2023-12-08T08:39:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.452+0000 7f3d64d43640 1 -- 172.21.15.16:0/3275835497 wait complete. 2023-12-08T08:39:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.453+0000 7f3d64d43640 1 Processor -- start 2023-12-08T08:39:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.453+0000 7f3d64d43640 1 -- start start 2023-12-08T08:39:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d64d43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d4c005180 0x7f3d600cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d64d43640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d600c4670 0x7f3d6014abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d64d43640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d6014b110 0x7f3d6014b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d64d43640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6014bad0 con 0x7f3d4c005180 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d64d43640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d60148cb0 con 0x7f3d600c4670 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d64d43640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d60157fd0 con 0x7f3d6014b110 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d5e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d4c005180 0x7f3d600cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d5e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d4c005180 0x7f3d600cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53986/0 (socket says 172.21.15.16:53986) 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d5e59c640 1 -- 172.21.15.16:0/3987776771 learned_addr learned my addr 172.21.15.16:0/3987776771 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d5dd9b640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d600c4670 0x7f3d6014abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d5e59c640 1 -- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d6014b110 msgr2=0x7f3d6014b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d5ed9d640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d6014b110 0x7f3d6014b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d5e59c640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d6014b110 0x7f3d6014b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d5e59c640 1 -- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d600c4670 msgr2=0x7f3d6014abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d5e59c640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d600c4670 0x7f3d6014abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d5e59c640 1 -- 172.21.15.16:0/3987776771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d600cddf0 con 0x7f3d4c005180 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d5dd9b640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d600c4670 0x7f3d6014abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:03.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.454+0000 7f3d5ed9d640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d6014b110 0x7f3d6014b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.455+0000 7f3d5e59c640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d4c005180 0x7f3d600cd760 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3d58011ac0 tx=0x7f3d58011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.455+0000 7f3d56ffd640 1 -- 172.21.15.16:0/3987776771 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5800fc10 con 0x7f3d4c005180 2023-12-08T08:39:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.455+0000 7f3d56ffd640 1 -- 172.21.15.16:0/3987776771 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5800fd70 con 0x7f3d4c005180 2023-12-08T08:39:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.455+0000 7f3d64d43640 1 -- 172.21.15.16:0/3987776771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d600ce0d0 con 0x7f3d4c005180 2023-12-08T08:39:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.455+0000 7f3d56ffd640 1 -- 172.21.15.16:0/3987776771 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5800fee0 con 0x7f3d4c005180 2023-12-08T08:39:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.455+0000 7f3d64d43640 1 -- 172.21.15.16:0/3987776771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d600ce690 con 0x7f3d4c005180 2023-12-08T08:39:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.456+0000 7f3d64d43640 1 -- 172.21.15.16:0/3987776771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d600719c0 con 0x7f3d4c005180 2023-12-08T08:39:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.456+0000 7f3d56ffd640 1 -- 172.21.15.16:0/3987776771 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3d58004260 con 0x7f3d4c005180 2023-12-08T08:39:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.457+0000 7f3d56ffd640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d4006ae70 0x7f3d4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.457+0000 7f3d56ffd640 1 -- 172.21.15.16:0/3987776771 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f3d5808a090 con 0x7f3d4c005180 2023-12-08T08:39:03.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.457+0000 7f3d5dd9b640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d4006ae70 0x7f3d4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.458+0000 7f3d5dd9b640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d4006ae70 0x7f3d4006d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3d48000c00 tx=0x7f3d48009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:03.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.461+0000 7f3d56ffd640 1 -- 172.21.15.16:0/3987776771 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d58054d60 con 0x7f3d4c005180 2023-12-08T08:39:03.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.636+0000 7f3d64d43640 1 -- 172.21.15.16:0/3987776771 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d60067430 con 0x7f3d4c005180 2023-12-08T08:39:03.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.636+0000 7f3d56ffd640 1 -- 172.21.15.16:0/3987776771 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d580220a0 con 0x7f3d4c005180 2023-12-08T08:39:03.639 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:39:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.638+0000 7f3d64d43640 1 -- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d4006ae70 msgr2=0x7f3d4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.638+0000 7f3d64d43640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d4006ae70 0x7f3d4006d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3d48000c00 tx=0x7f3d48009040 comp rx=0 tx=0).stop 2023-12-08T08:39:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.638+0000 7f3d64d43640 1 -- 172.21.15.16:0/3987776771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d4c005180 msgr2=0x7f3d600cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.638+0000 7f3d64d43640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d4c005180 0x7f3d600cd760 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3d58011ac0 tx=0x7f3d58011f80 comp rx=0 tx=0).stop 2023-12-08T08:39:03.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.638+0000 7f3d64d43640 1 -- 172.21.15.16:0/3987776771 shutdown_connections 2023-12-08T08:39:03.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.638+0000 7f3d64d43640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d4006ae70 0x7f3d4006d330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.638+0000 7f3d64d43640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d6014b110 0x7f3d6014b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.638+0000 7f3d64d43640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d600c4670 0x7f3d6014abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.638+0000 7f3d64d43640 1 --2- 172.21.15.16:0/3987776771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d4c005180 0x7f3d600cd760 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.638+0000 7f3d64d43640 1 -- 172.21.15.16:0/3987776771 >> 172.21.15.16:0/3987776771 conn(0x7f3d60070960 msgr2=0x7f3d6015f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:03.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.638+0000 7f3d64d43640 1 -- 172.21.15.16:0/3987776771 shutdown_connections 2023-12-08T08:39:03.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.638+0000 7f3d64d43640 1 -- 172.21.15.16:0/3987776771 wait complete. 2023-12-08T08:39:03.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:39:03.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.762+0000 7f3519141640 1 Processor -- start 2023-12-08T08:39:03.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.763+0000 7f3519141640 1 -- start start 2023-12-08T08:39:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.763+0000 7f3519141640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f35140caae0 0x7f3514161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.763+0000 7f3519141640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35140caec0 0x7f351415d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.763+0000 7f3519141640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351415e600 con 0x7f351415dbc0 2023-12-08T08:39:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.763+0000 7f3519141640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351415e040 con 0x7f35140caec0 2023-12-08T08:39:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.763+0000 7f3519141640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351415e320 con 0x7f35140caae0 2023-12-08T08:39:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.763+0000 7f351259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35140caec0 0x7f351415d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.764+0000 7f351259c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35140caec0 0x7f351415d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53992/0 (socket says 172.21.15.16:53992) 2023-12-08T08:39:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.764+0000 7f351259c640 1 -- 172.21.15.16:0/2820583216 learned_addr learned my addr 172.21.15.16:0/2820583216 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.764+0000 7f3512d9d640 1 --2- 172.21.15.16:0/2820583216 >> v2:172.21.15.112:3300/0 conn(0x7f35140caae0 0x7f3514161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.764+0000 7f351259c640 1 -- 172.21.15.16:0/2820583216 >> v2:172.21.15.112:3300/0 conn(0x7f35140caae0 msgr2=0x7f3514161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.764+0000 7f351259c640 1 --2- 172.21.15.16:0/2820583216 >> v2:172.21.15.112:3300/0 conn(0x7f35140caae0 0x7f3514161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.764+0000 7f351259c640 1 -- 172.21.15.16:0/2820583216 >> v1:172.21.15.112:6789/0 conn(0x7f351415dbc0 legacy=0x7f351406a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.764+0000 7f351259c640 1 -- 172.21.15.16:0/2820583216 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3514160d10 con 0x7f35140caec0 2023-12-08T08:39:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.765+0000 7f351259c640 1 --2- 172.21.15.16:0/2820583216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140caec0 0x7f351415d680 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f35140b2920 tx=0x7f350802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a8796664d2e0f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.765+0000 7f3511d9b640 1 -- 172.21.15.16:0/2820583216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508031870 con 0x7f35140caec0 2023-12-08T08:39:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.765+0000 7f3511d9b640 1 -- 172.21.15.16:0/2820583216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35080319d0 con 0x7f35140caec0 2023-12-08T08:39:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.765+0000 7f3511d9b640 1 -- 172.21.15.16:0/2820583216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508031d60 con 0x7f35140caec0 2023-12-08T08:39:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.766+0000 7f3519141640 1 -- 172.21.15.16:0/2820583216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140caec0 msgr2=0x7f351415d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.766+0000 7f3519141640 1 --2- 172.21.15.16:0/2820583216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140caec0 0x7f351415d680 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f35140b2920 tx=0x7f350802b910 comp rx=0 tx=0).stop 2023-12-08T08:39:03.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.766+0000 7f3519141640 1 -- 172.21.15.16:0/2820583216 shutdown_connections 2023-12-08T08:39:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.766+0000 7f3519141640 1 --2- 172.21.15.16:0/2820583216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140caec0 0x7f351415d680 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.766+0000 7f3519141640 1 --2- 172.21.15.16:0/2820583216 >> v2:172.21.15.112:3300/0 conn(0x7f35140caae0 0x7f3514161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.766+0000 7f3519141640 1 -- 172.21.15.16:0/2820583216 >> 172.21.15.16:0/2820583216 conn(0x7f35140cfcd0 msgr2=0x7f35140d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.766+0000 7f3519141640 1 -- 172.21.15.16:0/2820583216 shutdown_connections 2023-12-08T08:39:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.766+0000 7f3519141640 1 -- 172.21.15.16:0/2820583216 wait complete. 2023-12-08T08:39:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.767+0000 7f3519141640 1 Processor -- start 2023-12-08T08:39:03.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.767+0000 7f3519141640 1 -- start start 2023-12-08T08:39:03.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.768+0000 7f3519141640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35140caae0 0x7f35141568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.768+0000 7f3519141640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140caec0 0x7f35140c5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.768+0000 7f3519141640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f351415dbc0 0x7f351406e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.768+0000 7f3519141640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35140c8730 con 0x7f35140caec0 2023-12-08T08:39:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.768+0000 7f3519141640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35140c7b20 con 0x7f35140caae0 2023-12-08T08:39:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.768+0000 7f3519141640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35140c7e00 con 0x7f351415dbc0 2023-12-08T08:39:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.768+0000 7f351259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140caec0 0x7f35140c5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.768+0000 7f351259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140caec0 0x7f35140c5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54008/0 (socket says 172.21.15.16:54008) 2023-12-08T08:39:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.768+0000 7f351259c640 1 -- 172.21.15.16:0/762734608 learned_addr learned my addr 172.21.15.16:0/762734608 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.768+0000 7f3512d9d640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35140caae0 0x7f35141568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.768+0000 7f351359e640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f351415dbc0 0x7f351406e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.769+0000 7f3512d9d640 1 -- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f351415dbc0 msgr2=0x7f351406e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.769+0000 7f3512d9d640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f351415dbc0 0x7f351406e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.769+0000 7f3512d9d640 1 -- 172.21.15.16:0/762734608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140caec0 msgr2=0x7f35140c5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.769+0000 7f3512d9d640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140caec0 0x7f35140c5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.769+0000 7f3512d9d640 1 -- 172.21.15.16:0/762734608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35080094d0 con 0x7f35140caae0 2023-12-08T08:39:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.769+0000 7f351259c640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140caec0 0x7f35140c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:39:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.769+0000 7f3512d9d640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35140caae0 0x7f35141568f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f350400e850 tx=0x7f350400ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.769+0000 7f34fb7fe640 1 -- 172.21.15.16:0/762734608 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35040095b0 con 0x7f35140caae0 2023-12-08T08:39:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.769+0000 7f34fb7fe640 1 -- 172.21.15.16:0/762734608 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350400c940 con 0x7f35140caae0 2023-12-08T08:39:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.769+0000 7f3519141640 1 -- 172.21.15.16:0/762734608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3514161bd0 con 0x7f35140caae0 2023-12-08T08:39:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.769+0000 7f34fb7fe640 1 -- 172.21.15.16:0/762734608 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350400ccd0 con 0x7f35140caae0 2023-12-08T08:39:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.769+0000 7f3519141640 1 -- 172.21.15.16:0/762734608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35140c83c0 con 0x7f35140caae0 2023-12-08T08:39:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.770+0000 7f3519141640 1 -- 172.21.15.16:0/762734608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3514067ad0 con 0x7f35140caae0 2023-12-08T08:39:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.771+0000 7f34fb7fe640 1 -- 172.21.15.16:0/762734608 <== mon.1 v2:172.21.15.112:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3504013470 con 0x7f35140caae0 2023-12-08T08:39:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.771+0000 7f34fb7fe640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f350006ae40 0x7f350006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.771+0000 7f34fb7fe640 1 -- 172.21.15.16:0/762734608 <== mon.1 v2:172.21.15.112:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f3504089b10 con 0x7f35140caae0 2023-12-08T08:39:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.772+0000 7f351259c640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f350006ae40 0x7f350006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.772+0000 7f351259c640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f350006ae40 0x7f350006d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f35140ceab0 tx=0x7f350803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:03.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.776+0000 7f34fb7fe640 1 -- 172.21.15.16:0/762734608 <== mon.1 v2:172.21.15.112:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3504054870 con 0x7f35140caae0 2023-12-08T08:39:03.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.961+0000 7f3519141640 1 -- 172.21.15.16:0/762734608 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f35140d0b40 con 0x7f35140caae0 2023-12-08T08:39:03.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.961+0000 7f34fb7fe640 1 -- 172.21.15.16:0/762734608 <== mon.1 v2:172.21.15.112:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f35040170a0 con 0x7f35140caae0 2023-12-08T08:39:03.964 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:03.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.963+0000 7f3519141640 1 -- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f350006ae40 msgr2=0x7f350006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.963+0000 7f3519141640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f350006ae40 0x7f350006d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f35140ceab0 tx=0x7f350803c040 comp rx=0 tx=0).stop 2023-12-08T08:39:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.963+0000 7f3519141640 1 -- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35140caae0 msgr2=0x7f35141568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.963+0000 7f3519141640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35140caae0 0x7f35141568f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f350400e850 tx=0x7f350400ed10 comp rx=0 tx=0).stop 2023-12-08T08:39:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.963+0000 7f3519141640 1 -- 172.21.15.16:0/762734608 shutdown_connections 2023-12-08T08:39:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.963+0000 7f3519141640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f350006ae40 0x7f350006d300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.963+0000 7f3519141640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f351415dbc0 0x7f351406e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.963+0000 7f3519141640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35140caec0 0x7f35140c5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.963+0000 7f3519141640 1 --2- 172.21.15.16:0/762734608 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f35140caae0 0x7f35141568f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.963+0000 7f3519141640 1 -- 172.21.15.16:0/762734608 >> 172.21.15.16:0/762734608 conn(0x7f35140cfcd0 msgr2=0x7f351406a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.963+0000 7f3519141640 1 -- 172.21.15.16:0/762734608 shutdown_connections 2023-12-08T08:39:03.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:03.963+0000 7f3519141640 1 -- 172.21.15.16:0/762734608 wait complete. 2023-12-08T08:39:03.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.081+0000 7efd0412c640 1 Processor -- start 2023-12-08T08:39:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.081+0000 7efd0412c640 1 -- start start 2023-12-08T08:39:04.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd0412c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7efcfc0ccd70 0x7efcfc0ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd0412c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efcfc0cf1e0 0x7efcfc06bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd0412c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcfc06c8f0 con 0x7efcfc06c480 2023-12-08T08:39:04.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd0412c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcfc0d0a60 con 0x7efcfc0cf1e0 2023-12-08T08:39:04.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd0412c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcfc0ce7b0 con 0x7efcfc0ccd70 2023-12-08T08:39:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd016c7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efcfc0cf1e0 0x7efcfc06bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd026c9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efcfc06c480 0x7efcfc15ca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38152/0 (socket says 172.21.15.16:38152) 2023-12-08T08:39:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd026c9640 1 -- 172.21.15.16:0/1348406150 learned_addr learned my addr 172.21.15.16:0/1348406150 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd01ec8640 1 --2- 172.21.15.16:0/1348406150 >> v2:172.21.15.112:3300/0 conn(0x7efcfc0ccd70 0x7efcfc0ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd016c7640 1 -- 172.21.15.16:0/1348406150 >> v2:172.21.15.112:3300/0 conn(0x7efcfc0ccd70 msgr2=0x7efcfc0ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd016c7640 1 --2- 172.21.15.16:0/1348406150 >> v2:172.21.15.112:3300/0 conn(0x7efcfc0ccd70 0x7efcfc0ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd016c7640 1 -- 172.21.15.16:0/1348406150 >> v1:172.21.15.16:6789/0 conn(0x7efcfc06c480 legacy=0x7efcfc15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd00ec6640 1 -- 172.21.15.16:0/1348406150 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355248711 0 0) 0x7efcf4001490 con 0x7efcfc06c480 2023-12-08T08:39:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd016c7640 1 -- 172.21.15.16:0/1348406150 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc0cf9c0 con 0x7efcfc0cf1e0 2023-12-08T08:39:04.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.082+0000 7efd01ec8640 1 --2- 172.21.15.16:0/1348406150 >> v2:172.21.15.112:3300/0 conn(0x7efcfc0ccd70 0x7efcfc0ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.083+0000 7efd016c7640 1 --2- 172.21.15.16:0/1348406150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcfc0cf1e0 0x7efcfc06bf40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efcfc071080 tx=0x7efcec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b8070266cbaba server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:04.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.083+0000 7efd00ec6640 1 -- 172.21.15.16:0/1348406150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec031830 con 0x7efcfc0cf1e0 2023-12-08T08:39:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.083+0000 7efd00ec6640 1 -- 172.21.15.16:0/1348406150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcec031990 con 0x7efcfc0cf1e0 2023-12-08T08:39:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.083+0000 7efd00ec6640 1 -- 172.21.15.16:0/1348406150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcec031d20 con 0x7efcfc0cf1e0 2023-12-08T08:39:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.084+0000 7efd0412c640 1 -- 172.21.15.16:0/1348406150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcfc0cf1e0 msgr2=0x7efcfc06bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.084+0000 7efd0412c640 1 --2- 172.21.15.16:0/1348406150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcfc0cf1e0 0x7efcfc06bf40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efcfc071080 tx=0x7efcec02b8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.084+0000 7efd0412c640 1 -- 172.21.15.16:0/1348406150 shutdown_connections 2023-12-08T08:39:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.084+0000 7efd0412c640 1 --2- 172.21.15.16:0/1348406150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcfc0cf1e0 0x7efcfc06bf40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.084+0000 7efd0412c640 1 --2- 172.21.15.16:0/1348406150 >> v2:172.21.15.112:3300/0 conn(0x7efcfc0ccd70 0x7efcfc0ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.084+0000 7efd0412c640 1 -- 172.21.15.16:0/1348406150 >> 172.21.15.16:0/1348406150 conn(0x7efcfc0d0de0 msgr2=0x7efcfc0d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.084+0000 7efd0412c640 1 -- 172.21.15.16:0/1348406150 shutdown_connections 2023-12-08T08:39:04.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.084+0000 7efd0412c640 1 -- 172.21.15.16:0/1348406150 wait complete. 2023-12-08T08:39:04.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.085+0000 7efd0412c640 1 Processor -- start 2023-12-08T08:39:04.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.085+0000 7efd0412c640 1 -- start start 2023-12-08T08:39:04.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd0412c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcfc06c480 0x7efcfc1f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd0412c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcfc0ccd70 0x7efcfc1f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd0412c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcfc0cf1e0 0x7efcfc0c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd0412c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc154e20 con 0x7efcfc06c480 2023-12-08T08:39:04.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd0412c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0c93d0 con 0x7efcfc0ccd70 2023-12-08T08:39:04.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd0412c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc1f39b0 con 0x7efcfc0cf1e0 2023-12-08T08:39:04.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd01ec8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcfc06c480 0x7efcfc1f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd01ec8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcfc06c480 0x7efcfc1f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54016/0 (socket says 172.21.15.16:54016) 2023-12-08T08:39:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd01ec8640 1 -- 172.21.15.16:0/281915318 learned_addr learned my addr 172.21.15.16:0/281915318 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd016c7640 1 --2- 172.21.15.16:0/281915318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcfc0ccd70 0x7efcfc1f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd01ec8640 1 -- 172.21.15.16:0/281915318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcfc0cf1e0 msgr2=0x7efcfc0c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd01ec8640 1 --2- 172.21.15.16:0/281915318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcfc0cf1e0 0x7efcfc0c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd01ec8640 1 -- 172.21.15.16:0/281915318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcfc0ccd70 msgr2=0x7efcfc1f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd01ec8640 1 --2- 172.21.15.16:0/281915318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcfc0ccd70 0x7efcfc1f3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd01ec8640 1 -- 172.21.15.16:0/281915318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcec009400 con 0x7efcfc06c480 2023-12-08T08:39:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd016c7640 1 --2- 172.21.15.16:0/281915318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcfc0ccd70 0x7efcfc1f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.086+0000 7efd01ec8640 1 --2- 172.21.15.16:0/281915318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcfc06c480 0x7efcfc1f6420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efcf800c7f0 tx=0x7efcf800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.087+0000 7efcf27fc640 1 -- 172.21.15.16:0/281915318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf8007a30 con 0x7efcfc06c480 2023-12-08T08:39:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.087+0000 7efcf27fc640 1 -- 172.21.15.16:0/281915318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf8007b90 con 0x7efcfc06c480 2023-12-08T08:39:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.087+0000 7efcf27fc640 1 -- 172.21.15.16:0/281915318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf800ce40 con 0x7efcfc06c480 2023-12-08T08:39:04.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.087+0000 7efd0412c640 1 -- 172.21.15.16:0/281915318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc0c9620 con 0x7efcfc06c480 2023-12-08T08:39:04.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.087+0000 7efd0412c640 1 -- 172.21.15.16:0/281915318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc0c9be0 con 0x7efcfc06c480 2023-12-08T08:39:04.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.088+0000 7efd0412c640 1 -- 172.21.15.16:0/281915318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc067430 con 0x7efcfc06c480 2023-12-08T08:39:04.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.088+0000 7efcf27fc640 1 -- 172.21.15.16:0/281915318 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efcf80113f0 con 0x7efcfc06c480 2023-12-08T08:39:04.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.088+0000 7efcf27fc640 1 --2- 172.21.15.16:0/281915318 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7efce006ae40 0x7efce006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.089+0000 7efcf27fc640 1 -- 172.21.15.16:0/281915318 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7efcf8089b00 con 0x7efcfc06c480 2023-12-08T08:39:04.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.089+0000 7efd016c7640 1 --2- 172.21.15.16:0/281915318 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7efce006ae40 0x7efce006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.089+0000 7efd016c7640 1 --2- 172.21.15.16:0/281915318 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7efce006ae40 0x7efce006d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efcec036000 tx=0x7efcec008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:04.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.093+0000 7efcf27fc640 1 -- 172.21.15.16:0/281915318 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efcf8054860 con 0x7efcfc06c480 2023-12-08T08:39:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.269+0000 7efd0412c640 1 -- 172.21.15.16:0/281915318 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcfc0cc600 con 0x7efcfc06c480 2023-12-08T08:39:04.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.271+0000 7efcf27fc640 1 -- 172.21.15.16:0/281915318 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7efcf801f090 con 0x7efcfc06c480 2023-12-08T08:39:04.274 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:04.274 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:04.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.273+0000 7efd0412c640 1 -- 172.21.15.16:0/281915318 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7efce006ae40 msgr2=0x7efce006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.273+0000 7efd0412c640 1 --2- 172.21.15.16:0/281915318 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7efce006ae40 0x7efce006d300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efcec036000 tx=0x7efcec008030 comp rx=0 tx=0).stop 2023-12-08T08:39:04.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.273+0000 7efd0412c640 1 -- 172.21.15.16:0/281915318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcfc06c480 msgr2=0x7efcfc1f6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.273+0000 7efd0412c640 1 --2- 172.21.15.16:0/281915318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcfc06c480 0x7efcfc1f6420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efcf800c7f0 tx=0x7efcf800cbf0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.273+0000 7efd0412c640 1 -- 172.21.15.16:0/281915318 shutdown_connections 2023-12-08T08:39:04.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.273+0000 7efd0412c640 1 --2- 172.21.15.16:0/281915318 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7efce006ae40 0x7efce006d300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.273+0000 7efd0412c640 1 --2- 172.21.15.16:0/281915318 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7efcfc0cf1e0 0x7efcfc0c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.273+0000 7efd0412c640 1 --2- 172.21.15.16:0/281915318 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7efcfc0ccd70 0x7efcfc1f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.273+0000 7efd0412c640 1 --2- 172.21.15.16:0/281915318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcfc06c480 0x7efcfc1f6420 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.273+0000 7efd0412c640 1 -- 172.21.15.16:0/281915318 >> 172.21.15.16:0/281915318 conn(0x7efcfc0d0de0 msgr2=0x7efcfc0b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:04.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.273+0000 7efd0412c640 1 -- 172.21.15.16:0/281915318 shutdown_connections 2023-12-08T08:39:04.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.273+0000 7efd0412c640 1 -- 172.21.15.16:0/281915318 wait complete. 2023-12-08T08:39:04.287 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:39:04.287 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.394+0000 7fd91ac17640 1 Processor -- start 2023-12-08T08:39:04.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.394+0000 7fd91ac17640 1 -- start start 2023-12-08T08:39:04.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd91ac17640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd91415ea20 0x7fd9140b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd91ac17640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9140b4df0 0x7fd91415b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd91ac17640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9140cfcc0 con 0x7fd9140ce780 2023-12-08T08:39:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd91ac17640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9140ceb60 con 0x7fd9140b4df0 2023-12-08T08:39:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd91ac17640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140b5280 con 0x7fd91415ea20 2023-12-08T08:39:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd9191b4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9140b4df0 0x7fd91415b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd9189b3640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd9140ce780 0x7fd9140cf0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38166/0 (socket says 172.21.15.16:38166) 2023-12-08T08:39:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd9189b3640 1 -- 172.21.15.16:0/2105983207 learned_addr learned my addr 172.21.15.16:0/2105983207 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd9191b4640 1 --2- 172.21.15.16:0/2105983207 >> v2:172.21.15.16:3300/0 conn(0x7fd9140b4df0 0x7fd91415b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54020/0 (socket says 172.21.15.16:54020) 2023-12-08T08:39:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd913fff640 1 --2- 172.21.15.16:0/2105983207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd91415ea20 0x7fd9140b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd9191b4640 1 -- 172.21.15.16:0/2105983207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd91415ea20 msgr2=0x7fd9140b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd9191b4640 1 --2- 172.21.15.16:0/2105983207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd91415ea20 0x7fd9140b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd9191b4640 1 -- 172.21.15.16:0/2105983207 >> v1:172.21.15.16:6789/0 conn(0x7fd9140ce780 legacy=0x7fd9140cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd9191b4640 1 -- 172.21.15.16:0/2105983207 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91415eea0 con 0x7fd9140b4df0 2023-12-08T08:39:04.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.395+0000 7fd913fff640 1 --2- 172.21.15.16:0/2105983207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd91415ea20 0x7fd9140b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.396+0000 7fd9191b4640 1 --2- 172.21.15.16:0/2105983207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9140b4df0 0x7fd91415b710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009d90 tx=0x7fd90c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69e694eb9b0544a server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.396+0000 7fd9137fe640 1 -- 172.21.15.16:0/2105983207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c02cc50 con 0x7fd9140b4df0 2023-12-08T08:39:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.396+0000 7fd9137fe640 1 -- 172.21.15.16:0/2105983207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c02cdb0 con 0x7fd9140b4df0 2023-12-08T08:39:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.396+0000 7fd9137fe640 1 -- 172.21.15.16:0/2105983207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c034660 con 0x7fd9140b4df0 2023-12-08T08:39:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.397+0000 7fd91ac17640 1 -- 172.21.15.16:0/2105983207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9140b4df0 msgr2=0x7fd91415b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.397+0000 7fd91ac17640 1 --2- 172.21.15.16:0/2105983207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9140b4df0 0x7fd91415b710 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009d90 tx=0x7fd90c02bea0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.397+0000 7fd91ac17640 1 -- 172.21.15.16:0/2105983207 shutdown_connections 2023-12-08T08:39:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.397+0000 7fd91ac17640 1 --2- 172.21.15.16:0/2105983207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9140b4df0 0x7fd91415b710 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.397+0000 7fd91ac17640 1 --2- 172.21.15.16:0/2105983207 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd91415ea20 0x7fd9140b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.397+0000 7fd91ac17640 1 -- 172.21.15.16:0/2105983207 >> 172.21.15.16:0/2105983207 conn(0x7fd91406b7c0 msgr2=0x7fd91406bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.397+0000 7fd91ac17640 1 -- 172.21.15.16:0/2105983207 shutdown_connections 2023-12-08T08:39:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.397+0000 7fd91ac17640 1 -- 172.21.15.16:0/2105983207 wait complete. 2023-12-08T08:39:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.398+0000 7fd91ac17640 1 Processor -- start 2023-12-08T08:39:04.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.398+0000 7fd91ac17640 1 -- start start 2023-12-08T08:39:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd91ac17640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9140b4df0 0x7fd9140af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd91ac17640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9140ce780 0x7fd91414e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd91ac17640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91415ea20 0x7fd914153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd91ac17640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141e6180 con 0x7fd91415ea20 2023-12-08T08:39:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd91ac17640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141e6460 con 0x7fd9140b4df0 2023-12-08T08:39:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd91ac17640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9141e6740 con 0x7fd9140ce780 2023-12-08T08:39:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd9191b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91415ea20 0x7fd914153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd9191b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91415ea20 0x7fd914153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54036/0 (socket says 172.21.15.16:54036) 2023-12-08T08:39:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd9191b4640 1 -- 172.21.15.16:0/325029629 learned_addr learned my addr 172.21.15.16:0/325029629 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd9189b3640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9140b4df0 0x7fd9140af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd913fff640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9140ce780 0x7fd91414e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd9191b4640 1 -- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9140ce780 msgr2=0x7fd91414e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd9191b4640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9140ce780 0x7fd91414e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd9191b4640 1 -- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9140b4df0 msgr2=0x7fd9140af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd9191b4640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9140b4df0 0x7fd9140af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd9191b4640 1 -- 172.21.15.16:0/325029629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c009a70 con 0x7fd91415ea20 2023-12-08T08:39:04.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd913fff640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9140ce780 0x7fd91414e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:04.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd9189b3640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9140b4df0 0x7fd9140af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:04.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.399+0000 7fd9191b4640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91415ea20 0x7fd914153df0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009d90 tx=0x7fd90c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:04.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.400+0000 7fd9117fa640 1 -- 172.21.15.16:0/325029629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c040650 con 0x7fd91415ea20 2023-12-08T08:39:04.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.400+0000 7fd9117fa640 1 -- 172.21.15.16:0/325029629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c0407d0 con 0x7fd91415ea20 2023-12-08T08:39:04.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.400+0000 7fd91ac17640 1 -- 172.21.15.16:0/325029629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9141e6a20 con 0x7fd91415ea20 2023-12-08T08:39:04.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.400+0000 7fd9117fa640 1 -- 172.21.15.16:0/325029629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c040b80 con 0x7fd91415ea20 2023-12-08T08:39:04.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.400+0000 7fd91ac17640 1 -- 172.21.15.16:0/325029629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9140b0890 con 0x7fd91415ea20 2023-12-08T08:39:04.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.401+0000 7fd91ac17640 1 -- 172.21.15.16:0/325029629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9140b0a20 con 0x7fd91415ea20 2023-12-08T08:39:04.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.401+0000 7fd9117fa640 1 -- 172.21.15.16:0/325029629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd90c040ce0 con 0x7fd91415ea20 2023-12-08T08:39:04.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.402+0000 7fd9117fa640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd8f806ae40 0x7fd8f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.402+0000 7fd9117fa640 1 -- 172.21.15.16:0/325029629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fd90c0ac810 con 0x7fd91415ea20 2023-12-08T08:39:04.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.402+0000 7fd9189b3640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd8f806ae40 0x7fd8f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.402+0000 7fd9189b3640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd8f806ae40 0x7fd8f806d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd908007fd0 tx=0x7fd908009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:04.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.406+0000 7fd9117fa640 1 -- 172.21.15.16:0/325029629 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd90c0774e0 con 0x7fd91415ea20 2023-12-08T08:39:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.580+0000 7fd91ac17640 1 -- 172.21.15.16:0/325029629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9141e6f40 con 0x7fd91415ea20 2023-12-08T08:39:04.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.580+0000 7fd9117fa640 1 -- 172.21.15.16:0/325029629 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9141e6f40 con 0x7fd91415ea20 2023-12-08T08:39:04.583 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.582+0000 7fd91ac17640 1 -- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd8f806ae40 msgr2=0x7fd8f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.582+0000 7fd91ac17640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd8f806ae40 0x7fd8f806d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd908007fd0 tx=0x7fd908009040 comp rx=0 tx=0).stop 2023-12-08T08:39:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.582+0000 7fd91ac17640 1 -- 172.21.15.16:0/325029629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91415ea20 msgr2=0x7fd914153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.582+0000 7fd91ac17640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91415ea20 0x7fd914153df0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009d90 tx=0x7fd90c004a40 comp rx=0 tx=0).stop 2023-12-08T08:39:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.582+0000 7fd91ac17640 1 -- 172.21.15.16:0/325029629 shutdown_connections 2023-12-08T08:39:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.582+0000 7fd91ac17640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd8f806ae40 0x7fd8f806d300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.582+0000 7fd91ac17640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd91415ea20 0x7fd914153df0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.582+0000 7fd91ac17640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd9140ce780 0x7fd91414e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.582+0000 7fd91ac17640 1 --2- 172.21.15.16:0/325029629 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd9140b4df0 0x7fd9140af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.582+0000 7fd91ac17640 1 -- 172.21.15.16:0/325029629 >> 172.21.15.16:0/325029629 conn(0x7fd91406b7c0 msgr2=0x7fd91415b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.582+0000 7fd91ac17640 1 -- 172.21.15.16:0/325029629 shutdown_connections 2023-12-08T08:39:04.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.582+0000 7fd91ac17640 1 -- 172.21.15.16:0/325029629 wait complete. 2023-12-08T08:39:04.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:39:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.702+0000 7fcc69eda640 1 Processor -- start 2023-12-08T08:39:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.702+0000 7fcc69eda640 1 -- start start 2023-12-08T08:39:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.703+0000 7fcc69eda640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc6415e790 0x7fcc640ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.703+0000 7fcc69eda640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc640cce00 con 0x7fcc640cc380 2023-12-08T08:39:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.703+0000 7fcc69eda640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc640b0f00 con 0x7fcc640cde00 2023-12-08T08:39:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.703+0000 7fcc69eda640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640ccb20 con 0x7fcc6415e790 2023-12-08T08:39:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.703+0000 7fcc62ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcc640cc380 0x7fcc640cd6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38178/0 (socket says 172.21.15.16:38178) 2023-12-08T08:39:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.703+0000 7fcc62ffd640 1 -- 172.21.15.16:0/3525121433 learned_addr learned my addr 172.21.15.16:0/3525121433 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc637fe640 1 --2- 172.21.15.16:0/3525121433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc6415e790 0x7fcc640ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 -- 172.21.15.16:0/3525121433 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665838372 0 0) 0x7fcc640cce00 con 0x7fcc640cc380 2023-12-08T08:39:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 -- 172.21.15.16:0/3525121433 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc4c003520 con 0x7fcc640cc380 2023-12-08T08:39:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 -- 172.21.15.16:0/3525121433 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3386948684 0 0) 0x7fcc4c003520 con 0x7fcc640cc380 2023-12-08T08:39:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 -- 172.21.15.16:0/3525121433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc6415e790 msgr2=0x7fcc640ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 --2- 172.21.15.16:0/3525121433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc6415e790 0x7fcc640ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 -- 172.21.15.16:0/3525121433 >> v1:172.21.15.112:6789/0 conn(0x7fcc640cde00 legacy=0x7fcc640b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 -- 172.21.15.16:0/3525121433 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc640b08b0 con 0x7fcc640cc380 2023-12-08T08:39:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 -- 172.21.15.16:0/3525121433 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7fcc58001670 con 0x7fcc640cc380 2023-12-08T08:39:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 -- 172.21.15.16:0/3525121433 >> v1:172.21.15.16:6789/0 conn(0x7fcc640cc380 legacy=0x7fcc640cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 --2- 172.21.15.16:0/3525121433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc4c0039a0 0x7fcc4c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 --2- 172.21.15.16:0/3525121433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc6415e790 0x7fcc4c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 --2- 172.21.15.16:0/3525121433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c004d00 0x7fcc4c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 -- 172.21.15.16:0/3525121433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640ccb20 con 0x7fcc4c004d00 2023-12-08T08:39:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 -- 172.21.15.16:0/3525121433 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c00ba30 con 0x7fcc4c0039a0 2023-12-08T08:39:04.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc627fc640 1 -- 172.21.15.16:0/3525121433 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4c00bbf0 con 0x7fcc6415e790 2023-12-08T08:39:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc63fff640 1 --2- 172.21.15.16:0/3525121433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c004d00 0x7fcc4c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc62ffd640 1 --2- 172.21.15.16:0/3525121433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc6415e790 0x7fcc4c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc637fe640 1 --2- 172.21.15.16:0/3525121433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc4c0039a0 0x7fcc4c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc63fff640 1 -- 172.21.15.16:0/3525121433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc6415e790 msgr2=0x7fcc4c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc63fff640 1 --2- 172.21.15.16:0/3525121433 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc6415e790 0x7fcc4c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc63fff640 1 -- 172.21.15.16:0/3525121433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc4c0039a0 msgr2=0x7fcc4c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc63fff640 1 --2- 172.21.15.16:0/3525121433 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc4c0039a0 0x7fcc4c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.704+0000 7fcc63fff640 1 -- 172.21.15.16:0/3525121433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c003520 con 0x7fcc4c004d00 2023-12-08T08:39:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.705+0000 7fcc63fff640 1 --2- 172.21.15.16:0/3525121433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c004d00 0x7fcc4c005180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c001fe0 tx=0x7fcc4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7d370c6d6020b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.705+0000 7fcc627fc640 1 -- 172.21.15.16:0/3525121433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c02c730 con 0x7fcc4c004d00 2023-12-08T08:39:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.705+0000 7fcc627fc640 1 -- 172.21.15.16:0/3525121433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c02c890 con 0x7fcc4c004d00 2023-12-08T08:39:04.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.705+0000 7fcc627fc640 1 -- 172.21.15.16:0/3525121433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c02cc20 con 0x7fcc4c004d00 2023-12-08T08:39:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.706+0000 7fcc69eda640 1 -- 172.21.15.16:0/3525121433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c004d00 msgr2=0x7fcc4c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.706+0000 7fcc69eda640 1 --2- 172.21.15.16:0/3525121433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc4c004d00 0x7fcc4c005180 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c001fe0 tx=0x7fcc4c0057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.706+0000 7fcc63fff640 1 -- 172.21.15.16:0/3525121433 reap_dead start 2023-12-08T08:39:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.706+0000 7fcc69eda640 1 -- 172.21.15.16:0/3525121433 shutdown_connections 2023-12-08T08:39:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.706+0000 7fcc69eda640 1 -- 172.21.15.16:0/3525121433 >> 172.21.15.16:0/3525121433 conn(0x7fcc640d0e00 msgr2=0x7fcc640d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.706+0000 7fcc69eda640 1 -- 172.21.15.16:0/3525121433 shutdown_connections 2023-12-08T08:39:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.706+0000 7fcc69eda640 1 -- 172.21.15.16:0/3525121433 wait complete. 2023-12-08T08:39:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.707+0000 7fcc69eda640 1 Processor -- start 2023-12-08T08:39:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.707+0000 7fcc69eda640 1 -- start start 2023-12-08T08:39:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.708+0000 7fcc69eda640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc640b5ed0 0x7fcc6414b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.708+0000 7fcc69eda640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc640b62b0 0x7fcc641f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.708+0000 7fcc69eda640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc641f44f0 0x7fcc640af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.708+0000 7fcc69eda640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6414b540 con 0x7fcc641f44f0 2023-12-08T08:39:04.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.708+0000 7fcc69eda640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6414bea0 con 0x7fcc640b5ed0 2023-12-08T08:39:04.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.708+0000 7fcc69eda640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6414aec0 con 0x7fcc640b62b0 2023-12-08T08:39:04.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.708+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc641f44f0 0x7fcc640af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.708+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc641f44f0 0x7fcc640af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54050/0 (socket says 172.21.15.16:54050) 2023-12-08T08:39:04.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.708+0000 7fcc63fff640 1 -- 172.21.15.16:0/1279110110 learned_addr learned my addr 172.21.15.16:0/1279110110 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:04.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.708+0000 7fcc637fe640 1 --2- 172.21.15.16:0/1279110110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc640b5ed0 0x7fcc6414b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.708+0000 7fcc62ffd640 1 --2- 172.21.15.16:0/1279110110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc640b62b0 0x7fcc641f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.708+0000 7fcc63fff640 1 -- 172.21.15.16:0/1279110110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc640b62b0 msgr2=0x7fcc641f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.708+0000 7fcc63fff640 1 --2- 172.21.15.16:0/1279110110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc640b62b0 0x7fcc641f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.709+0000 7fcc63fff640 1 -- 172.21.15.16:0/1279110110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc640b5ed0 msgr2=0x7fcc6414b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.709+0000 7fcc63fff640 1 --2- 172.21.15.16:0/1279110110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc640b5ed0 0x7fcc6414b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.709+0000 7fcc63fff640 1 -- 172.21.15.16:0/1279110110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc641f4a50 con 0x7fcc641f44f0 2023-12-08T08:39:04.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.709+0000 7fcc63fff640 1 --2- 172.21.15.16:0/1279110110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc641f44f0 0x7fcc640af810 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c001fe0 tx=0x7fcc5c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:04.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.709+0000 7fcc4bfff640 1 -- 172.21.15.16:0/1279110110 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c003b70 con 0x7fcc641f44f0 2023-12-08T08:39:04.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.709+0000 7fcc4bfff640 1 -- 172.21.15.16:0/1279110110 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c003cd0 con 0x7fcc641f44f0 2023-12-08T08:39:04.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.709+0000 7fcc69eda640 1 -- 172.21.15.16:0/1279110110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc641f4d30 con 0x7fcc641f44f0 2023-12-08T08:39:04.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.709+0000 7fcc4bfff640 1 -- 172.21.15.16:0/1279110110 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c005290 con 0x7fcc641f44f0 2023-12-08T08:39:04.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.709+0000 7fcc69eda640 1 -- 172.21.15.16:0/1279110110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6414ebf0 con 0x7fcc641f44f0 2023-12-08T08:39:04.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.710+0000 7fcc69eda640 1 -- 172.21.15.16:0/1279110110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6414b680 con 0x7fcc641f44f0 2023-12-08T08:39:04.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.711+0000 7fcc4bfff640 1 -- 172.21.15.16:0/1279110110 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc5c03b040 con 0x7fcc641f44f0 2023-12-08T08:39:04.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.711+0000 7fcc4bfff640 1 --2- 172.21.15.16:0/1279110110 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcc5006ae70 0x7fcc5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:04.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.711+0000 7fcc4bfff640 1 -- 172.21.15.16:0/1279110110 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fcc5c0aa310 con 0x7fcc641f44f0 2023-12-08T08:39:04.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.711+0000 7fcc637fe640 1 --2- 172.21.15.16:0/1279110110 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcc5006ae70 0x7fcc5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:04.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.712+0000 7fcc637fe640 1 --2- 172.21.15.16:0/1279110110 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcc5006ae70 0x7fcc5006d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc6414f490 tx=0x7fcc54009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:04.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.716+0000 7fcc4bfff640 1 -- 172.21.15.16:0/1279110110 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc6414b680 con 0x7fcc641f44f0 2023-12-08T08:39:04.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.892+0000 7fcc69eda640 1 -- 172.21.15.16:0/1279110110 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc6414b680 con 0x7fcc641f44f0 2023-12-08T08:39:04.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.892+0000 7fcc4bfff640 1 -- 172.21.15.16:0/1279110110 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc6414b680 con 0x7fcc641f44f0 2023-12-08T08:39:04.895 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.894+0000 7fcc69eda640 1 -- 172.21.15.16:0/1279110110 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcc5006ae70 msgr2=0x7fcc5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.894+0000 7fcc69eda640 1 --2- 172.21.15.16:0/1279110110 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcc5006ae70 0x7fcc5006d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc6414f490 tx=0x7fcc54009190 comp rx=0 tx=0).stop 2023-12-08T08:39:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.894+0000 7fcc69eda640 1 -- 172.21.15.16:0/1279110110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc641f44f0 msgr2=0x7fcc640af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.894+0000 7fcc69eda640 1 --2- 172.21.15.16:0/1279110110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc641f44f0 0x7fcc640af810 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c001fe0 tx=0x7fcc5c001630 comp rx=0 tx=0).stop 2023-12-08T08:39:04.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.894+0000 7fcc69eda640 1 -- 172.21.15.16:0/1279110110 shutdown_connections 2023-12-08T08:39:04.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.894+0000 7fcc69eda640 1 --2- 172.21.15.16:0/1279110110 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcc5006ae70 0x7fcc5006d330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.894+0000 7fcc69eda640 1 --2- 172.21.15.16:0/1279110110 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc641f44f0 0x7fcc640af810 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.894+0000 7fcc69eda640 1 --2- 172.21.15.16:0/1279110110 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcc640b62b0 0x7fcc641f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.894+0000 7fcc69eda640 1 --2- 172.21.15.16:0/1279110110 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcc640b5ed0 0x7fcc6414b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:04.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.894+0000 7fcc69eda640 1 -- 172.21.15.16:0/1279110110 >> 172.21.15.16:0/1279110110 conn(0x7fcc640d0e00 msgr2=0x7fcc6406e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:04.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.894+0000 7fcc69eda640 1 -- 172.21.15.16:0/1279110110 shutdown_connections 2023-12-08T08:39:04.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:04.894+0000 7fcc69eda640 1 -- 172.21.15.16:0/1279110110 wait complete. 2023-12-08T08:39:04.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:39:05.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.014+0000 7f7d58b97640 1 Processor -- start 2023-12-08T08:39:05.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.014+0000 7f7d58b97640 1 -- start start 2023-12-08T08:39:05.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.015+0000 7f7d58b97640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d54157a20 0x7f7d5415d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.015+0000 7f7d58b97640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7d5406d7d0 0x7f7d540b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.015+0000 7f7d58b97640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d54157e00 con 0x7f7d540cb370 2023-12-08T08:39:05.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.015+0000 7f7d58b97640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5406dd20 con 0x7f7d5406d7d0 2023-12-08T08:39:05.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.015+0000 7f7d58b97640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d54070cd0 con 0x7f7d54157a20 2023-12-08T08:39:05.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.015+0000 7f7d52d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7d5406d7d0 0x7f7d540b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.015+0000 7f7d5259c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d54157a20 0x7f7d5415d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.015+0000 7f7d52d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7d5406d7d0 0x7f7d540b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:54550/0 (socket says 172.21.15.16:54550) 2023-12-08T08:39:05.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.015+0000 7f7d52d9d640 1 -- 172.21.15.16:0/2963101752 learned_addr learned my addr 172.21.15.16:0/2963101752 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:05.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.015+0000 7f7d4bfff640 1 --1- 172.21.15.16:0/2963101752 >> v1:172.21.15.112:6789/0 conn(0x7f7d540cb370 0x7f7d5415df10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:33992/0 (socket says 172.21.15.16:33992) 2023-12-08T08:39:05.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.016+0000 7f7d52d9d640 1 -- 172.21.15.16:0/2963101752 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d54157a20 msgr2=0x7f7d5415d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:05.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.016+0000 7f7d52d9d640 1 --2- 172.21.15.16:0/2963101752 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d54157a20 0x7f7d5415d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.016+0000 7f7d52d9d640 1 -- 172.21.15.16:0/2963101752 >> v1:172.21.15.112:6789/0 conn(0x7f7d540cb370 legacy=0x7f7d5415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.016+0000 7f7d51d9b640 1 -- 172.21.15.16:0/2963101752 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218745345 0 0) 0x7f7d54157e00 con 0x7f7d540cb370 2023-12-08T08:39:05.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.016+0000 7f7d52d9d640 1 -- 172.21.15.16:0/2963101752 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d54070fb0 con 0x7f7d5406d7d0 2023-12-08T08:39:05.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.017+0000 7f7d52d9d640 1 --2- 172.21.15.16:0/2963101752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d5406d7d0 0x7f7d540b1b50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009e50 tx=0x7f7d4c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22a249d58eb23f0a server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:05.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.017+0000 7f7d51d9b640 1 -- 172.21.15.16:0/2963101752 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c02cb00 con 0x7f7d5406d7d0 2023-12-08T08:39:05.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.017+0000 7f7d51d9b640 1 -- 172.21.15.16:0/2963101752 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c02cc60 con 0x7f7d5406d7d0 2023-12-08T08:39:05.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.017+0000 7f7d51d9b640 1 -- 172.21.15.16:0/2963101752 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c0343f0 con 0x7f7d5406d7d0 2023-12-08T08:39:05.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.017+0000 7f7d58b97640 1 -- 172.21.15.16:0/2963101752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d5406d7d0 msgr2=0x7f7d540b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.017+0000 7f7d58b97640 1 --2- 172.21.15.16:0/2963101752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d5406d7d0 0x7f7d540b1b50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009e50 tx=0x7f7d4c02bf60 comp rx=0 tx=0).stop 2023-12-08T08:39:05.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.017+0000 7f7d58b97640 1 -- 172.21.15.16:0/2963101752 shutdown_connections 2023-12-08T08:39:05.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.017+0000 7f7d58b97640 1 --2- 172.21.15.16:0/2963101752 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d5406d7d0 0x7f7d540b1b50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.017+0000 7f7d58b97640 1 --2- 172.21.15.16:0/2963101752 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d54157a20 0x7f7d5415d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.017+0000 7f7d58b97640 1 -- 172.21.15.16:0/2963101752 >> 172.21.15.16:0/2963101752 conn(0x7f7d540b0850 msgr2=0x7f7d540b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:05.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.017+0000 7f7d58b97640 1 -- 172.21.15.16:0/2963101752 shutdown_connections 2023-12-08T08:39:05.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.017+0000 7f7d58b97640 1 -- 172.21.15.16:0/2963101752 wait complete. 2023-12-08T08:39:05.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.019+0000 7f7d58b97640 1 Processor -- start 2023-12-08T08:39:05.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.019+0000 7f7d58b97640 1 -- start start 2023-12-08T08:39:05.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.019+0000 7f7d58b97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5406d7d0 0x7f7d540cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.019+0000 7f7d58b97640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d540cb370 0x7f7d540cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.019+0000 7f7d58b97640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d54157a20 0x7f7d54150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.019+0000 7f7d58b97640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d541480b0 con 0x7f7d5406d7d0 2023-12-08T08:39:05.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.019+0000 7f7d58b97640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d54158a10 con 0x7f7d54157a20 2023-12-08T08:39:05.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.019+0000 7f7d58b97640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d54157e00 con 0x7f7d540cb370 2023-12-08T08:39:05.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d5259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5406d7d0 0x7f7d540cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d5259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5406d7d0 0x7f7d540cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54058/0 (socket says 172.21.15.16:54058) 2023-12-08T08:39:05.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d5259c640 1 -- 172.21.15.16:0/410151590 learned_addr learned my addr 172.21.15.16:0/410151590 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:05.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d4bfff640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d540cb370 0x7f7d540cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d52d9d640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d54157a20 0x7f7d54150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d5259c640 1 -- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d540cb370 msgr2=0x7f7d540cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d5259c640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d540cb370 0x7f7d540cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d5259c640 1 -- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d54157a20 msgr2=0x7f7d54150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d5259c640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d54157a20 0x7f7d54150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d5259c640 1 -- 172.21.15.16:0/410151590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4c009b30 con 0x7f7d5406d7d0 2023-12-08T08:39:05.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d52d9d640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d54157a20 0x7f7d54150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:05.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d5259c640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5406d7d0 0x7f7d540cd260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400e870 tx=0x7f7d4400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:05.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d4affd640 1 -- 172.21.15.16:0/410151590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d440094f0 con 0x7f7d5406d7d0 2023-12-08T08:39:05.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d4affd640 1 -- 172.21.15.16:0/410151590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4400c940 con 0x7f7d5406d7d0 2023-12-08T08:39:05.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d4affd640 1 -- 172.21.15.16:0/410151590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4400ccf0 con 0x7f7d5406d7d0 2023-12-08T08:39:05.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d58b97640 1 -- 172.21.15.16:0/410151590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d541580e0 con 0x7f7d5406d7d0 2023-12-08T08:39:05.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.020+0000 7f7d58b97640 1 -- 172.21.15.16:0/410151590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d541586a0 con 0x7f7d5406d7d0 2023-12-08T08:39:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.021+0000 7f7d58b97640 1 -- 172.21.15.16:0/410151590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5414b150 con 0x7f7d5406d7d0 2023-12-08T08:39:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.022+0000 7f7d4affd640 1 -- 172.21.15.16:0/410151590 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d44013470 con 0x7f7d5406d7d0 2023-12-08T08:39:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.022+0000 7f7d4affd640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7d3406ae40 0x7f7d3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.022+0000 7f7d4affd640 1 -- 172.21.15.16:0/410151590 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f7d44089a10 con 0x7f7d5406d7d0 2023-12-08T08:39:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.022+0000 7f7d4bfff640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7d3406ae40 0x7f7d3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.023+0000 7f7d4bfff640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7d3406ae40 0x7f7d3406d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7d54071eb0 tx=0x7f7d3c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:05.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.026+0000 7f7d4affd640 1 -- 172.21.15.16:0/410151590 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7d44054770 con 0x7f7d5406d7d0 2023-12-08T08:39:05.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.202+0000 7f7d58b97640 1 -- 172.21.15.16:0/410151590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7d54158830 con 0x7f7d5406d7d0 2023-12-08T08:39:05.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.203+0000 7f7d4affd640 1 -- 172.21.15.16:0/410151590 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7d44057d90 con 0x7f7d5406d7d0 2023-12-08T08:39:05.206 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:39:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.204+0000 7f7d58b97640 1 -- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7d3406ae40 msgr2=0x7f7d3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.204+0000 7f7d58b97640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7d3406ae40 0x7f7d3406d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7d54071eb0 tx=0x7f7d3c009040 comp rx=0 tx=0).stop 2023-12-08T08:39:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.204+0000 7f7d58b97640 1 -- 172.21.15.16:0/410151590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5406d7d0 msgr2=0x7f7d540cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.204+0000 7f7d58b97640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5406d7d0 0x7f7d540cd260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7d4400e870 tx=0x7f7d4400ed30 comp rx=0 tx=0).stop 2023-12-08T08:39:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.205+0000 7f7d58b97640 1 -- 172.21.15.16:0/410151590 shutdown_connections 2023-12-08T08:39:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.205+0000 7f7d58b97640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7d3406ae40 0x7f7d3406d300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.205+0000 7f7d58b97640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7d54157a20 0x7f7d54150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.205+0000 7f7d58b97640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7d540cb370 0x7f7d540cd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.205+0000 7f7d58b97640 1 --2- 172.21.15.16:0/410151590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5406d7d0 0x7f7d540cd260 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.205+0000 7f7d58b97640 1 -- 172.21.15.16:0/410151590 >> 172.21.15.16:0/410151590 conn(0x7f7d540b0850 msgr2=0x7f7d540b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.205+0000 7f7d58b97640 1 -- 172.21.15.16:0/410151590 shutdown_connections 2023-12-08T08:39:05.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.205+0000 7f7d58b97640 1 -- 172.21.15.16:0/410151590 wait complete. 2023-12-08T08:39:05.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:39:05.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.324+0000 7fb6f3cc9640 1 Processor -- start 2023-12-08T08:39:05.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.325+0000 7fb6f3cc9640 1 -- start start 2023-12-08T08:39:05.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.325+0000 7fb6f3cc9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6ec0d1950 0x7fb6ec0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.325+0000 7fb6f3cc9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ec0ceba0 con 0x7fb6ec0cc9f0 2023-12-08T08:39:05.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.325+0000 7fb6f3cc9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ec0cee80 con 0x7fb6ec15e6a0 2023-12-08T08:39:05.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.325+0000 7fb6f3cc9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ec0cbcb0 con 0x7fb6ec0d1950 2023-12-08T08:39:05.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.325+0000 7fb6f1264640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6ec0d1950 0x7fb6ec0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.326+0000 7fb6f2266640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb6ec0cc9f0 0x7fb6ec0b3c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38192/0 (socket says 172.21.15.16:38192) 2023-12-08T08:39:05.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.326+0000 7fb6f2266640 1 -- 172.21.15.16:0/1113377514 learned_addr learned my addr 172.21.15.16:0/1113377514 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:05.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.326+0000 7fb6f1264640 1 -- 172.21.15.16:0/1113377514 >> v1:172.21.15.112:6789/0 conn(0x7fb6ec15e6a0 legacy=0x7fb6ec0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:05.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.326+0000 7fb6f1264640 1 -- 172.21.15.16:0/1113377514 >> v1:172.21.15.16:6789/0 conn(0x7fb6ec0cc9f0 legacy=0x7fb6ec0b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.326+0000 7fb6f0a63640 1 -- 172.21.15.16:0/1113377514 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064259233 0 0) 0x7fb6ec0ceba0 con 0x7fb6ec0cc9f0 2023-12-08T08:39:05.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.326+0000 7fb6f1264640 1 -- 172.21.15.16:0/1113377514 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec0cbf90 con 0x7fb6ec0d1950 2023-12-08T08:39:05.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.327+0000 7fb6f1264640 1 --2- 172.21.15.16:0/1113377514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ec0d1950 0x7fb6ec0cc4b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0097a0 tx=0x7fb6dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281e1c1081113ab8 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:05.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.327+0000 7fb6f0a63640 1 -- 172.21.15.16:0/1113377514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc031830 con 0x7fb6ec0d1950 2023-12-08T08:39:05.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.327+0000 7fb6f0a63640 1 -- 172.21.15.16:0/1113377514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc031990 con 0x7fb6ec0d1950 2023-12-08T08:39:05.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.327+0000 7fb6f0a63640 1 -- 172.21.15.16:0/1113377514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc031d20 con 0x7fb6ec0d1950 2023-12-08T08:39:05.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.327+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/1113377514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ec0d1950 msgr2=0x7fb6ec0cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.327+0000 7fb6f3cc9640 1 --2- 172.21.15.16:0/1113377514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ec0d1950 0x7fb6ec0cc4b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0097a0 tx=0x7fb6dc02b8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.327+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/1113377514 shutdown_connections 2023-12-08T08:39:05.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.327+0000 7fb6f3cc9640 1 --2- 172.21.15.16:0/1113377514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ec0d1950 0x7fb6ec0cc4b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.327+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/1113377514 >> 172.21.15.16:0/1113377514 conn(0x7fb6ec0d0e30 msgr2=0x7fb6ec0d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:05.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.327+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/1113377514 shutdown_connections 2023-12-08T08:39:05.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.327+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/1113377514 wait complete. 2023-12-08T08:39:05.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.329+0000 7fb6f3cc9640 1 Processor -- start 2023-12-08T08:39:05.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.329+0000 7fb6f3cc9640 1 -- start start 2023-12-08T08:39:05.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.329+0000 7fb6f3cc9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec0cc9f0 0x7fb6ec14b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.329+0000 7fb6f3cc9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6ec0d1950 0x7fb6ec0c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.329+0000 7fb6f3cc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ec15e6a0 0x7fb6ec0c5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.329+0000 7fb6f3cc9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0ca4e0 con 0x7fb6ec15e6a0 2023-12-08T08:39:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.329+0000 7fb6f3cc9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0c49c0 con 0x7fb6ec0cc9f0 2023-12-08T08:39:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.329+0000 7fb6f3cc9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0c4ca0 con 0x7fb6ec0d1950 2023-12-08T08:39:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.329+0000 7fb6f2266640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ec15e6a0 0x7fb6ec0c5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.329+0000 7fb6f2266640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ec15e6a0 0x7fb6ec0c5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54086/0 (socket says 172.21.15.16:54086) 2023-12-08T08:39:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.329+0000 7fb6f2266640 1 -- 172.21.15.16:0/2937340872 learned_addr learned my addr 172.21.15.16:0/2937340872 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6f1264640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6ec0d1950 0x7fb6ec0c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6f1a65640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec0cc9f0 0x7fb6ec14b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6f2266640 1 -- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6ec0d1950 msgr2=0x7fb6ec0c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6f2266640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6ec0d1950 0x7fb6ec0c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6f2266640 1 -- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec0cc9f0 msgr2=0x7fb6ec14b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6f2266640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec0cc9f0 0x7fb6ec14b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6f2266640 1 -- 172.21.15.16:0/2937340872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6dc009400 con 0x7fb6ec15e6a0 2023-12-08T08:39:05.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6f1264640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6ec0d1950 0x7fb6ec0c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:05.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6f1a65640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec0cc9f0 0x7fb6ec14b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:05.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6f2266640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ec15e6a0 0x7fb6ec0c5780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400c7e0 tx=0x7fb6e400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:05.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6e27fc640 1 -- 172.21.15.16:0/2937340872 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e400a640 con 0x7fb6ec15e6a0 2023-12-08T08:39:05.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6e27fc640 1 -- 172.21.15.16:0/2937340872 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e400a7c0 con 0x7fb6ec15e6a0 2023-12-08T08:39:05.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/2937340872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ec0c4f80 con 0x7fb6ec15e6a0 2023-12-08T08:39:05.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6e27fc640 1 -- 172.21.15.16:0/2937340872 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e400ab70 con 0x7fb6ec15e6a0 2023-12-08T08:39:05.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.330+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/2937340872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ec14b680 con 0x7fb6ec15e6a0 2023-12-08T08:39:05.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.331+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/2937340872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ec0c54a0 con 0x7fb6ec15e6a0 2023-12-08T08:39:05.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.332+0000 7fb6e27fc640 1 -- 172.21.15.16:0/2937340872 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb6e400acd0 con 0x7fb6ec15e6a0 2023-12-08T08:39:05.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.332+0000 7fb6e27fc640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb6d006ae40 0x7fb6d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.332+0000 7fb6e27fc640 1 -- 172.21.15.16:0/2937340872 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fb6e4089200 con 0x7fb6ec15e6a0 2023-12-08T08:39:05.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.332+0000 7fb6f1a65640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb6d006ae40 0x7fb6d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.333+0000 7fb6f1a65640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb6d006ae40 0x7fb6d006d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8006fd0 tx=0x7fb6e8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:05.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.336+0000 7fb6e27fc640 1 -- 172.21.15.16:0/2937340872 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6ec0c54a0 con 0x7fb6ec15e6a0 2023-12-08T08:39:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.511+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/2937340872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb6ec0c54a0 con 0x7fb6ec15e6a0 2023-12-08T08:39:05.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.512+0000 7fb6e27fc640 1 -- 172.21.15.16:0/2937340872 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb6ec0c54a0 con 0x7fb6ec15e6a0 2023-12-08T08:39:05.515 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.513+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb6d006ae40 msgr2=0x7fb6d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.513+0000 7fb6f3cc9640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb6d006ae40 0x7fb6d006d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb6e8006fd0 tx=0x7fb6e8008040 comp rx=0 tx=0).stop 2023-12-08T08:39:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.513+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/2937340872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ec15e6a0 msgr2=0x7fb6ec0c5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.513+0000 7fb6f3cc9640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ec15e6a0 0x7fb6ec0c5780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400c7e0 tx=0x7fb6e400cca0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.513+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/2937340872 shutdown_connections 2023-12-08T08:39:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.513+0000 7fb6f3cc9640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb6d006ae40 0x7fb6d006d300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.513+0000 7fb6f3cc9640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6ec15e6a0 0x7fb6ec0c5780 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.513+0000 7fb6f3cc9640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb6ec0d1950 0x7fb6ec0c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.513+0000 7fb6f3cc9640 1 --2- 172.21.15.16:0/2937340872 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb6ec0cc9f0 0x7fb6ec14b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.513+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/2937340872 >> 172.21.15.16:0/2937340872 conn(0x7fb6ec0d0e30 msgr2=0x7fb6ec15a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.513+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/2937340872 shutdown_connections 2023-12-08T08:39:05.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.514+0000 7fb6f3cc9640 1 -- 172.21.15.16:0/2937340872 wait complete. 2023-12-08T08:39:05.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.631+0000 7f12c5583640 1 Processor -- start 2023-12-08T08:39:05.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.631+0000 7f12c5583640 1 -- start start 2023-12-08T08:39:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.632+0000 7f12c5583640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12c00ce3e0 0x7f12c00b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.632+0000 7f12c5583640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12c0071e50 0x7f12c00b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.632+0000 7f12c5583640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c00b29b0 con 0x7f12c0140a10 2023-12-08T08:39:05.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.632+0000 7f12c5583640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c0070ba0 con 0x7f12c00ce3e0 2023-12-08T08:39:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.632+0000 7f12c5583640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c0070db0 con 0x7f12c0071e50 2023-12-08T08:39:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.632+0000 7f12b67fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12c00ce3e0 0x7f12c00b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.632+0000 7f12b67fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12c00ce3e0 0x7f12c00b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54094/0 (socket says 172.21.15.16:54094) 2023-12-08T08:39:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.632+0000 7f12b67fc640 1 -- 172.21.15.16:0/533206360 learned_addr learned my addr 172.21.15.16:0/533206360 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.632+0000 7f12bf7fe640 1 --2- 172.21.15.16:0/533206360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12c0071e50 0x7f12c00b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.632+0000 7f12b67fc640 1 -- 172.21.15.16:0/533206360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12c0071e50 msgr2=0x7f12c00b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.632+0000 7f12b67fc640 1 --2- 172.21.15.16:0/533206360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12c0071e50 0x7f12c00b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.632+0000 7f12b67fc640 1 -- 172.21.15.16:0/533206360 >> v1:172.21.15.112:6789/0 conn(0x7f12c0140a10 legacy=0x7f12c00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.632+0000 7f12b67fc640 1 -- 172.21.15.16:0/533206360 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c0071090 con 0x7f12c00ce3e0 2023-12-08T08:39:05.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.632+0000 7f12bf7fe640 1 --2- 172.21.15.16:0/533206360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12c0071e50 0x7f12c00b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.633+0000 7f12b67fc640 1 --2- 172.21.15.16:0/533206360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c00ce3e0 0x7f12c00b1f30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009870 tx=0x7f12a802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326afad45147f5f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.633+0000 7f12be7fc640 1 -- 172.21.15.16:0/533206360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8030840 con 0x7f12c00ce3e0 2023-12-08T08:39:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.633+0000 7f12be7fc640 1 -- 172.21.15.16:0/533206360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a80309a0 con 0x7f12c00ce3e0 2023-12-08T08:39:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.633+0000 7f12be7fc640 1 -- 172.21.15.16:0/533206360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8030d30 con 0x7f12c00ce3e0 2023-12-08T08:39:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.634+0000 7f12c5583640 1 -- 172.21.15.16:0/533206360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c00ce3e0 msgr2=0x7f12c00b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.634+0000 7f12c5583640 1 --2- 172.21.15.16:0/533206360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c00ce3e0 0x7f12c00b1f30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009870 tx=0x7f12a802b980 comp rx=0 tx=0).stop 2023-12-08T08:39:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.634+0000 7f12c5583640 1 -- 172.21.15.16:0/533206360 shutdown_connections 2023-12-08T08:39:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.634+0000 7f12c5583640 1 --2- 172.21.15.16:0/533206360 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12c0071e50 0x7f12c00b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.634+0000 7f12c5583640 1 --2- 172.21.15.16:0/533206360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c00ce3e0 0x7f12c00b1f30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.634+0000 7f12c5583640 1 -- 172.21.15.16:0/533206360 >> 172.21.15.16:0/533206360 conn(0x7f12c00b09f0 msgr2=0x7f12c00b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.634+0000 7f12c5583640 1 -- 172.21.15.16:0/533206360 shutdown_connections 2023-12-08T08:39:05.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.634+0000 7f12c5583640 1 -- 172.21.15.16:0/533206360 wait complete. 2023-12-08T08:39:05.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.635+0000 7f12c5583640 1 Processor -- start 2023-12-08T08:39:05.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.635+0000 7f12c5583640 1 -- start start 2023-12-08T08:39:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12c5583640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c0071e50 0x7f12c013d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12c5583640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c00ce3e0 0x7f12c009bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12c5583640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12c0140a10 0x7f12c009c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12c5583640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c01f0b80 con 0x7f12c0071e50 2023-12-08T08:39:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12c5583640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c01f0e60 con 0x7f12c00ce3e0 2023-12-08T08:39:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12c5583640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c01f1140 con 0x7f12c0140a10 2023-12-08T08:39:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12beffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c0071e50 0x7f12c013d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12beffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c0071e50 0x7f12c013d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54106/0 (socket says 172.21.15.16:54106) 2023-12-08T08:39:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12beffd640 1 -- 172.21.15.16:0/1836165815 learned_addr learned my addr 172.21.15.16:0/1836165815 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12b67fc640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c00ce3e0 0x7f12c009bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12beffd640 1 -- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12c0140a10 msgr2=0x7f12c009c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12bf7fe640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12c0140a10 0x7f12c009c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12beffd640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12c0140a10 0x7f12c009c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12beffd640 1 -- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c00ce3e0 msgr2=0x7f12c009bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12beffd640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c00ce3e0 0x7f12c009bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12beffd640 1 -- 172.21.15.16:0/1836165815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a80094d0 con 0x7f12c0071e50 2023-12-08T08:39:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12b67fc640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c00ce3e0 0x7f12c009bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.636+0000 7f12beffd640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c0071e50 0x7f12c013d350 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f12b000b3f0 tx=0x7f12b000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.637+0000 7f12b7fff640 1 -- 172.21.15.16:0/1836165815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0011830 con 0x7f12c0071e50 2023-12-08T08:39:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.637+0000 7f12b7fff640 1 -- 172.21.15.16:0/1836165815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b0011990 con 0x7f12c0071e50 2023-12-08T08:39:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.637+0000 7f12c5583640 1 -- 172.21.15.16:0/1836165815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c01f1420 con 0x7f12c0071e50 2023-12-08T08:39:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.637+0000 7f12b7fff640 1 -- 172.21.15.16:0/1836165815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0011d40 con 0x7f12c0071e50 2023-12-08T08:39:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.637+0000 7f12c5583640 1 -- 172.21.15.16:0/1836165815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c01f1bf0 con 0x7f12c0071e50 2023-12-08T08:39:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.638+0000 7f12c5583640 1 -- 172.21.15.16:0/1836165815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c00ae2f0 con 0x7f12c0071e50 2023-12-08T08:39:05.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.638+0000 7f12b7fff640 1 -- 172.21.15.16:0/1836165815 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f12b001a400 con 0x7f12c0071e50 2023-12-08T08:39:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.638+0000 7f12b7fff640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f12a006ae40 0x7f12a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.639+0000 7f12b7fff640 1 -- 172.21.15.16:0/1836165815 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f12b00898a0 con 0x7f12c0071e50 2023-12-08T08:39:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.639+0000 7f12b67fc640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f12a006ae40 0x7f12a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.639+0000 7f12b67fc640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f12a006ae40 0x7f12a006d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f12a8036000 tx=0x7f12a803b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:05.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.643+0000 7f12b7fff640 1 -- 172.21.15.16:0/1836165815 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12b0054570 con 0x7f12c0071e50 2023-12-08T08:39:05.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.818+0000 7f12c5583640 1 -- 172.21.15.16:0/1836165815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12c0067430 con 0x7f12c0071e50 2023-12-08T08:39:05.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.819+0000 7f12b7fff640 1 -- 172.21.15.16:0/1836165815 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7f12b0057b90 con 0x7f12c0071e50 2023-12-08T08:39:05.823 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:05.823 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:05.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.821+0000 7f12c5583640 1 -- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f12a006ae40 msgr2=0x7f12a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.821+0000 7f12c5583640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f12a006ae40 0x7f12a006d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f12a8036000 tx=0x7f12a803b040 comp rx=0 tx=0).stop 2023-12-08T08:39:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.821+0000 7f12c5583640 1 -- 172.21.15.16:0/1836165815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c0071e50 msgr2=0x7f12c013d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.821+0000 7f12c5583640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c0071e50 0x7f12c013d350 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f12b000b3f0 tx=0x7f12b000b8b0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.821+0000 7f12c5583640 1 -- 172.21.15.16:0/1836165815 shutdown_connections 2023-12-08T08:39:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.821+0000 7f12c5583640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f12a006ae40 0x7f12a006d300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.821+0000 7f12c5583640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f12c0140a10 0x7f12c009c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.821+0000 7f12c5583640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f12c00ce3e0 0x7f12c009bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.821+0000 7f12c5583640 1 --2- 172.21.15.16:0/1836165815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c0071e50 0x7f12c013d350 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.821+0000 7f12c5583640 1 -- 172.21.15.16:0/1836165815 >> 172.21.15.16:0/1836165815 conn(0x7f12c00b09f0 msgr2=0x7f12c00cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.822+0000 7f12c5583640 1 -- 172.21.15.16:0/1836165815 shutdown_connections 2023-12-08T08:39:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.822+0000 7f12c5583640 1 -- 172.21.15.16:0/1836165815 wait complete. 2023-12-08T08:39:05.836 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:39:05.836 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:05.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.945+0000 7f1bfb872640 1 Processor -- start 2023-12-08T08:39:05.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.945+0000 7f1bfb872640 1 -- start start 2023-12-08T08:39:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bfb872640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bf40cfa30 0x7f1bf40b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bfb872640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1bf40b4150 0x7f1bf40b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bfb872640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1bf40cfe10 0x7f1bf4161260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bfb872640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bf40c6be0 con 0x7f1bf40cfe10 2023-12-08T08:39:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bfb872640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bf40d3ff0 con 0x7f1bf40b4150 2023-12-08T08:39:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bfb872640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf40b4860 con 0x7f1bf40cfa30 2023-12-08T08:39:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bf9e0f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1bf40cfe10 0x7f1bf4161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bf9e0f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1bf40cfe10 0x7f1bf4161260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54118/0 (socket says 172.21.15.16:54118) 2023-12-08T08:39:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bf9e0f640 1 -- 172.21.15.16:0/392931569 learned_addr learned my addr 172.21.15.16:0/392931569 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bf960e640 1 --2- 172.21.15.16:0/392931569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bf40cfa30 0x7f1bf40b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bf8e0d640 1 --2- 172.21.15.16:0/392931569 >> v2:172.21.15.112:3300/0 conn(0x7f1bf40b4150 0x7f1bf40b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bf9e0f640 1 -- 172.21.15.16:0/392931569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bf40cfa30 msgr2=0x7f1bf40b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bf9e0f640 1 --2- 172.21.15.16:0/392931569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bf40cfa30 0x7f1bf40b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bf9e0f640 1 -- 172.21.15.16:0/392931569 >> v2:172.21.15.112:3300/0 conn(0x7f1bf40b4150 msgr2=0x7f1bf40b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bf9e0f640 1 --2- 172.21.15.16:0/392931569 >> v2:172.21.15.112:3300/0 conn(0x7f1bf40b4150 0x7f1bf40b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bf9e0f640 1 -- 172.21.15.16:0/392931569 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf40d3d50 con 0x7f1bf40cfe10 2023-12-08T08:39:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.946+0000 7f1bf8e0d640 1 --2- 172.21.15.16:0/392931569 >> v2:172.21.15.112:3300/0 conn(0x7f1bf40b4150 0x7f1bf40b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.947+0000 7f1bf9e0f640 1 --2- 172.21.15.16:0/392931569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf40cfe10 0x7f1bf4161260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1bf408b2b0 tx=0x7f1bec02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38efa5a1970842e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.947+0000 7f1bebfff640 1 -- 172.21.15.16:0/392931569 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec02c9e0 con 0x7f1bf40cfe10 2023-12-08T08:39:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.947+0000 7f1bebfff640 1 -- 172.21.15.16:0/392931569 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bec02cb40 con 0x7f1bf40cfe10 2023-12-08T08:39:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.947+0000 7f1bebfff640 1 -- 172.21.15.16:0/392931569 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec0343f0 con 0x7f1bf40cfe10 2023-12-08T08:39:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.948+0000 7f1bfb872640 1 -- 172.21.15.16:0/392931569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf40cfe10 msgr2=0x7f1bf4161260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.948+0000 7f1bfb872640 1 --2- 172.21.15.16:0/392931569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf40cfe10 0x7f1bf4161260 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1bf408b2b0 tx=0x7f1bec02be40 comp rx=0 tx=0).stop 2023-12-08T08:39:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.948+0000 7f1bfb872640 1 -- 172.21.15.16:0/392931569 shutdown_connections 2023-12-08T08:39:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.948+0000 7f1bfb872640 1 --2- 172.21.15.16:0/392931569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf40cfe10 0x7f1bf4161260 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.948+0000 7f1bfb872640 1 --2- 172.21.15.16:0/392931569 >> v2:172.21.15.112:3300/0 conn(0x7f1bf40b4150 0x7f1bf40b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.948+0000 7f1bfb872640 1 --2- 172.21.15.16:0/392931569 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bf40cfa30 0x7f1bf40b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.948+0000 7f1bfb872640 1 -- 172.21.15.16:0/392931569 >> 172.21.15.16:0/392931569 conn(0x7f1bf40d3090 msgr2=0x7f1bf40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:05.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.948+0000 7f1bfb872640 1 -- 172.21.15.16:0/392931569 shutdown_connections 2023-12-08T08:39:05.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.948+0000 7f1bfb872640 1 -- 172.21.15.16:0/392931569 wait complete. 2023-12-08T08:39:05.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.949+0000 7f1bfb872640 1 Processor -- start 2023-12-08T08:39:05.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.949+0000 7f1bfb872640 1 -- start start 2023-12-08T08:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bfb872640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bf40b4150 0x7f1bf406bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bfb872640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bf40cfa30 0x7f1bf406c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bfb872640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf40cfe10 0x7f1bf41f6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bfb872640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf415ae50 con 0x7f1bf40cfe10 2023-12-08T08:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bfb872640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf41f7280 con 0x7f1bf40b4150 2023-12-08T08:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bfb872640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bf40afcf0 con 0x7f1bf40cfa30 2023-12-08T08:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bf9e0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf40cfe10 0x7f1bf41f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bf9e0f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf40cfe10 0x7f1bf41f6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54126/0 (socket says 172.21.15.16:54126) 2023-12-08T08:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bf9e0f640 1 -- 172.21.15.16:0/2535076060 learned_addr learned my addr 172.21.15.16:0/2535076060 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:05.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bf8e0d640 1 --2- 172.21.15.16:0/2535076060 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bf40cfa30 0x7f1bf406c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bf960e640 1 --2- 172.21.15.16:0/2535076060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bf40b4150 0x7f1bf406bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bf9e0f640 1 -- 172.21.15.16:0/2535076060 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bf40cfa30 msgr2=0x7f1bf406c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bf9e0f640 1 --2- 172.21.15.16:0/2535076060 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bf40cfa30 0x7f1bf406c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bf9e0f640 1 -- 172.21.15.16:0/2535076060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bf40b4150 msgr2=0x7f1bf406bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bf9e0f640 1 --2- 172.21.15.16:0/2535076060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bf40b4150 0x7f1bf406bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bf9e0f640 1 -- 172.21.15.16:0/2535076060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec009b30 con 0x7f1bf40cfe10 2023-12-08T08:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.950+0000 7f1bf9e0f640 1 --2- 172.21.15.16:0/2535076060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf40cfe10 0x7f1bf41f6500 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1bec036040 tx=0x7f1bec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.951+0000 7f1be9ffb640 1 -- 172.21.15.16:0/2535076060 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec009510 con 0x7f1bf40cfe10 2023-12-08T08:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.951+0000 7f1be9ffb640 1 -- 172.21.15.16:0/2535076060 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bec03e450 con 0x7f1bf40cfe10 2023-12-08T08:39:05.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.951+0000 7f1be9ffb640 1 -- 172.21.15.16:0/2535076060 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bec03e800 con 0x7f1bf40cfe10 2023-12-08T08:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.951+0000 7f1bfb872640 1 -- 172.21.15.16:0/2535076060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf40affd0 con 0x7f1bf40cfe10 2023-12-08T08:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.951+0000 7f1bfb872640 1 -- 172.21.15.16:0/2535076060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf40b0590 con 0x7f1bf40cfe10 2023-12-08T08:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.952+0000 7f1bfb872640 1 -- 172.21.15.16:0/2535076060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bf4067430 con 0x7f1bf40cfe10 2023-12-08T08:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.952+0000 7f1be9ffb640 1 -- 172.21.15.16:0/2535076060 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1bec03e960 con 0x7f1bf40cfe10 2023-12-08T08:39:05.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.953+0000 7f1be9ffb640 1 --2- 172.21.15.16:0/2535076060 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1bd806ae40 0x7f1bd806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.953+0000 7f1be9ffb640 1 -- 172.21.15.16:0/2535076060 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f1bec0ac700 con 0x7f1bf40cfe10 2023-12-08T08:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.953+0000 7f1bf960e640 1 --2- 172.21.15.16:0/2535076060 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1bd806ae40 0x7f1bd806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:05.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.953+0000 7f1bf960e640 1 --2- 172.21.15.16:0/2535076060 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1bd806ae40 0x7f1bd806d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009b40 tx=0x7f1bf0009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:05.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:05.957+0000 7f1be9ffb640 1 -- 172.21.15.16:0/2535076060 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1bec077360 con 0x7f1bf40cfe10 2023-12-08T08:39:06.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.132+0000 7f1bfb872640 1 -- 172.21.15.16:0/2535076060 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1bf40c9940 con 0x7f1bf40cfe10 2023-12-08T08:39:06.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.132+0000 7f1be9ffb640 1 -- 172.21.15.16:0/2535076060 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1bec07a980 con 0x7f1bf40cfe10 2023-12-08T08:39:06.136 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.134+0000 7f1bfb872640 1 -- 172.21.15.16:0/2535076060 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1bd806ae40 msgr2=0x7f1bd806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.134+0000 7f1bfb872640 1 --2- 172.21.15.16:0/2535076060 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1bd806ae40 0x7f1bd806d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0009b40 tx=0x7f1bf0009190 comp rx=0 tx=0).stop 2023-12-08T08:39:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.134+0000 7f1bfb872640 1 -- 172.21.15.16:0/2535076060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf40cfe10 msgr2=0x7f1bf41f6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.134+0000 7f1bfb872640 1 --2- 172.21.15.16:0/2535076060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf40cfe10 0x7f1bf41f6500 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1bec036040 tx=0x7f1bec004a40 comp rx=0 tx=0).stop 2023-12-08T08:39:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.134+0000 7f1bfb872640 1 -- 172.21.15.16:0/2535076060 shutdown_connections 2023-12-08T08:39:06.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.134+0000 7f1bfb872640 1 --2- 172.21.15.16:0/2535076060 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1bd806ae40 0x7f1bd806d300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.134+0000 7f1bfb872640 1 --2- 172.21.15.16:0/2535076060 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1bf40cfe10 0x7f1bf41f6500 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.134+0000 7f1bfb872640 1 --2- 172.21.15.16:0/2535076060 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1bf40cfa30 0x7f1bf406c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.134+0000 7f1bfb872640 1 --2- 172.21.15.16:0/2535076060 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1bf40b4150 0x7f1bf406bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.134+0000 7f1bfb872640 1 -- 172.21.15.16:0/2535076060 >> 172.21.15.16:0/2535076060 conn(0x7f1bf40d3090 msgr2=0x7f1bf40ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.134+0000 7f1bfb872640 1 -- 172.21.15.16:0/2535076060 shutdown_connections 2023-12-08T08:39:06.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.134+0000 7f1bfb872640 1 -- 172.21.15.16:0/2535076060 wait complete. 2023-12-08T08:39:06.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:39:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.255+0000 7f7e3df61640 1 Processor -- start 2023-12-08T08:39:06.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.255+0000 7f7e3df61640 1 -- start start 2023-12-08T08:39:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e3df61640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e380cfa30 0x7f7e380b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e3df61640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e380cfe10 0x7f7e380cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e3df61640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e380c6be0 con 0x7f7e380b4150 2023-12-08T08:39:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e3df61640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e380d3ff0 con 0x7f7e380cfa30 2023-12-08T08:39:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e3df61640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e380b4860 con 0x7f7e380cfe10 2023-12-08T08:39:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e377fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e380cfa30 0x7f7e380b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e36ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7e380b4150 0x7f7e380b3460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38200/0 (socket says 172.21.15.16:38200) 2023-12-08T08:39:06.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e36ffd640 1 -- 172.21.15.16:0/2687723660 learned_addr learned my addr 172.21.15.16:0/2687723660 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e377fe640 1 -- 172.21.15.16:0/2687723660 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e380cfe10 msgr2=0x7f7e380cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e37fff640 1 --2- 172.21.15.16:0/2687723660 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e380cfe10 0x7f7e380cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e377fe640 1 --2- 172.21.15.16:0/2687723660 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e380cfe10 0x7f7e380cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e377fe640 1 -- 172.21.15.16:0/2687723660 >> v1:172.21.15.16:6789/0 conn(0x7f7e380b4150 legacy=0x7f7e380b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e367fc640 1 -- 172.21.15.16:0/2687723660 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698924989 0 0) 0x7f7e380c6be0 con 0x7f7e380b4150 2023-12-08T08:39:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e377fe640 1 -- 172.21.15.16:0/2687723660 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e380d3d50 con 0x7f7e380cfa30 2023-12-08T08:39:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.256+0000 7f7e37fff640 1 --2- 172.21.15.16:0/2687723660 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e380cfe10 0x7f7e380cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.257+0000 7f7e377fe640 1 --2- 172.21.15.16:0/2687723660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e380cfa30 0x7f7e380b2f20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7e280097a0 tx=0x7f7e2802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985fc1730bae2c15 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.257+0000 7f7e367fc640 1 -- 172.21.15.16:0/2687723660 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2802f050 con 0x7f7e380cfa30 2023-12-08T08:39:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.257+0000 7f7e367fc640 1 -- 172.21.15.16:0/2687723660 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e28031700 con 0x7f7e380cfa30 2023-12-08T08:39:06.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.257+0000 7f7e367fc640 1 -- 172.21.15.16:0/2687723660 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28031a90 con 0x7f7e380cfa30 2023-12-08T08:39:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.258+0000 7f7e3df61640 1 -- 172.21.15.16:0/2687723660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e380cfa30 msgr2=0x7f7e380b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.258+0000 7f7e3df61640 1 --2- 172.21.15.16:0/2687723660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e380cfa30 0x7f7e380b2f20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7e280097a0 tx=0x7f7e2802b7a0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.258+0000 7f7e3df61640 1 -- 172.21.15.16:0/2687723660 shutdown_connections 2023-12-08T08:39:06.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.258+0000 7f7e3df61640 1 --2- 172.21.15.16:0/2687723660 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e380cfe10 0x7f7e380cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.258+0000 7f7e3df61640 1 --2- 172.21.15.16:0/2687723660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e380cfa30 0x7f7e380b2f20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.258+0000 7f7e3df61640 1 -- 172.21.15.16:0/2687723660 >> 172.21.15.16:0/2687723660 conn(0x7f7e380d3090 msgr2=0x7f7e380d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.258+0000 7f7e3df61640 1 -- 172.21.15.16:0/2687723660 shutdown_connections 2023-12-08T08:39:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.258+0000 7f7e3df61640 1 -- 172.21.15.16:0/2687723660 wait complete. 2023-12-08T08:39:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.259+0000 7f7e3df61640 1 Processor -- start 2023-12-08T08:39:06.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.259+0000 7f7e3df61640 1 -- start start 2023-12-08T08:39:06.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e3df61640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e380b4150 0x7f7e3814b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e3df61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e380cfa30 0x7f7e380b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e3df61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e380cfe10 0x7f7e380cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e3df61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3814ba60 con 0x7f7e380cfe10 2023-12-08T08:39:06.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e3df61640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e380cafe0 con 0x7f7e380cfa30 2023-12-08T08:39:06.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e3df61640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e380cb2c0 con 0x7f7e380b4150 2023-12-08T08:39:06.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e380cfe10 0x7f7e380cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e37fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e380cfe10 0x7f7e380cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54156/0 (socket says 172.21.15.16:54156) 2023-12-08T08:39:06.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e37fff640 1 -- 172.21.15.16:0/1613692457 learned_addr learned my addr 172.21.15.16:0/1613692457 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e377fe640 1 --2- 172.21.15.16:0/1613692457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e380b4150 0x7f7e3814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e36ffd640 1 --2- 172.21.15.16:0/1613692457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e380cfa30 0x7f7e380b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e37fff640 1 -- 172.21.15.16:0/1613692457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e380b4150 msgr2=0x7f7e3814b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e37fff640 1 --2- 172.21.15.16:0/1613692457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e380b4150 0x7f7e3814b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e37fff640 1 -- 172.21.15.16:0/1613692457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e380cfa30 msgr2=0x7f7e380b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e37fff640 1 --2- 172.21.15.16:0/1613692457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e380cfa30 0x7f7e380b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e37fff640 1 -- 172.21.15.16:0/1613692457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e28009400 con 0x7f7e380cfe10 2023-12-08T08:39:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.260+0000 7f7e37fff640 1 --2- 172.21.15.16:0/1613692457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e380cfe10 0x7f7e380cdf50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000cd50 tx=0x7f7e3000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.261+0000 7f7e1ffff640 1 -- 172.21.15.16:0/1613692457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3000f680 con 0x7f7e380cfe10 2023-12-08T08:39:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.261+0000 7f7e1ffff640 1 -- 172.21.15.16:0/1613692457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3000f800 con 0x7f7e380cfe10 2023-12-08T08:39:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.261+0000 7f7e3df61640 1 -- 172.21.15.16:0/1613692457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e380cb5a0 con 0x7f7e380cfe10 2023-12-08T08:39:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.261+0000 7f7e1ffff640 1 -- 172.21.15.16:0/1613692457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3000fbb0 con 0x7f7e380cfe10 2023-12-08T08:39:06.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.261+0000 7f7e3df61640 1 -- 172.21.15.16:0/1613692457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e380cbb60 con 0x7f7e380cfe10 2023-12-08T08:39:06.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.262+0000 7f7e3df61640 1 -- 172.21.15.16:0/1613692457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e380cbd50 con 0x7f7e380cfe10 2023-12-08T08:39:06.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.262+0000 7f7e1ffff640 1 -- 172.21.15.16:0/1613692457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e3000b040 con 0x7f7e380cfe10 2023-12-08T08:39:06.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.262+0000 7f7e1ffff640 1 --2- 172.21.15.16:0/1613692457 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7e2406ae40 0x7f7e2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.262+0000 7f7e1ffff640 1 -- 172.21.15.16:0/1613692457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f7e3008a510 con 0x7f7e380cfe10 2023-12-08T08:39:06.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.263+0000 7f7e377fe640 1 --2- 172.21.15.16:0/1613692457 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7e2406ae40 0x7f7e2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.263+0000 7f7e377fe640 1 --2- 172.21.15.16:0/1613692457 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7e2406ae40 0x7f7e2406d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7e28035000 tx=0x7f7e28008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:06.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.267+0000 7f7e1ffff640 1 -- 172.21.15.16:0/1613692457 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e30055160 con 0x7f7e380cfe10 2023-12-08T08:39:06.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.442+0000 7f7e3df61640 1 -- 172.21.15.16:0/1613692457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e38067430 con 0x7f7e380cfe10 2023-12-08T08:39:06.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.442+0000 7f7e1ffff640 1 -- 172.21.15.16:0/1613692457 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7e30058780 con 0x7f7e380cfe10 2023-12-08T08:39:06.446 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:06.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.444+0000 7f7e3df61640 1 -- 172.21.15.16:0/1613692457 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7e2406ae40 msgr2=0x7f7e2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.444+0000 7f7e3df61640 1 --2- 172.21.15.16:0/1613692457 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7e2406ae40 0x7f7e2406d300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7e28035000 tx=0x7f7e28008030 comp rx=0 tx=0).stop 2023-12-08T08:39:06.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.444+0000 7f7e3df61640 1 -- 172.21.15.16:0/1613692457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e380cfe10 msgr2=0x7f7e380cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.444+0000 7f7e3df61640 1 --2- 172.21.15.16:0/1613692457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e380cfe10 0x7f7e380cdf50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000cd50 tx=0x7f7e3000a650 comp rx=0 tx=0).stop 2023-12-08T08:39:06.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.444+0000 7f7e3df61640 1 -- 172.21.15.16:0/1613692457 shutdown_connections 2023-12-08T08:39:06.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.444+0000 7f7e3df61640 1 --2- 172.21.15.16:0/1613692457 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7e2406ae40 0x7f7e2406d300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.445+0000 7f7e3df61640 1 --2- 172.21.15.16:0/1613692457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e380cfe10 0x7f7e380cdf50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.445+0000 7f7e3df61640 1 --2- 172.21.15.16:0/1613692457 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e380cfa30 0x7f7e380b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.445+0000 7f7e3df61640 1 --2- 172.21.15.16:0/1613692457 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e380b4150 0x7f7e3814b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.445+0000 7f7e3df61640 1 -- 172.21.15.16:0/1613692457 >> 172.21.15.16:0/1613692457 conn(0x7f7e380d3090 msgr2=0x7f7e3806b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:06.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.445+0000 7f7e3df61640 1 -- 172.21.15.16:0/1613692457 shutdown_connections 2023-12-08T08:39:06.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.445+0000 7f7e3df61640 1 -- 172.21.15.16:0/1613692457 wait complete. 2023-12-08T08:39:06.459 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:39:06.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.569+0000 7f2f9c9b0640 1 Processor -- start 2023-12-08T08:39:06.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.569+0000 7f2f9c9b0640 1 -- start start 2023-12-08T08:39:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.570+0000 7f2f9c9b0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2f980b4150 0x7f2f980b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.570+0000 7f2f9c9b0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f980cfe10 0x7f2f980cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.570+0000 7f2f9c9b0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f980c6be0 con 0x7f2f980cfa30 2023-12-08T08:39:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.570+0000 7f2f9c9b0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f980d3ff0 con 0x7f2f980cfe10 2023-12-08T08:39:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.570+0000 7f2f9c9b0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f980b4860 con 0x7f2f980b4150 2023-12-08T08:39:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.570+0000 7f2f96d9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f980cfe10 0x7f2f980cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.570+0000 7f2f9659c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2f980cfa30 0x7f2f980b2f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50610/0 (socket says 172.21.15.16:50610) 2023-12-08T08:39:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.570+0000 7f2f9659c640 1 -- 172.21.15.16:0/3684910579 learned_addr learned my addr 172.21.15.16:0/3684910579 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.571+0000 7f2f95d9b640 1 --2- 172.21.15.16:0/3684910579 >> v2:172.21.15.112:3300/0 conn(0x7f2f980b4150 0x7f2f980b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.571+0000 7f2f96d9d640 1 -- 172.21.15.16:0/3684910579 >> v2:172.21.15.112:3300/0 conn(0x7f2f980b4150 msgr2=0x7f2f980b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:06.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.571+0000 7f2f96d9d640 1 --2- 172.21.15.16:0/3684910579 >> v2:172.21.15.112:3300/0 conn(0x7f2f980b4150 0x7f2f980b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.571+0000 7f2f96d9d640 1 -- 172.21.15.16:0/3684910579 >> v1:172.21.15.16:6789/0 conn(0x7f2f980cfa30 legacy=0x7f2f980b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.571+0000 7f2f96d9d640 1 -- 172.21.15.16:0/3684910579 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980d3d50 con 0x7f2f980cfe10 2023-12-08T08:39:06.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.572+0000 7f2f96d9d640 1 --2- 172.21.15.16:0/3684910579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f980cfe10 0x7f2f980cdf50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2f98162300 tx=0x7f2f8802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab128f4e72b554dc server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:06.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.572+0000 7f2f9559a640 1 -- 172.21.15.16:0/3684910579 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8802cbf0 con 0x7f2f980cfe10 2023-12-08T08:39:06.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.572+0000 7f2f9559a640 1 -- 172.21.15.16:0/3684910579 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8802cd50 con 0x7f2f980cfe10 2023-12-08T08:39:06.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.572+0000 7f2f9559a640 1 -- 172.21.15.16:0/3684910579 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88034660 con 0x7f2f980cfe10 2023-12-08T08:39:06.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.572+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/3684910579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f980cfe10 msgr2=0x7f2f980cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.572+0000 7f2f9c9b0640 1 --2- 172.21.15.16:0/3684910579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f980cfe10 0x7f2f980cdf50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2f98162300 tx=0x7f2f8802be40 comp rx=0 tx=0).stop 2023-12-08T08:39:06.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.572+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/3684910579 shutdown_connections 2023-12-08T08:39:06.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.572+0000 7f2f9c9b0640 1 --2- 172.21.15.16:0/3684910579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f980cfe10 0x7f2f980cdf50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.572+0000 7f2f9c9b0640 1 --2- 172.21.15.16:0/3684910579 >> v2:172.21.15.112:3300/0 conn(0x7f2f980b4150 0x7f2f980b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.572+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/3684910579 >> 172.21.15.16:0/3684910579 conn(0x7f2f980d3090 msgr2=0x7f2f980d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:06.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.572+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/3684910579 shutdown_connections 2023-12-08T08:39:06.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.572+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/3684910579 wait complete. 2023-12-08T08:39:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.574+0000 7f2f9c9b0640 1 Processor -- start 2023-12-08T08:39:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.574+0000 7f2f9c9b0640 1 -- start start 2023-12-08T08:39:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.574+0000 7f2f9c9b0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f980b4150 0x7f2f98153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.574+0000 7f2f9c9b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f980cfa30 0x7f2f9815f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.574+0000 7f2f9c9b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f980cfe10 0x7f2f9815fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.574+0000 7f2f9c9b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980cd2c0 con 0x7f2f980cfa30 2023-12-08T08:39:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.574+0000 7f2f9c9b0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980cba30 con 0x7f2f980cfe10 2023-12-08T08:39:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.574+0000 7f2f9c9b0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980cb750 con 0x7f2f980b4150 2023-12-08T08:39:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f95d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f980cfa30 0x7f2f9815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f95d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f980cfa30 0x7f2f9815f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34916/0 (socket says 172.21.15.16:34916) 2023-12-08T08:39:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f95d9b640 1 -- 172.21.15.16:0/462774761 learned_addr learned my addr 172.21.15.16:0/462774761 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f96d9d640 1 --2- 172.21.15.16:0/462774761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f980cfe10 0x7f2f9815fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f9659c640 1 --2- 172.21.15.16:0/462774761 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f980b4150 0x7f2f98153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f95d9b640 1 -- 172.21.15.16:0/462774761 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f980b4150 msgr2=0x7f2f98153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f95d9b640 1 --2- 172.21.15.16:0/462774761 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f980b4150 0x7f2f98153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f95d9b640 1 -- 172.21.15.16:0/462774761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f980cfe10 msgr2=0x7f2f9815fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f95d9b640 1 --2- 172.21.15.16:0/462774761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f980cfe10 0x7f2f9815fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f95d9b640 1 -- 172.21.15.16:0/462774761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f88009a70 con 0x7f2f980cfa30 2023-12-08T08:39:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f95d9b640 1 --2- 172.21.15.16:0/462774761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f980cfa30 0x7f2f9815f780 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000c7f0 tx=0x7f2f8000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f8effd640 1 -- 172.21.15.16:0/462774761 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f80004220 con 0x7f2f980cfa30 2023-12-08T08:39:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/462774761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f980b7390 con 0x7f2f980cfa30 2023-12-08T08:39:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f8effd640 1 -- 172.21.15.16:0/462774761 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f80004380 con 0x7f2f980cfa30 2023-12-08T08:39:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f8effd640 1 -- 172.21.15.16:0/462774761 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f80007840 con 0x7f2f980cfa30 2023-12-08T08:39:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.575+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/462774761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f980b7950 con 0x7f2f980cfa30 2023-12-08T08:39:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.576+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/462774761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f980b71f0 con 0x7f2f980cfa30 2023-12-08T08:39:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.577+0000 7f2f8effd640 1 -- 172.21.15.16:0/462774761 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f80007bb0 con 0x7f2f980cfa30 2023-12-08T08:39:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.577+0000 7f2f8effd640 1 --2- 172.21.15.16:0/462774761 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f7806ae40 0x7f2f7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.577+0000 7f2f8effd640 1 -- 172.21.15.16:0/462774761 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f2f800899c0 con 0x7f2f980cfa30 2023-12-08T08:39:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.577+0000 7f2f9659c640 1 --2- 172.21.15.16:0/462774761 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f7806ae40 0x7f2f7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.578+0000 7f2f9659c640 1 --2- 172.21.15.16:0/462774761 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f7806ae40 0x7f2f7806d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2f90005880 tx=0x7f2f90009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.582+0000 7f2f8effd640 1 -- 172.21.15.16:0/462774761 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f980b71f0 con 0x7f2f980cfa30 2023-12-08T08:39:06.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.760+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/462774761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f980b71f0 con 0x7f2f980cfa30 2023-12-08T08:39:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.760+0000 7f2f8effd640 1 -- 172.21.15.16:0/462774761 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f980b71f0 con 0x7f2f980cfa30 2023-12-08T08:39:06.764 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:39:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.762+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/462774761 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f7806ae40 msgr2=0x7f2f7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.762+0000 7f2f9c9b0640 1 --2- 172.21.15.16:0/462774761 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f7806ae40 0x7f2f7806d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2f90005880 tx=0x7f2f90009040 comp rx=0 tx=0).stop 2023-12-08T08:39:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.762+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/462774761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f980cfa30 msgr2=0x7f2f9815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.762+0000 7f2f9c9b0640 1 --2- 172.21.15.16:0/462774761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f980cfa30 0x7f2f9815f780 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000c7f0 tx=0x7f2f8000ccb0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.762+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/462774761 shutdown_connections 2023-12-08T08:39:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.762+0000 7f2f9c9b0640 1 --2- 172.21.15.16:0/462774761 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f7806ae40 0x7f2f7806d300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.762+0000 7f2f9c9b0640 1 --2- 172.21.15.16:0/462774761 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f980cfe10 0x7f2f9815fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.762+0000 7f2f9c9b0640 1 --2- 172.21.15.16:0/462774761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f980cfa30 0x7f2f9815f780 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.762+0000 7f2f9c9b0640 1 --2- 172.21.15.16:0/462774761 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f980b4150 0x7f2f98153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.762+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/462774761 >> 172.21.15.16:0/462774761 conn(0x7f2f980d3090 msgr2=0x7f2f980c8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.762+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/462774761 shutdown_connections 2023-12-08T08:39:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.762+0000 7f2f9c9b0640 1 -- 172.21.15.16:0/462774761 wait complete. 2023-12-08T08:39:06.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:39:06.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.883+0000 7f0e93e3d640 1 Processor -- start 2023-12-08T08:39:06.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.883+0000 7f0e93e3d640 1 -- start start 2023-12-08T08:39:06.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.884+0000 7f0e93e3d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e8c0b0c20 0x7f0e8c166150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.884+0000 7f0e93e3d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8c0ce360 con 0x7f0e8c15e730 2023-12-08T08:39:06.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.884+0000 7f0e93e3d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8c0ce640 con 0x7f0e8c0cd1c0 2023-12-08T08:39:06.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.884+0000 7f0e93e3d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0cbda0 con 0x7f0e8c0b0c20 2023-12-08T08:39:06.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.884+0000 7f0e91bd9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0e8c15e730 0x7f0e8c0cc5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50618/0 (socket says 172.21.15.16:50618) 2023-12-08T08:39:06.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.884+0000 7f0e91bd9640 1 -- 172.21.15.16:0/3530735804 learned_addr learned my addr 172.21.15.16:0/3530735804 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.884+0000 7f0e90bd7640 1 -- 172.21.15.16:0/3530735804 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886656372 0 0) 0x7f0e8c0ce360 con 0x7f0e8c15e730 2023-12-08T08:39:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.884+0000 7f0e90bd7640 1 -- 172.21.15.16:0/3530735804 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e74003520 con 0x7f0e8c15e730 2023-12-08T08:39:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.884+0000 7f0e923da640 1 --2- 172.21.15.16:0/3530735804 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e8c0b0c20 0x7f0e8c166150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e90bd7640 1 -- 172.21.15.16:0/3530735804 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 447255238 0 0) 0x7f0e74003520 con 0x7f0e8c15e730 2023-12-08T08:39:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e90bd7640 1 -- 172.21.15.16:0/3530735804 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e8c0b0c20 msgr2=0x7f0e8c166150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e90bd7640 1 --2- 172.21.15.16:0/3530735804 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e8c0b0c20 0x7f0e8c166150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e90bd7640 1 -- 172.21.15.16:0/3530735804 >> v1:172.21.15.112:6789/0 conn(0x7f0e8c0cd1c0 legacy=0x7f0e8c0b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e90bd7640 1 -- 172.21.15.16:0/3530735804 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c0cc080 con 0x7f0e8c15e730 2023-12-08T08:39:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e90bd7640 1 -- 172.21.15.16:0/3530735804 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f0e84001670 con 0x7f0e8c15e730 2023-12-08T08:39:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e90bd7640 1 -- 172.21.15.16:0/3530735804 >> v1:172.21.15.16:6789/0 conn(0x7f0e8c15e730 legacy=0x7f0e8c0cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e90bd7640 1 --2- 172.21.15.16:0/3530735804 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e8c0b0c20 0x7f0e740039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e90bd7640 1 --2- 172.21.15.16:0/3530735804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e74003f20 0x7f0e740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e90bd7640 1 --2- 172.21.15.16:0/3530735804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74004d00 0x7f0e74005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e90bd7640 1 -- 172.21.15.16:0/3530735804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0cbda0 con 0x7f0e74004d00 2023-12-08T08:39:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e90bd7640 1 -- 172.21.15.16:0/3530735804 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7400ba60 con 0x7f0e74003f20 2023-12-08T08:39:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e90bd7640 1 -- 172.21.15.16:0/3530735804 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7400bc20 con 0x7f0e8c0b0c20 2023-12-08T08:39:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e923da640 1 --2- 172.21.15.16:0/3530735804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74004d00 0x7f0e74005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e91bd9640 1 --2- 172.21.15.16:0/3530735804 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e8c0b0c20 0x7f0e740039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e913d8640 1 --2- 172.21.15.16:0/3530735804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e74003f20 0x7f0e740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e923da640 1 -- 172.21.15.16:0/3530735804 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e8c0b0c20 msgr2=0x7f0e740039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e923da640 1 --2- 172.21.15.16:0/3530735804 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e8c0b0c20 0x7f0e740039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e923da640 1 -- 172.21.15.16:0/3530735804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e74003f20 msgr2=0x7f0e740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e923da640 1 --2- 172.21.15.16:0/3530735804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e74003f20 0x7f0e740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e923da640 1 -- 172.21.15.16:0/3530735804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e74003520 con 0x7f0e74004d00 2023-12-08T08:39:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.885+0000 7f0e913d8640 1 --2- 172.21.15.16:0/3530735804 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e74003f20 0x7f0e740047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.886+0000 7f0e923da640 1 --2- 172.21.15.16:0/3530735804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74004d00 0x7f0e74005180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009f90 tx=0x7f0e740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6ad10b35dac49b server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.886+0000 7f0e90bd7640 1 -- 172.21.15.16:0/3530735804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c02cbd0 con 0x7f0e74004d00 2023-12-08T08:39:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.886+0000 7f0e90bd7640 1 -- 172.21.15.16:0/3530735804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7c00f040 con 0x7f0e74004d00 2023-12-08T08:39:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.886+0000 7f0e90bd7640 1 -- 172.21.15.16:0/3530735804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c034440 con 0x7f0e74004d00 2023-12-08T08:39:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.887+0000 7f0e93e3d640 1 -- 172.21.15.16:0/3530735804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74004d00 msgr2=0x7f0e74005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.887+0000 7f0e93e3d640 1 --2- 172.21.15.16:0/3530735804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e74004d00 0x7f0e74005180 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009f90 tx=0x7f0e740057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.887+0000 7f0e923da640 1 -- 172.21.15.16:0/3530735804 reap_dead start 2023-12-08T08:39:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.887+0000 7f0e93e3d640 1 -- 172.21.15.16:0/3530735804 shutdown_connections 2023-12-08T08:39:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.887+0000 7f0e93e3d640 1 -- 172.21.15.16:0/3530735804 >> 172.21.15.16:0/3530735804 conn(0x7f0e8c0d0e10 msgr2=0x7f0e8c0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.887+0000 7f0e93e3d640 1 -- 172.21.15.16:0/3530735804 shutdown_connections 2023-12-08T08:39:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.887+0000 7f0e93e3d640 1 -- 172.21.15.16:0/3530735804 wait complete. 2023-12-08T08:39:06.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.888+0000 7f0e93e3d640 1 Processor -- start 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.888+0000 7f0e93e3d640 1 -- start start 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e93e3d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e8c1f6a30 0x7f0e8c06bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e93e3d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e8c1f6e10 0x7f0e8c0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e93e3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8c0ca840 0x7f0e8c1f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e93e3d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c06d030 con 0x7f0e8c0ca840 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e93e3d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c0cacc0 con 0x7f0e8c1f6e10 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e93e3d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c06c420 con 0x7f0e8c1f6a30 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e923da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8c0ca840 0x7f0e8c1f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e923da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8c0ca840 0x7f0e8c1f31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34922/0 (socket says 172.21.15.16:34922) 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e923da640 1 -- 172.21.15.16:0/1550734692 learned_addr learned my addr 172.21.15.16:0/1550734692 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e913d8640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e8c1f6e10 0x7f0e8c0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e91bd9640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e8c1f6a30 0x7f0e8c06bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e923da640 1 -- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e8c1f6a30 msgr2=0x7f0e8c06bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e923da640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e8c1f6a30 0x7f0e8c06bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e923da640 1 -- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e8c1f6e10 msgr2=0x7f0e8c0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e923da640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e8c1f6e10 0x7f0e8c0ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e923da640 1 -- 172.21.15.16:0/1550734692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c06c700 con 0x7f0e8c0ca840 2023-12-08T08:39:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.889+0000 7f0e913d8640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e8c1f6e10 0x7f0e8c0ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.890+0000 7f0e923da640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8c0ca840 0x7f0e8c1f31d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009f90 tx=0x7f0e7c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.890+0000 7f0e8a7fc640 1 -- 172.21.15.16:0/1550734692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c003b90 con 0x7f0e8c0ca840 2023-12-08T08:39:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.890+0000 7f0e8a7fc640 1 -- 172.21.15.16:0/1550734692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7c003d10 con 0x7f0e8c0ca840 2023-12-08T08:39:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.890+0000 7f0e93e3d640 1 -- 172.21.15.16:0/1550734692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8c06c9e0 con 0x7f0e8c0ca840 2023-12-08T08:39:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.890+0000 7f0e8a7fc640 1 -- 172.21.15.16:0/1550734692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c034ac0 con 0x7f0e8c0ca840 2023-12-08T08:39:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.890+0000 7f0e93e3d640 1 -- 172.21.15.16:0/1550734692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8c06d4c0 con 0x7f0e8c0ca840 2023-12-08T08:39:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.891+0000 7f0e93e3d640 1 -- 172.21.15.16:0/1550734692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8c06d1a0 con 0x7f0e8c0ca840 2023-12-08T08:39:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.891+0000 7f0e8a7fc640 1 -- 172.21.15.16:0/1550734692 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0e7c034c20 con 0x7f0e8c0ca840 2023-12-08T08:39:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.892+0000 7f0e8a7fc640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0e7806ae40 0x7f0e7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.892+0000 7f0e8a7fc640 1 -- 172.21.15.16:0/1550734692 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f0e7c0ab2b0 con 0x7f0e8c0ca840 2023-12-08T08:39:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.892+0000 7f0e91bd9640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0e7806ae40 0x7f0e7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.892+0000 7f0e91bd9640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0e7806ae40 0x7f0e7806d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0e84004840 tx=0x7f0e84009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:06.896+0000 7f0e8a7fc640 1 -- 172.21.15.16:0/1550734692 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e7c0760c0 con 0x7f0e8c0ca840 2023-12-08T08:39:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.075+0000 7f0e93e3d640 1 -- 172.21.15.16:0/1550734692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0e8c067430 con 0x7f0e8c0ca840 2023-12-08T08:39:07.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.075+0000 7f0e8a7fc640 1 -- 172.21.15.16:0/1550734692 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0e7c0796e0 con 0x7f0e8c0ca840 2023-12-08T08:39:07.078 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:07.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.077+0000 7f0e93e3d640 1 -- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0e7806ae40 msgr2=0x7f0e7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.077+0000 7f0e93e3d640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0e7806ae40 0x7f0e7806d300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0e84004840 tx=0x7f0e84009040 comp rx=0 tx=0).stop 2023-12-08T08:39:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.077+0000 7f0e93e3d640 1 -- 172.21.15.16:0/1550734692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8c0ca840 msgr2=0x7f0e8c1f31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.077+0000 7f0e93e3d640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8c0ca840 0x7f0e8c1f31d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009f90 tx=0x7f0e7c009750 comp rx=0 tx=0).stop 2023-12-08T08:39:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.077+0000 7f0e93e3d640 1 -- 172.21.15.16:0/1550734692 shutdown_connections 2023-12-08T08:39:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.077+0000 7f0e93e3d640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0e7806ae40 0x7f0e7806d300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.077+0000 7f0e93e3d640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e8c0ca840 0x7f0e8c1f31d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.077+0000 7f0e93e3d640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0e8c1f6e10 0x7f0e8c0ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.077+0000 7f0e93e3d640 1 --2- 172.21.15.16:0/1550734692 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0e8c1f6a30 0x7f0e8c06bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.077+0000 7f0e93e3d640 1 -- 172.21.15.16:0/1550734692 >> 172.21.15.16:0/1550734692 conn(0x7f0e8c0d0e10 msgr2=0x7f0e8c06e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.077+0000 7f0e93e3d640 1 -- 172.21.15.16:0/1550734692 shutdown_connections 2023-12-08T08:39:07.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.077+0000 7f0e93e3d640 1 -- 172.21.15.16:0/1550734692 wait complete. 2023-12-08T08:39:07.116 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2023-12-08T08:39:07.117 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:07.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.220+0000 7f0ff93ce640 1 Processor -- start 2023-12-08T08:39:07.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.220+0000 7f0ff93ce640 1 -- start start 2023-12-08T08:39:07.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.221+0000 7f0ff93ce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ff40cffe0 0x7f0ff40d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.221+0000 7f0ff93ce640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff40b28e0 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.221+0000 7f0ff93ce640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff40ce7c0 con 0x7f0ff4160a00 2023-12-08T08:39:07.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.221+0000 7f0ff93ce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40b2610 con 0x7f0ff40cffe0 2023-12-08T08:39:07.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.221+0000 7f0ff37fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0ff40b1ea0 0x7f0ff40b2d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50632/0 (socket says 172.21.15.16:50632) 2023-12-08T08:39:07.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.221+0000 7f0ff37fe640 1 -- 172.21.15.16:0/2559655893 learned_addr learned my addr 172.21.15.16:0/2559655893 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:07.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.222+0000 7f0ff27fc640 1 --2- 172.21.15.16:0/2559655893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ff40cffe0 0x7f0ff40d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:07.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.222+0000 7f0ff1ffb640 1 -- 172.21.15.16:0/2559655893 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454866485 0 0) 0x7f0ff40b28e0 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.222+0000 7f0ff1ffb640 1 -- 172.21.15.16:0/2559655893 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fe0003520 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.222+0000 7f0ff1ffb640 1 -- 172.21.15.16:0/2559655893 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459214272 0 0) 0x7f0ff40ce7c0 con 0x7f0ff4160a00 2023-12-08T08:39:07.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.222+0000 7f0ff1ffb640 1 -- 172.21.15.16:0/2559655893 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff40b28e0 con 0x7f0ff4160a00 2023-12-08T08:39:07.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.222+0000 7f0ff27fc640 1 -- 172.21.15.16:0/2559655893 >> v1:172.21.15.112:6789/0 conn(0x7f0ff4160a00 legacy=0x7f0ff40cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.222+0000 7f0ff27fc640 1 -- 172.21.15.16:0/2559655893 >> v1:172.21.15.16:6789/0 conn(0x7f0ff40b1ea0 legacy=0x7f0ff40b2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.222+0000 7f0ff27fc640 1 -- 172.21.15.16:0/2559655893 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff40ceaa0 con 0x7f0ff40cffe0 2023-12-08T08:39:07.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.223+0000 7f0ff27fc640 1 --2- 172.21.15.16:0/2559655893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ff40cffe0 0x7f0ff40d3a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0097d0 tx=0x7f0fdc02b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aed422bb1730896 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.223+0000 7f0ff1ffb640 1 -- 172.21.15.16:0/2559655893 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc031890 con 0x7f0ff40cffe0 2023-12-08T08:39:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.223+0000 7f0ff1ffb640 1 -- 172.21.15.16:0/2559655893 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fdc0319f0 con 0x7f0ff40cffe0 2023-12-08T08:39:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.223+0000 7f0ff1ffb640 1 -- 172.21.15.16:0/2559655893 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc031d60 con 0x7f0ff40cffe0 2023-12-08T08:39:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.223+0000 7f0ff93ce640 1 -- 172.21.15.16:0/2559655893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ff40cffe0 msgr2=0x7f0ff40d3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.223+0000 7f0ff93ce640 1 --2- 172.21.15.16:0/2559655893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ff40cffe0 0x7f0ff40d3a60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc0097d0 tx=0x7f0fdc02b930 comp rx=0 tx=0).stop 2023-12-08T08:39:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.223+0000 7f0ff93ce640 1 -- 172.21.15.16:0/2559655893 shutdown_connections 2023-12-08T08:39:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.223+0000 7f0ff93ce640 1 --2- 172.21.15.16:0/2559655893 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ff40cffe0 0x7f0ff40d3a60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.223+0000 7f0ff93ce640 1 -- 172.21.15.16:0/2559655893 >> 172.21.15.16:0/2559655893 conn(0x7f0ff409c300 msgr2=0x7f0ff40d2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.224+0000 7f0ff93ce640 1 -- 172.21.15.16:0/2559655893 shutdown_connections 2023-12-08T08:39:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.224+0000 7f0ff93ce640 1 -- 172.21.15.16:0/2559655893 wait complete. 2023-12-08T08:39:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.225+0000 7f0ff93ce640 1 Processor -- start 2023-12-08T08:39:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.225+0000 7f0ff93ce640 1 -- start start 2023-12-08T08:39:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff93ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ff40b1ea0 0x7f0ff41f3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff93ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ff40cffe0 0x7f0ff40cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff93ce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ff4160a00 0x7f0ff414f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff93ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40b6b10 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff93ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff41f3a80 con 0x7f0ff40cffe0 2023-12-08T08:39:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff93ce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40b5f00 con 0x7f0ff4160a00 2023-12-08T08:39:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ff40b1ea0 0x7f0ff41f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ff40b1ea0 0x7f0ff41f3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34926/0 (socket says 172.21.15.16:34926) 2023-12-08T08:39:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff2ffd640 1 -- 172.21.15.16:0/3809323633 learned_addr learned my addr 172.21.15.16:0/3809323633 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff37fe640 1 --2- 172.21.15.16:0/3809323633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ff4160a00 0x7f0ff414f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff27fc640 1 --2- 172.21.15.16:0/3809323633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ff40cffe0 0x7f0ff40cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff2ffd640 1 -- 172.21.15.16:0/3809323633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ff4160a00 msgr2=0x7f0ff414f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff2ffd640 1 --2- 172.21.15.16:0/3809323633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ff4160a00 0x7f0ff414f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff2ffd640 1 -- 172.21.15.16:0/3809323633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ff40cffe0 msgr2=0x7f0ff40cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff2ffd640 1 --2- 172.21.15.16:0/3809323633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ff40cffe0 0x7f0ff40cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff2ffd640 1 -- 172.21.15.16:0/3809323633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc009400 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff2ffd640 1 --2- 172.21.15.16:0/3809323633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ff40b1ea0 0x7f0ff41f3540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00c460 tx=0x7f0fec00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0feb7fe640 1 -- 172.21.15.16:0/3809323633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0052d0 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0feb7fe640 1 -- 172.21.15.16:0/3809323633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fec005450 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0feb7fe640 1 -- 172.21.15.16:0/3809323633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec003830 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff93ce640 1 -- 172.21.15.16:0/3809323633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff40b61e0 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.226+0000 7f0ff93ce640 1 -- 172.21.15.16:0/3809323633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff40b67a0 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.227+0000 7f0ff93ce640 1 -- 172.21.15.16:0/3809323633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff40b6930 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.228+0000 7f0feb7fe640 1 -- 172.21.15.16:0/3809323633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0fec003990 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.228+0000 7f0feb7fe640 1 --2- 172.21.15.16:0/3809323633 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0fd406ae40 0x7f0fd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.228+0000 7f0feb7fe640 1 -- 172.21.15.16:0/3809323633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f0fec088f70 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.229+0000 7f0ff27fc640 1 --2- 172.21.15.16:0/3809323633 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0fd406ae40 0x7f0fd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.229+0000 7f0ff27fc640 1 --2- 172.21.15.16:0/3809323633 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0fd406ae40 0x7f0fd406d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc036000 tx=0x7f0fdc03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.233+0000 7f0feb7fe640 1 -- 172.21.15.16:0/3809323633 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0fec0bbe90 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.410+0000 7f0ff93ce640 1 -- 172.21.15.16:0/3809323633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ff4067ad0 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.412+0000 7f0feb7fe640 1 -- 172.21.15.16:0/3809323633 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7f0fec017090 con 0x7f0ff40b1ea0 2023-12-08T08:39:07.415 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:07.415 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:07.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.413+0000 7f0ff93ce640 1 -- 172.21.15.16:0/3809323633 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0fd406ae40 msgr2=0x7f0fd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.413+0000 7f0ff93ce640 1 --2- 172.21.15.16:0/3809323633 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0fd406ae40 0x7f0fd406d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc036000 tx=0x7f0fdc03c040 comp rx=0 tx=0).stop 2023-12-08T08:39:07.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.413+0000 7f0ff93ce640 1 -- 172.21.15.16:0/3809323633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ff40b1ea0 msgr2=0x7f0ff41f3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.413+0000 7f0ff93ce640 1 --2- 172.21.15.16:0/3809323633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ff40b1ea0 0x7f0ff41f3540 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00c460 tx=0x7f0fec00c920 comp rx=0 tx=0).stop 2023-12-08T08:39:07.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.413+0000 7f0ff93ce640 1 -- 172.21.15.16:0/3809323633 shutdown_connections 2023-12-08T08:39:07.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.413+0000 7f0ff93ce640 1 --2- 172.21.15.16:0/3809323633 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0fd406ae40 0x7f0fd406d300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.413+0000 7f0ff93ce640 1 --2- 172.21.15.16:0/3809323633 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0ff4160a00 0x7f0ff414f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.413+0000 7f0ff93ce640 1 --2- 172.21.15.16:0/3809323633 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0ff40cffe0 0x7f0ff40cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.413+0000 7f0ff93ce640 1 --2- 172.21.15.16:0/3809323633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ff40b1ea0 0x7f0ff41f3540 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.413+0000 7f0ff93ce640 1 -- 172.21.15.16:0/3809323633 >> 172.21.15.16:0/3809323633 conn(0x7f0ff409c300 msgr2=0x7f0ff40cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:07.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.414+0000 7f0ff93ce640 1 -- 172.21.15.16:0/3809323633 shutdown_connections 2023-12-08T08:39:07.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.414+0000 7f0ff93ce640 1 -- 172.21.15.16:0/3809323633 wait complete. 2023-12-08T08:39:07.427 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:39:07.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:07.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.534+0000 7f6843c64640 1 Processor -- start 2023-12-08T08:39:07.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.535+0000 7f6843c64640 1 -- start start 2023-12-08T08:39:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.535+0000 7f6843c64640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f683c0cf9b0 0x7f683c0b7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.535+0000 7f6843c64640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c0b5f00 con 0x7f683c0cf570 2023-12-08T08:39:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.535+0000 7f6843c64640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683c0b6170 con 0x7f683c0b8950 2023-12-08T08:39:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.535+0000 7f6843c64640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f683c0b6450 con 0x7f683c0cf9b0 2023-12-08T08:39:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.536+0000 7f68411ff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f683c0cf570 0x7f683c0b6c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50638/0 (socket says 172.21.15.16:50638) 2023-12-08T08:39:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.536+0000 7f68411ff640 1 -- 172.21.15.16:0/1691702045 learned_addr learned my addr 172.21.15.16:0/1691702045 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.536+0000 7f6842201640 1 --2- 172.21.15.16:0/1691702045 >> v2:172.21.15.16:3300/0 conn(0x7f683c0cf9b0 0x7f683c0b7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.536+0000 7f68409fe640 1 -- 172.21.15.16:0/1691702045 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959897805 0 0) 0x7f683c0b5f00 con 0x7f683c0cf570 2023-12-08T08:39:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.536+0000 7f68409fe640 1 -- 172.21.15.16:0/1691702045 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6824003520 con 0x7f683c0cf570 2023-12-08T08:39:07.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.536+0000 7f6842201640 1 -- 172.21.15.16:0/1691702045 >> v1:172.21.15.112:6789/0 conn(0x7f683c0b8950 legacy=0x7f683c0ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.536+0000 7f6842201640 1 -- 172.21.15.16:0/1691702045 >> v1:172.21.15.16:6789/0 conn(0x7f683c0cf570 legacy=0x7f683c0b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.536+0000 7f68409fe640 1 -- 172.21.15.16:0/1691702045 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051263840 0 0) 0x7f683c0b6170 con 0x7f683c0b8950 2023-12-08T08:39:07.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.536+0000 7f6842201640 1 -- 172.21.15.16:0/1691702045 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c0b6730 con 0x7f683c0cf9b0 2023-12-08T08:39:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.537+0000 7f6842201640 1 --2- 172.21.15.16:0/1691702045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c0cf9b0 0x7f683c0b7360 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f68340097a0 tx=0x7f683402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2b28fa26726468 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.537+0000 7f68409fe640 1 -- 172.21.15.16:0/1691702045 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834031830 con 0x7f683c0cf9b0 2023-12-08T08:39:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.537+0000 7f68409fe640 1 -- 172.21.15.16:0/1691702045 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6834031990 con 0x7f683c0cf9b0 2023-12-08T08:39:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.537+0000 7f68409fe640 1 -- 172.21.15.16:0/1691702045 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834031d20 con 0x7f683c0cf9b0 2023-12-08T08:39:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.538+0000 7f6843c64640 1 -- 172.21.15.16:0/1691702045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c0cf9b0 msgr2=0x7f683c0b7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.538+0000 7f6843c64640 1 --2- 172.21.15.16:0/1691702045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c0cf9b0 0x7f683c0b7360 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f68340097a0 tx=0x7f683402b8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.538+0000 7f6843c64640 1 -- 172.21.15.16:0/1691702045 shutdown_connections 2023-12-08T08:39:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.538+0000 7f6843c64640 1 --2- 172.21.15.16:0/1691702045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c0cf9b0 0x7f683c0b7360 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.538+0000 7f6843c64640 1 -- 172.21.15.16:0/1691702045 >> 172.21.15.16:0/1691702045 conn(0x7f683c0d3010 msgr2=0x7f683c0d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.538+0000 7f6843c64640 1 -- 172.21.15.16:0/1691702045 shutdown_connections 2023-12-08T08:39:07.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.538+0000 7f6843c64640 1 -- 172.21.15.16:0/1691702045 wait complete. 2023-12-08T08:39:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.539+0000 7f6843c64640 1 Processor -- start 2023-12-08T08:39:07.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.539+0000 7f6843c64640 1 -- start start 2023-12-08T08:39:07.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f6843c64640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f683c0b8950 0x7f683c1f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f6843c64640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c0cf570 0x7f683c09ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f6843c64640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f683c0cf9b0 0x7f683c0cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f6843c64640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c09e360 con 0x7f683c0cf570 2023-12-08T08:39:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f6843c64640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683c1f5b90 con 0x7f683c0b8950 2023-12-08T08:39:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f6843c64640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683c09d750 con 0x7f683c0cf9b0 2023-12-08T08:39:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f68411ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c0cf570 0x7f683c09ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f68411ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c0cf570 0x7f683c09ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34946/0 (socket says 172.21.15.16:34946) 2023-12-08T08:39:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f68411ff640 1 -- 172.21.15.16:0/3132240325 learned_addr learned my addr 172.21.15.16:0/3132240325 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f68411ff640 1 -- 172.21.15.16:0/3132240325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f683c0cf9b0 msgr2=0x7f683c0cdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f68411ff640 1 --2- 172.21.15.16:0/3132240325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f683c0cf9b0 0x7f683c0cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f68411ff640 1 -- 172.21.15.16:0/3132240325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f683c0b8950 msgr2=0x7f683c1f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f68411ff640 1 --2- 172.21.15.16:0/3132240325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f683c0b8950 0x7f683c1f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f68411ff640 1 -- 172.21.15.16:0/3132240325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6834009400 con 0x7f683c0cf570 2023-12-08T08:39:07.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f68411ff640 1 --2- 172.21.15.16:0/3132240325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c0cf570 0x7f683c09ccb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f683000c500 tx=0x7f683000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f683a7fc640 1 -- 172.21.15.16:0/3132240325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6830005ce0 con 0x7f683c0cf570 2023-12-08T08:39:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f683a7fc640 1 -- 172.21.15.16:0/3132240325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6830005e40 con 0x7f683c0cf570 2023-12-08T08:39:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f683a7fc640 1 -- 172.21.15.16:0/3132240325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6830007330 con 0x7f683c0cf570 2023-12-08T08:39:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.540+0000 7f6843c64640 1 -- 172.21.15.16:0/3132240325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683c09da30 con 0x7f683c0cf570 2023-12-08T08:39:07.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.541+0000 7f6843c64640 1 -- 172.21.15.16:0/3132240325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683c09dff0 con 0x7f683c0cf570 2023-12-08T08:39:07.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.541+0000 7f6843c64640 1 -- 172.21.15.16:0/3132240325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683c067430 con 0x7f683c0cf570 2023-12-08T08:39:07.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.542+0000 7f683a7fc640 1 -- 172.21.15.16:0/3132240325 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6830007550 con 0x7f683c0cf570 2023-12-08T08:39:07.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.542+0000 7f683a7fc640 1 --2- 172.21.15.16:0/3132240325 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f682806ae40 0x7f682806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.542+0000 7f683a7fc640 1 -- 172.21.15.16:0/3132240325 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f6830089020 con 0x7f683c0cf570 2023-12-08T08:39:07.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.543+0000 7f6841a00640 1 --2- 172.21.15.16:0/3132240325 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f682806ae40 0x7f682806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:07.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.543+0000 7f6841a00640 1 --2- 172.21.15.16:0/3132240325 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f682806ae40 0x7f682806d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f682c001d00 tx=0x7f682c007c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:07.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.547+0000 7f683a7fc640 1 -- 172.21.15.16:0/3132240325 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6830053dc0 con 0x7f683c0cf570 2023-12-08T08:39:07.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.722+0000 7f6843c64640 1 -- 172.21.15.16:0/3132240325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f683c0b3fc0 con 0x7f683c0cf570 2023-12-08T08:39:07.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.722+0000 7f683a7fc640 1 -- 172.21.15.16:0/3132240325 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68300573e0 con 0x7f683c0cf570 2023-12-08T08:39:07.725 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:07.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.724+0000 7f6843c64640 1 -- 172.21.15.16:0/3132240325 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f682806ae40 msgr2=0x7f682806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.724+0000 7f6843c64640 1 --2- 172.21.15.16:0/3132240325 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f682806ae40 0x7f682806d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f682c001d00 tx=0x7f682c007c90 comp rx=0 tx=0).stop 2023-12-08T08:39:07.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.724+0000 7f6843c64640 1 -- 172.21.15.16:0/3132240325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c0cf570 msgr2=0x7f683c09ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.724+0000 7f6843c64640 1 --2- 172.21.15.16:0/3132240325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c0cf570 0x7f683c09ccb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f683000c500 tx=0x7f683000c9c0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.724+0000 7f6843c64640 1 -- 172.21.15.16:0/3132240325 shutdown_connections 2023-12-08T08:39:07.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.724+0000 7f6843c64640 1 --2- 172.21.15.16:0/3132240325 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f682806ae40 0x7f682806d300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.724+0000 7f6843c64640 1 --2- 172.21.15.16:0/3132240325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f683c0cf9b0 0x7f683c0cdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.724+0000 7f6843c64640 1 --2- 172.21.15.16:0/3132240325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f683c0cf570 0x7f683c09ccb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.724+0000 7f6843c64640 1 --2- 172.21.15.16:0/3132240325 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f683c0b8950 0x7f683c1f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.724+0000 7f6843c64640 1 -- 172.21.15.16:0/3132240325 >> 172.21.15.16:0/3132240325 conn(0x7f683c0d3010 msgr2=0x7f683c06b570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:07.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.724+0000 7f6843c64640 1 -- 172.21.15.16:0/3132240325 shutdown_connections 2023-12-08T08:39:07.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.724+0000 7f6843c64640 1 -- 172.21.15.16:0/3132240325 wait complete. 2023-12-08T08:39:07.738 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:39:07.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.845+0000 7fa4069e5640 1 Processor -- start 2023-12-08T08:39:07.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.845+0000 7fa4069e5640 1 -- start start 2023-12-08T08:39:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.845+0000 7fa4069e5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa400157a20 0x7fa40015d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.845+0000 7fa4069e5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fa4000cb370 0x7fa40015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.845+0000 7fa4069e5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4000b2b60 con 0x7fa40006d7d0 2023-12-08T08:39:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.845+0000 7fa4069e5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa400157e00 con 0x7fa400157a20 2023-12-08T08:39:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.845+0000 7fa4069e5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40006dd20 con 0x7fa4000cb370 2023-12-08T08:39:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.845+0000 7fa3fffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa400157a20 0x7fa40015d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.846+0000 7fa3fffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa400157a20 0x7fa40015d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34954/0 (socket says 172.21.15.16:34954) 2023-12-08T08:39:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.846+0000 7fa3fffff640 1 -- 172.21.15.16:0/1436560773 learned_addr learned my addr 172.21.15.16:0/1436560773 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.846+0000 7fa404f82640 1 --1- 172.21.15.16:0/1436560773 >> v1:172.21.15.16:6789/0 conn(0x7fa40006d7d0 0x7fa40015a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50640/0 (socket says 172.21.15.16:50640) 2023-12-08T08:39:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.846+0000 7fa3fffff640 1 -- 172.21.15.16:0/1436560773 >> v2:172.21.15.112:3300/0 conn(0x7fa4000cb370 msgr2=0x7fa40015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:07.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.846+0000 7fa3fffff640 1 --2- 172.21.15.16:0/1436560773 >> v2:172.21.15.112:3300/0 conn(0x7fa4000cb370 0x7fa40015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.846+0000 7fa3fffff640 1 -- 172.21.15.16:0/1436560773 >> v1:172.21.15.16:6789/0 conn(0x7fa40006d7d0 legacy=0x7fa40015a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.846+0000 7fa3fffff640 1 -- 172.21.15.16:0/1436560773 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa400070cd0 con 0x7fa400157a20 2023-12-08T08:39:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.847+0000 7fa3fffff640 1 --2- 172.21.15.16:0/1436560773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa400157a20 0x7fa40015d9d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa4000b17f0 tx=0x7fa3f002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f065d9fe1c01c625 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.847+0000 7fa3feffd640 1 -- 172.21.15.16:0/1436560773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0030860 con 0x7fa400157a20 2023-12-08T08:39:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.847+0000 7fa3feffd640 1 -- 172.21.15.16:0/1436560773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f00309c0 con 0x7fa400157a20 2023-12-08T08:39:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.847+0000 7fa3feffd640 1 -- 172.21.15.16:0/1436560773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0030d50 con 0x7fa400157a20 2023-12-08T08:39:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.847+0000 7fa4069e5640 1 -- 172.21.15.16:0/1436560773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa400157a20 msgr2=0x7fa40015d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.847+0000 7fa4069e5640 1 --2- 172.21.15.16:0/1436560773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa400157a20 0x7fa40015d9d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa4000b17f0 tx=0x7fa3f002b9a0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.847+0000 7fa4069e5640 1 -- 172.21.15.16:0/1436560773 shutdown_connections 2023-12-08T08:39:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.847+0000 7fa4069e5640 1 --2- 172.21.15.16:0/1436560773 >> v2:172.21.15.112:3300/0 conn(0x7fa4000cb370 0x7fa40015df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.847+0000 7fa4069e5640 1 --2- 172.21.15.16:0/1436560773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa400157a20 0x7fa40015d9d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.847+0000 7fa4069e5640 1 -- 172.21.15.16:0/1436560773 >> 172.21.15.16:0/1436560773 conn(0x7fa4000b0850 msgr2=0x7fa4000b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.847+0000 7fa4069e5640 1 -- 172.21.15.16:0/1436560773 shutdown_connections 2023-12-08T08:39:07.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.847+0000 7fa4069e5640 1 -- 172.21.15.16:0/1436560773 wait complete. 2023-12-08T08:39:07.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.848+0000 7fa4069e5640 1 Processor -- start 2023-12-08T08:39:07.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.849+0000 7fa4069e5640 1 -- start start 2023-12-08T08:39:07.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.849+0000 7fa4069e5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa40006d7d0 0x7fa400143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.849+0000 7fa4069e5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4000cb370 0x7fa400156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.849+0000 7fa4069e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa400157a20 0x7fa400156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.849+0000 7fa4069e5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa400155380 con 0x7fa400157a20 2023-12-08T08:39:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.849+0000 7fa4069e5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa400155660 con 0x7fa40006d7d0 2023-12-08T08:39:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.849+0000 7fa4069e5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa400155940 con 0x7fa4000cb370 2023-12-08T08:39:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.849+0000 7fa404f82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa400157a20 0x7fa400156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.849+0000 7fa404f82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa400157a20 0x7fa400156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34964/0 (socket says 172.21.15.16:34964) 2023-12-08T08:39:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.850+0000 7fa404f82640 1 -- 172.21.15.16:0/730930133 learned_addr learned my addr 172.21.15.16:0/730930133 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.850+0000 7fa3fffff640 1 --2- 172.21.15.16:0/730930133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa40006d7d0 0x7fa400143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.850+0000 7fa3ff7fe640 1 --2- 172.21.15.16:0/730930133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4000cb370 0x7fa400156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.850+0000 7fa404f82640 1 -- 172.21.15.16:0/730930133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4000cb370 msgr2=0x7fa400156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.850+0000 7fa404f82640 1 --2- 172.21.15.16:0/730930133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4000cb370 0x7fa400156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.850+0000 7fa404f82640 1 -- 172.21.15.16:0/730930133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa40006d7d0 msgr2=0x7fa400143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.850+0000 7fa404f82640 1 --2- 172.21.15.16:0/730930133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa40006d7d0 0x7fa400143650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:07.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.850+0000 7fa404f82640 1 -- 172.21.15.16:0/730930133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f00094d0 con 0x7fa400157a20 2023-12-08T08:39:07.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.850+0000 7fa404f82640 1 --2- 172.21.15.16:0/730930133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa400157a20 0x7fa400156d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800c780 tx=0x7fa3f800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:07.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.850+0000 7fa3fcff9640 1 -- 172.21.15.16:0/730930133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800ce90 con 0x7fa400157a20 2023-12-08T08:39:07.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.850+0000 7fa3fcff9640 1 -- 172.21.15.16:0/730930133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f800a640 con 0x7fa400157a20 2023-12-08T08:39:07.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.850+0000 7fa4069e5640 1 -- 172.21.15.16:0/730930133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa400155c20 con 0x7fa400157a20 2023-12-08T08:39:07.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.850+0000 7fa3fcff9640 1 -- 172.21.15.16:0/730930133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800a9f0 con 0x7fa400157a20 2023-12-08T08:39:07.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.850+0000 7fa4069e5640 1 -- 172.21.15.16:0/730930133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa400158730 con 0x7fa400157a20 2023-12-08T08:39:07.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.851+0000 7fa4069e5640 1 -- 172.21.15.16:0/730930133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa400067430 con 0x7fa400157a20 2023-12-08T08:39:07.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.852+0000 7fa3fcff9640 1 -- 172.21.15.16:0/730930133 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa3f800ab50 con 0x7fa400157a20 2023-12-08T08:39:07.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.852+0000 7fa3fcff9640 1 --2- 172.21.15.16:0/730930133 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3ec06ae40 0x7fa3ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:07.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.852+0000 7fa3fcff9640 1 -- 172.21.15.16:0/730930133 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fa3f8089030 con 0x7fa400157a20 2023-12-08T08:39:07.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.852+0000 7fa3fffff640 1 --2- 172.21.15.16:0/730930133 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3ec06ae40 0x7fa3ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:07.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.853+0000 7fa3fffff640 1 --2- 172.21.15.16:0/730930133 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3ec06ae40 0x7fa3ec06d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0036000 tx=0x7fa3f003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:07.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:07.856+0000 7fa3fcff9640 1 -- 172.21.15.16:0/730930133 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3f8053e40 con 0x7fa400157a20 2023-12-08T08:39:08.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.034+0000 7fa4069e5640 1 -- 172.21.15.16:0/730930133 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4000b3030 con 0x7fa400157a20 2023-12-08T08:39:08.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.035+0000 7fa3fcff9640 1 -- 172.21.15.16:0/730930133 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa3f8017020 con 0x7fa400157a20 2023-12-08T08:39:08.038 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.037+0000 7fa4069e5640 1 -- 172.21.15.16:0/730930133 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3ec06ae40 msgr2=0x7fa3ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.037+0000 7fa4069e5640 1 --2- 172.21.15.16:0/730930133 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3ec06ae40 0x7fa3ec06d300 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0036000 tx=0x7fa3f003b040 comp rx=0 tx=0).stop 2023-12-08T08:39:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.037+0000 7fa4069e5640 1 -- 172.21.15.16:0/730930133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa400157a20 msgr2=0x7fa400156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.037+0000 7fa4069e5640 1 --2- 172.21.15.16:0/730930133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa400157a20 0x7fa400156d60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800c780 tx=0x7fa3f800cc40 comp rx=0 tx=0).stop 2023-12-08T08:39:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.037+0000 7fa4069e5640 1 -- 172.21.15.16:0/730930133 shutdown_connections 2023-12-08T08:39:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.037+0000 7fa4069e5640 1 --2- 172.21.15.16:0/730930133 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa3ec06ae40 0x7fa3ec06d300 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.037+0000 7fa4069e5640 1 --2- 172.21.15.16:0/730930133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa400157a20 0x7fa400156d60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.037+0000 7fa4069e5640 1 --2- 172.21.15.16:0/730930133 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa4000cb370 0x7fa400156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.037+0000 7fa4069e5640 1 --2- 172.21.15.16:0/730930133 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa40006d7d0 0x7fa400143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.037+0000 7fa4069e5640 1 -- 172.21.15.16:0/730930133 >> 172.21.15.16:0/730930133 conn(0x7fa4000b0850 msgr2=0x7fa4000afe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:08.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.037+0000 7fa4069e5640 1 -- 172.21.15.16:0/730930133 shutdown_connections 2023-12-08T08:39:08.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.037+0000 7fa4069e5640 1 -- 172.21.15.16:0/730930133 wait complete. 2023-12-08T08:39:08.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:39:08.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb67d89640 1 Processor -- start 2023-12-08T08:39:08.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb67d89640 1 -- start start 2023-12-08T08:39:08.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb67d89640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdb600b8950 0x7fdb600ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb67d89640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb600cfc80 0x7fdb600b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb67d89640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb600b5f00 con 0x7fdb600cebc0 2023-12-08T08:39:08.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb67d89640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb600b6170 con 0x7fdb600cfc80 2023-12-08T08:39:08.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb67d89640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb600b6450 con 0x7fdb600b8950 2023-12-08T08:39:08.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb66326640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb600cfc80 0x7fdb600b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb66326640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb600cfc80 0x7fdb600b6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34980/0 (socket says 172.21.15.16:34980) 2023-12-08T08:39:08.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb66326640 1 -- 172.21.15.16:0/1808795063 learned_addr learned my addr 172.21.15.16:0/1808795063 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:08.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb65b25640 1 --2- 172.21.15.16:0/1808795063 >> v2:172.21.15.112:3300/0 conn(0x7fdb600b8950 0x7fdb600ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb66326640 1 -- 172.21.15.16:0/1808795063 >> v2:172.21.15.112:3300/0 conn(0x7fdb600b8950 msgr2=0x7fdb600ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:08.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb66326640 1 --2- 172.21.15.16:0/1808795063 >> v2:172.21.15.112:3300/0 conn(0x7fdb600b8950 0x7fdb600ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb66326640 1 -- 172.21.15.16:0/1808795063 >> v1:172.21.15.112:6789/0 conn(0x7fdb600cebc0 legacy=0x7fdb600cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.158+0000 7fdb66326640 1 -- 172.21.15.16:0/1808795063 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb600b6730 con 0x7fdb600cfc80 2023-12-08T08:39:08.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.159+0000 7fdb66326640 1 --2- 172.21.15.16:0/1808795063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb600cfc80 0x7fdb600b6c50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdb6008b950 tx=0x7fdb5802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5aa6c3b18a937c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:08.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.159+0000 7fdb64b23640 1 -- 172.21.15.16:0/1808795063 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5802ca80 con 0x7fdb600cfc80 2023-12-08T08:39:08.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.159+0000 7fdb64b23640 1 -- 172.21.15.16:0/1808795063 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5802cbe0 con 0x7fdb600cfc80 2023-12-08T08:39:08.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.159+0000 7fdb64b23640 1 -- 172.21.15.16:0/1808795063 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb580343f0 con 0x7fdb600cfc80 2023-12-08T08:39:08.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.160+0000 7fdb67d89640 1 -- 172.21.15.16:0/1808795063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb600cfc80 msgr2=0x7fdb600b6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.160+0000 7fdb67d89640 1 --2- 172.21.15.16:0/1808795063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb600cfc80 0x7fdb600b6c50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdb6008b950 tx=0x7fdb5802bee0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.160+0000 7fdb67d89640 1 -- 172.21.15.16:0/1808795063 shutdown_connections 2023-12-08T08:39:08.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.160+0000 7fdb67d89640 1 --2- 172.21.15.16:0/1808795063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb600cfc80 0x7fdb600b6c50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.160+0000 7fdb67d89640 1 --2- 172.21.15.16:0/1808795063 >> v2:172.21.15.112:3300/0 conn(0x7fdb600b8950 0x7fdb600ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.160+0000 7fdb67d89640 1 -- 172.21.15.16:0/1808795063 >> 172.21.15.16:0/1808795063 conn(0x7fdb600d3010 msgr2=0x7fdb600d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:08.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.160+0000 7fdb67d89640 1 -- 172.21.15.16:0/1808795063 shutdown_connections 2023-12-08T08:39:08.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.160+0000 7fdb67d89640 1 -- 172.21.15.16:0/1808795063 wait complete. 2023-12-08T08:39:08.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.161+0000 7fdb67d89640 1 Processor -- start 2023-12-08T08:39:08.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.161+0000 7fdb67d89640 1 -- start start 2023-12-08T08:39:08.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb67d89640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb600b8950 0x7fdb601f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb67d89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb600cebc0 0x7fdb6014a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb67d89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb600cfc80 0x7fdb600d0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb67d89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6014a9f0 con 0x7fdb600cebc0 2023-12-08T08:39:08.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb67d89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb60157640 con 0x7fdb600cfc80 2023-12-08T08:39:08.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb67d89640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb60157920 con 0x7fdb600b8950 2023-12-08T08:39:08.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb65324640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb600cebc0 0x7fdb6014a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb65324640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb600cebc0 0x7fdb6014a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34988/0 (socket says 172.21.15.16:34988) 2023-12-08T08:39:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb65324640 1 -- 172.21.15.16:0/158190462 learned_addr learned my addr 172.21.15.16:0/158190462 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb65324640 1 -- 172.21.15.16:0/158190462 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb600b8950 msgr2=0x7fdb601f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb65324640 1 --2- 172.21.15.16:0/158190462 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb600b8950 0x7fdb601f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb65324640 1 -- 172.21.15.16:0/158190462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb600cfc80 msgr2=0x7fdb600d0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb66326640 1 --2- 172.21.15.16:0/158190462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb600cfc80 0x7fdb600d0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb65324640 1 --2- 172.21.15.16:0/158190462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb600cfc80 0x7fdb600d0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb65324640 1 -- 172.21.15.16:0/158190462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb58009b30 con 0x7fdb600cebc0 2023-12-08T08:39:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb65324640 1 --2- 172.21.15.16:0/158190462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb600cebc0 0x7fdb6014a4b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000c400 tx=0x7fdb5000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb567fc640 1 -- 172.21.15.16:0/158190462 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb500052d0 con 0x7fdb600cebc0 2023-12-08T08:39:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.162+0000 7fdb567fc640 1 -- 172.21.15.16:0/158190462 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb50005430 con 0x7fdb600cebc0 2023-12-08T08:39:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.163+0000 7fdb67d89640 1 -- 172.21.15.16:0/158190462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb60157c00 con 0x7fdb600cebc0 2023-12-08T08:39:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.163+0000 7fdb567fc640 1 -- 172.21.15.16:0/158190462 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50003830 con 0x7fdb600cebc0 2023-12-08T08:39:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.163+0000 7fdb67d89640 1 -- 172.21.15.16:0/158190462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb601581c0 con 0x7fdb600cebc0 2023-12-08T08:39:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.164+0000 7fdb67d89640 1 -- 172.21.15.16:0/158190462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb6008b150 con 0x7fdb600cebc0 2023-12-08T08:39:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.164+0000 7fdb567fc640 1 -- 172.21.15.16:0/158190462 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb50003990 con 0x7fdb600cebc0 2023-12-08T08:39:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.164+0000 7fdb567fc640 1 --2- 172.21.15.16:0/158190462 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdb4406ae40 0x7fdb4406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.165+0000 7fdb567fc640 1 -- 172.21.15.16:0/158190462 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fdb50088f90 con 0x7fdb600cebc0 2023-12-08T08:39:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.165+0000 7fdb65b25640 1 --2- 172.21.15.16:0/158190462 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdb4406ae40 0x7fdb4406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.165+0000 7fdb65b25640 1 --2- 172.21.15.16:0/158190462 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdb4406ae40 0x7fdb4406d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c006fd0 tx=0x7fdb5c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:08.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.170+0000 7fdb567fc640 1 -- 172.21.15.16:0/158190462 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb500bbe90 con 0x7fdb600cebc0 2023-12-08T08:39:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.346+0000 7fdb67d89640 1 -- 172.21.15.16:0/158190462 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb60067430 con 0x7fdb600cebc0 2023-12-08T08:39:08.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.346+0000 7fdb567fc640 1 -- 172.21.15.16:0/158190462 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb500170a0 con 0x7fdb600cebc0 2023-12-08T08:39:08.349 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:39:08.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.348+0000 7fdb67d89640 1 -- 172.21.15.16:0/158190462 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdb4406ae40 msgr2=0x7fdb4406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.348+0000 7fdb67d89640 1 --2- 172.21.15.16:0/158190462 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdb4406ae40 0x7fdb4406d300 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c006fd0 tx=0x7fdb5c008040 comp rx=0 tx=0).stop 2023-12-08T08:39:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.348+0000 7fdb67d89640 1 -- 172.21.15.16:0/158190462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb600cebc0 msgr2=0x7fdb6014a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.348+0000 7fdb67d89640 1 --2- 172.21.15.16:0/158190462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb600cebc0 0x7fdb6014a4b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000c400 tx=0x7fdb5000c8c0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.348+0000 7fdb67d89640 1 -- 172.21.15.16:0/158190462 shutdown_connections 2023-12-08T08:39:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.348+0000 7fdb67d89640 1 --2- 172.21.15.16:0/158190462 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdb4406ae40 0x7fdb4406d300 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.348+0000 7fdb67d89640 1 --2- 172.21.15.16:0/158190462 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdb600cfc80 0x7fdb600d0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.348+0000 7fdb67d89640 1 --2- 172.21.15.16:0/158190462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb600cebc0 0x7fdb6014a4b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.348+0000 7fdb67d89640 1 --2- 172.21.15.16:0/158190462 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdb600b8950 0x7fdb601f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.348+0000 7fdb67d89640 1 -- 172.21.15.16:0/158190462 >> 172.21.15.16:0/158190462 conn(0x7fdb600d3010 msgr2=0x7fdb600b3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.348+0000 7fdb67d89640 1 -- 172.21.15.16:0/158190462 shutdown_connections 2023-12-08T08:39:08.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.348+0000 7fdb67d89640 1 -- 172.21.15.16:0/158190462 wait complete. 2023-12-08T08:39:08.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:39:08.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.469+0000 7f9b49f9e640 1 Processor -- start 2023-12-08T08:39:08.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.469+0000 7f9b49f9e640 1 -- start start 2023-12-08T08:39:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b49f9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b440d1ba0 0x7f9b440cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b49f9e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b440b2310 con 0x7f9b440b40a0 2023-12-08T08:39:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b49f9e640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b440d39d0 con 0x7f9b4415d3f0 2023-12-08T08:39:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b49f9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440d02a0 con 0x7f9b440d1ba0 2023-12-08T08:39:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b4979d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9b440b40a0 0x7f9b440cdf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50652/0 (socket says 172.21.15.16:50652) 2023-12-08T08:39:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b4979d640 1 -- 172.21.15.16:0/1893314530 learned_addr learned my addr 172.21.15.16:0/1893314530 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:08.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3bfff640 1 --2- 172.21.15.16:0/1893314530 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b440d1ba0 0x7f9b440cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 -- 172.21.15.16:0/1893314530 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455871320 0 0) 0x7f9b440b2310 con 0x7f9b440b40a0 2023-12-08T08:39:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 -- 172.21.15.16:0/1893314530 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b2c003520 con 0x7f9b440b40a0 2023-12-08T08:39:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 -- 172.21.15.16:0/1893314530 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2147865851 0 0) 0x7f9b2c003520 con 0x7f9b440b40a0 2023-12-08T08:39:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 -- 172.21.15.16:0/1893314530 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b440d1ba0 msgr2=0x7f9b440cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 --2- 172.21.15.16:0/1893314530 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b440d1ba0 0x7f9b440cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 -- 172.21.15.16:0/1893314530 >> v1:172.21.15.112:6789/0 conn(0x7f9b4415d3f0 legacy=0x7f9b440d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 -- 172.21.15.16:0/1893314530 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b440d3c90 con 0x7f9b440b40a0 2023-12-08T08:39:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 -- 172.21.15.16:0/1893314530 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f9b3c0019f0 con 0x7f9b440b40a0 2023-12-08T08:39:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 -- 172.21.15.16:0/1893314530 >> v1:172.21.15.16:6789/0 conn(0x7f9b440b40a0 legacy=0x7f9b440cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 --2- 172.21.15.16:0/1893314530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b2c0039a0 0x7f9b2c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 --2- 172.21.15.16:0/1893314530 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b440d1ba0 0x7f9b2c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 --2- 172.21.15.16:0/1893314530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c004d00 0x7f9b2c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 -- 172.21.15.16:0/1893314530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440d02a0 con 0x7f9b2c004d00 2023-12-08T08:39:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 -- 172.21.15.16:0/1893314530 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c00b9b0 con 0x7f9b2c0039a0 2023-12-08T08:39:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3b7fe640 1 -- 172.21.15.16:0/1893314530 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c00bb70 con 0x7f9b440d1ba0 2023-12-08T08:39:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b4979d640 1 --2- 172.21.15.16:0/1893314530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c004d00 0x7f9b2c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b48f9c640 1 --2- 172.21.15.16:0/1893314530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b2c0039a0 0x7f9b2c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b3bfff640 1 --2- 172.21.15.16:0/1893314530 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b440d1ba0 0x7f9b2c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b4979d640 1 -- 172.21.15.16:0/1893314530 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b440d1ba0 msgr2=0x7f9b2c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b4979d640 1 --2- 172.21.15.16:0/1893314530 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b440d1ba0 0x7f9b2c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b4979d640 1 -- 172.21.15.16:0/1893314530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b2c0039a0 msgr2=0x7f9b2c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b4979d640 1 --2- 172.21.15.16:0/1893314530 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b2c0039a0 0x7f9b2c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.470+0000 7f9b4979d640 1 -- 172.21.15.16:0/1893314530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c003520 con 0x7f9b2c004d00 2023-12-08T08:39:08.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.471+0000 7f9b4979d640 1 --2- 172.21.15.16:0/1893314530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c004d00 0x7f9b2c005180 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00a7d0 tx=0x7f9b2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80193001eadc7dd server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:08.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.471+0000 7f9b3b7fe640 1 -- 172.21.15.16:0/1893314530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c034630 con 0x7f9b2c004d00 2023-12-08T08:39:08.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.471+0000 7f9b3b7fe640 1 -- 172.21.15.16:0/1893314530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3c02f070 con 0x7f9b2c004d00 2023-12-08T08:39:08.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.471+0000 7f9b3b7fe640 1 -- 172.21.15.16:0/1893314530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c0349f0 con 0x7f9b2c004d00 2023-12-08T08:39:08.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.471+0000 7f9b49f9e640 1 -- 172.21.15.16:0/1893314530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c004d00 msgr2=0x7f9b2c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.471+0000 7f9b49f9e640 1 --2- 172.21.15.16:0/1893314530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b2c004d00 0x7f9b2c005180 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00a7d0 tx=0x7f9b2c0057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.471+0000 7f9b4979d640 1 -- 172.21.15.16:0/1893314530 reap_dead start 2023-12-08T08:39:08.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.471+0000 7f9b49f9e640 1 -- 172.21.15.16:0/1893314530 shutdown_connections 2023-12-08T08:39:08.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.471+0000 7f9b49f9e640 1 -- 172.21.15.16:0/1893314530 >> 172.21.15.16:0/1893314530 conn(0x7f9b440d2eb0 msgr2=0x7f9b440d32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:08.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.471+0000 7f9b49f9e640 1 -- 172.21.15.16:0/1893314530 shutdown_connections 2023-12-08T08:39:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.472+0000 7f9b49f9e640 1 -- 172.21.15.16:0/1893314530 wait complete. 2023-12-08T08:39:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.474+0000 7f9b49f9e640 1 Processor -- start 2023-12-08T08:39:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.474+0000 7f9b49f9e640 1 -- start start 2023-12-08T08:39:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b49f9e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b4415ad10 0x7f9b4415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b49f9e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b4415b0f0 0x7f9b440b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b49f9e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b440b5fb0 0x7f9b4409d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b49f9e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440b6460 con 0x7f9b4415ad10 2023-12-08T08:39:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b49f9e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4415db40 con 0x7f9b440b5fb0 2023-12-08T08:39:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b49f9e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b4415de20 con 0x7f9b4415b0f0 2023-12-08T08:39:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b48f9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b4415ad10 0x7f9b4415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b48f9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b4415ad10 0x7f9b4415ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35000/0 (socket says 172.21.15.16:35000) 2023-12-08T08:39:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b48f9c640 1 -- 172.21.15.16:0/4232795343 learned_addr learned my addr 172.21.15.16:0/4232795343 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b3bfff640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b4415b0f0 0x7f9b440b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b4979d640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b440b5fb0 0x7f9b4409d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b48f9c640 1 -- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b4415b0f0 msgr2=0x7f9b440b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b48f9c640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b4415b0f0 0x7f9b440b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b48f9c640 1 -- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b440b5fb0 msgr2=0x7f9b4409d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b48f9c640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b440b5fb0 0x7f9b4409d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b48f9c640 1 -- 172.21.15.16:0/4232795343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4415e100 con 0x7f9b4415ad10 2023-12-08T08:39:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b3bfff640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b4415b0f0 0x7f9b440b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b4979d640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b440b5fb0 0x7f9b4409d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b48f9c640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b4415ad10 0x7f9b4415ebe0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000f870 tx=0x7f9b4000fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b397fa640 1 -- 172.21.15.16:0/4232795343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40005320 con 0x7f9b4415ad10 2023-12-08T08:39:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b397fa640 1 -- 172.21.15.16:0/4232795343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b400054a0 con 0x7f9b4415ad10 2023-12-08T08:39:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b397fa640 1 -- 172.21.15.16:0/4232795343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b40007290 con 0x7f9b4415ad10 2023-12-08T08:39:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b49f9e640 1 -- 172.21.15.16:0/4232795343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b4415e3e0 con 0x7f9b4415ad10 2023-12-08T08:39:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.475+0000 7f9b49f9e640 1 -- 172.21.15.16:0/4232795343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b4415d7d0 con 0x7f9b4415ad10 2023-12-08T08:39:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.476+0000 7f9b49f9e640 1 -- 172.21.15.16:0/4232795343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4415e900 con 0x7f9b4415ad10 2023-12-08T08:39:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.476+0000 7f9b397fa640 1 -- 172.21.15.16:0/4232795343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9b400073f0 con 0x7f9b4415ad10 2023-12-08T08:39:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.476+0000 7f9b397fa640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b3006ae70 0x7f9b3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.476+0000 7f9b397fa640 1 -- 172.21.15.16:0/4232795343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f9b400891b0 con 0x7f9b4415ad10 2023-12-08T08:39:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.477+0000 7f9b3bfff640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b3006ae70 0x7f9b3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.477+0000 7f9b3bfff640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b3006ae70 0x7f9b3006d330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9b4415fd00 tx=0x7f9b34008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:08.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.482+0000 7f9b397fa640 1 -- 172.21.15.16:0/4232795343 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9b4415e900 con 0x7f9b4415ad10 2023-12-08T08:39:08.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.657+0000 7f9b49f9e640 1 -- 172.21.15.16:0/4232795343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9b4415e900 con 0x7f9b4415ad10 2023-12-08T08:39:08.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.658+0000 7f9b397fa640 1 -- 172.21.15.16:0/4232795343 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9b4415e900 con 0x7f9b4415ad10 2023-12-08T08:39:08.660 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:08.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.659+0000 7f9b49f9e640 1 -- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b3006ae70 msgr2=0x7f9b3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.659+0000 7f9b49f9e640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b3006ae70 0x7f9b3006d330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9b4415fd00 tx=0x7f9b34008040 comp rx=0 tx=0).stop 2023-12-08T08:39:08.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.659+0000 7f9b49f9e640 1 -- 172.21.15.16:0/4232795343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b4415ad10 msgr2=0x7f9b4415ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.659+0000 7f9b49f9e640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b4415ad10 0x7f9b4415ebe0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000f870 tx=0x7f9b4000fd30 comp rx=0 tx=0).stop 2023-12-08T08:39:08.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.660+0000 7f9b49f9e640 1 -- 172.21.15.16:0/4232795343 shutdown_connections 2023-12-08T08:39:08.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.660+0000 7f9b49f9e640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f9b3006ae70 0x7f9b3006d330 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.660+0000 7f9b49f9e640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f9b440b5fb0 0x7f9b4409d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.660+0000 7f9b49f9e640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f9b4415b0f0 0x7f9b440b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.660+0000 7f9b49f9e640 1 --2- 172.21.15.16:0/4232795343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9b4415ad10 0x7f9b4415ebe0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.660+0000 7f9b49f9e640 1 -- 172.21.15.16:0/4232795343 >> 172.21.15.16:0/4232795343 conn(0x7f9b440d2eb0 msgr2=0x7f9b4406c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:08.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.660+0000 7f9b49f9e640 1 -- 172.21.15.16:0/4232795343 shutdown_connections 2023-12-08T08:39:08.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.660+0000 7f9b49f9e640 1 -- 172.21.15.16:0/4232795343 wait complete. 2023-12-08T08:39:08.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2023-12-08T08:39:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.777+0000 7f755db51640 1 Processor -- start 2023-12-08T08:39:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.777+0000 7f755db51640 1 -- start start 2023-12-08T08:39:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.778+0000 7f755db51640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f75580d1d00 0x7f75580d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.778+0000 7f755db51640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75580b3bf0 0x7f75580b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.778+0000 7f755db51640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75580cdc20 con 0x7f75580b3340 2023-12-08T08:39:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.778+0000 7f755db51640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75580d11d0 con 0x7f75580d1d00 2023-12-08T08:39:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.778+0000 7f755db51640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75580d3a60 con 0x7f75580b3bf0 2023-12-08T08:39:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.778+0000 7f7557fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75580b3bf0 0x7f75580b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.778+0000 7f7556ffd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f75580d1d00 0x7f75580d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.778+0000 7f7557fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75580b3bf0 0x7f75580b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:45192/0 (socket says 172.21.15.16:45192) 2023-12-08T08:39:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.778+0000 7f7557fff640 1 -- 172.21.15.16:0/119200263 learned_addr learned my addr 172.21.15.16:0/119200263 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.778+0000 7f75577fe640 1 --1- 172.21.15.16:0/119200263 >> v1:172.21.15.112:6789/0 conn(0x7f75580b3340 0x7f75580d15f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:33476/0 (socket says 172.21.15.16:33476) 2023-12-08T08:39:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.779+0000 7f7557fff640 1 -- 172.21.15.16:0/119200263 >> v2:172.21.15.112:3300/0 conn(0x7f75580d1d00 msgr2=0x7f75580d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.779+0000 7f7557fff640 1 --2- 172.21.15.16:0/119200263 >> v2:172.21.15.112:3300/0 conn(0x7f75580d1d00 0x7f75580d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.779+0000 7f7557fff640 1 -- 172.21.15.16:0/119200263 >> v1:172.21.15.112:6789/0 conn(0x7f75580b3340 legacy=0x7f75580d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.779+0000 7f7557fff640 1 -- 172.21.15.16:0/119200263 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75580d3d40 con 0x7f75580b3bf0 2023-12-08T08:39:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.780+0000 7f7557fff640 1 --2- 172.21.15.16:0/119200263 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75580b3bf0 0x7f75580b5e40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7550009e50 tx=0x7f755002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc73f47a0fa2cd88 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.780+0000 7f75567fc640 1 -- 172.21.15.16:0/119200263 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755002ca70 con 0x7f75580b3bf0 2023-12-08T08:39:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.780+0000 7f75567fc640 1 -- 172.21.15.16:0/119200263 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755002cbd0 con 0x7f75580b3bf0 2023-12-08T08:39:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.780+0000 7f75567fc640 1 -- 172.21.15.16:0/119200263 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75500343f0 con 0x7f75580b3bf0 2023-12-08T08:39:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.780+0000 7f755db51640 1 -- 172.21.15.16:0/119200263 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75580b3bf0 msgr2=0x7f75580b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.780+0000 7f755db51640 1 --2- 172.21.15.16:0/119200263 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75580b3bf0 0x7f75580b5e40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7550009e50 tx=0x7f755002bed0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.780+0000 7f755db51640 1 -- 172.21.15.16:0/119200263 shutdown_connections 2023-12-08T08:39:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.780+0000 7f755db51640 1 --2- 172.21.15.16:0/119200263 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75580b3bf0 0x7f75580b5e40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.780+0000 7f755db51640 1 --2- 172.21.15.16:0/119200263 >> v2:172.21.15.112:3300/0 conn(0x7f75580d1d00 0x7f75580d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.780+0000 7f755db51640 1 -- 172.21.15.16:0/119200263 >> 172.21.15.16:0/119200263 conn(0x7f755809c2f0 msgr2=0x7f75580d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.780+0000 7f755db51640 1 -- 172.21.15.16:0/119200263 shutdown_connections 2023-12-08T08:39:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.780+0000 7f755db51640 1 -- 172.21.15.16:0/119200263 wait complete. 2023-12-08T08:39:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.781+0000 7f755db51640 1 Processor -- start 2023-12-08T08:39:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.782+0000 7f755db51640 1 -- start start 2023-12-08T08:39:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.782+0000 7f755db51640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75580b3340 0x7f75580a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.782+0000 7f755db51640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75580b3bf0 0x7f75580afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.782+0000 7f755db51640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75580d1d00 0x7f75580b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.782+0000 7f755db51640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755815f200 con 0x7f75580d1d00 2023-12-08T08:39:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.782+0000 7f755db51640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75580b0750 con 0x7f75580b3bf0 2023-12-08T08:39:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.782+0000 7f755db51640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75580b0a30 con 0x7f75580b3340 2023-12-08T08:39:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f7557fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75580d1d00 0x7f75580b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f7557fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75580d1d00 0x7f75580b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35012/0 (socket says 172.21.15.16:35012) 2023-12-08T08:39:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f7557fff640 1 -- 172.21.15.16:0/2903013689 learned_addr learned my addr 172.21.15.16:0/2903013689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f75577fe640 1 --2- 172.21.15.16:0/2903013689 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75580b3340 0x7f75580a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f7556ffd640 1 --2- 172.21.15.16:0/2903013689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75580b3bf0 0x7f75580afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f7557fff640 1 -- 172.21.15.16:0/2903013689 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75580b3340 msgr2=0x7f75580a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f7557fff640 1 --2- 172.21.15.16:0/2903013689 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75580b3340 0x7f75580a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f7557fff640 1 -- 172.21.15.16:0/2903013689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75580b3bf0 msgr2=0x7f75580afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f7557fff640 1 --2- 172.21.15.16:0/2903013689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75580b3bf0 0x7f75580afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f7557fff640 1 -- 172.21.15.16:0/2903013689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7550009b30 con 0x7f75580d1d00 2023-12-08T08:39:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f7557fff640 1 --2- 172.21.15.16:0/2903013689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75580d1d00 0x7f75580b0120 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7550009e50 tx=0x7f7550004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f753ffff640 1 -- 172.21.15.16:0/2903013689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7550009510 con 0x7f75580d1d00 2023-12-08T08:39:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f753ffff640 1 -- 172.21.15.16:0/2903013689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755003e450 con 0x7f75580d1d00 2023-12-08T08:39:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f755db51640 1 -- 172.21.15.16:0/2903013689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75580b0d10 con 0x7f75580d1d00 2023-12-08T08:39:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f753ffff640 1 -- 172.21.15.16:0/2903013689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755003e800 con 0x7f75580d1d00 2023-12-08T08:39:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.783+0000 7f755db51640 1 -- 172.21.15.16:0/2903013689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75580b12d0 con 0x7f75580d1d00 2023-12-08T08:39:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.784+0000 7f755db51640 1 -- 172.21.15.16:0/2903013689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7558067430 con 0x7f75580d1d00 2023-12-08T08:39:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.785+0000 7f753ffff640 1 -- 172.21.15.16:0/2903013689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f755003e960 con 0x7f75580d1d00 2023-12-08T08:39:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.785+0000 7f753ffff640 1 --2- 172.21.15.16:0/2903013689 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f754406ae40 0x7f754406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.785+0000 7f753ffff640 1 -- 172.21.15.16:0/2903013689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f75500ac660 con 0x7f75580d1d00 2023-12-08T08:39:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.786+0000 7f75577fe640 1 --2- 172.21.15.16:0/2903013689 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f754406ae40 0x7f754406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.786+0000 7f75577fe640 1 --2- 172.21.15.16:0/2903013689 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f754406ae40 0x7f754406d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7548000f60 tx=0x7f7548000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:08.790+0000 7f753ffff640 1 -- 172.21.15.16:0/2903013689 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7550077340 con 0x7f75580d1d00 2023-12-08T08:39:09.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.034+0000 7f755db51640 1 -- 172.21.15.16:0/2903013689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) v1 -- 0x7f75580b14c0 con 0x7f75580d1d00 2023-12-08T08:39:09.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.821+0000 7f753ffff640 1 -- 172.21.15.16:0/2903013689 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v72) v1 ==== 147+0+0 (secure 0 0 0) 0x7f755007a960 con 0x7f75580d1d00 2023-12-08T08:39:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.824+0000 7f755db51640 1 -- 172.21.15.16:0/2903013689 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f754406ae40 msgr2=0x7f754406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.824+0000 7f755db51640 1 --2- 172.21.15.16:0/2903013689 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f754406ae40 0x7f754406d300 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7548000f60 tx=0x7f7548000ed0 comp rx=0 tx=0).stop 2023-12-08T08:39:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.824+0000 7f755db51640 1 -- 172.21.15.16:0/2903013689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75580d1d00 msgr2=0x7f75580b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.824+0000 7f755db51640 1 --2- 172.21.15.16:0/2903013689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75580d1d00 0x7f75580b0120 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7550009e50 tx=0x7f7550004a40 comp rx=0 tx=0).stop 2023-12-08T08:39:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.824+0000 7f755db51640 1 -- 172.21.15.16:0/2903013689 shutdown_connections 2023-12-08T08:39:09.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.824+0000 7f755db51640 1 --2- 172.21.15.16:0/2903013689 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f754406ae40 0x7f754406d300 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.824+0000 7f755db51640 1 --2- 172.21.15.16:0/2903013689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75580d1d00 0x7f75580b0120 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.824+0000 7f755db51640 1 --2- 172.21.15.16:0/2903013689 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f75580b3bf0 0x7f75580afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.824+0000 7f755db51640 1 --2- 172.21.15.16:0/2903013689 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f75580b3340 0x7f75580a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.824+0000 7f755db51640 1 -- 172.21.15.16:0/2903013689 >> 172.21.15.16:0/2903013689 conn(0x7f755809c2f0 msgr2=0x7f75580b5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.824+0000 7f755db51640 1 -- 172.21.15.16:0/2903013689 shutdown_connections 2023-12-08T08:39:09.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.824+0000 7f755db51640 1 -- 172.21.15.16:0/2903013689 wait complete. 2023-12-08T08:39:09.840 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-08T08:39:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.954+0000 7f6f4da44640 1 Processor -- start 2023-12-08T08:39:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.954+0000 7f6f4da44640 1 -- start start 2023-12-08T08:39:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.955+0000 7f6f4da44640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f4806c7c0 0x7f6f480cb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.955+0000 7f6f4da44640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f480c9660 0x7f6f4806cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.955+0000 7f6f4da44640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f480705b0 con 0x7f6f4806d0e0 2023-12-08T08:39:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.955+0000 7f6f4da44640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f4815b480 con 0x7f6f480c9660 2023-12-08T08:39:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.955+0000 7f6f4da44640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480ac6f0 con 0x7f6f4806c7c0 2023-12-08T08:39:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.955+0000 7f6f467fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f480c9660 0x7f6f4806cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.955+0000 7f6f467fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f480c9660 0x7f6f4806cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35016/0 (socket says 172.21.15.16:35016) 2023-12-08T08:39:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.955+0000 7f6f467fc640 1 -- 172.21.15.16:0/664305921 learned_addr learned my addr 172.21.15.16:0/664305921 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.955+0000 7f6f46ffd640 1 --2- 172.21.15.16:0/664305921 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f4806c7c0 0x7f6f480cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.955+0000 7f6f467fc640 1 -- 172.21.15.16:0/664305921 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f4806c7c0 msgr2=0x7f6f480cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.955+0000 7f6f467fc640 1 --2- 172.21.15.16:0/664305921 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f4806c7c0 0x7f6f480cb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.955+0000 7f6f467fc640 1 -- 172.21.15.16:0/664305921 >> v1:172.21.15.112:6789/0 conn(0x7f6f4806d0e0 legacy=0x7f6f480acb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.955+0000 7f6f467fc640 1 -- 172.21.15.16:0/664305921 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480cb680 con 0x7f6f480c9660 2023-12-08T08:39:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.955+0000 7f6f46ffd640 1 --2- 172.21.15.16:0/664305921 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f4806c7c0 0x7f6f480cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.956+0000 7f6f467fc640 1 --2- 172.21.15.16:0/664305921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f480c9660 0x7f6f4806cba0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009870 tx=0x7f6f3c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12a4e9e5c799ed4 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.956+0000 7f6f45ffb640 1 -- 172.21.15.16:0/664305921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c030840 con 0x7f6f480c9660 2023-12-08T08:39:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.956+0000 7f6f45ffb640 1 -- 172.21.15.16:0/664305921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c0309a0 con 0x7f6f480c9660 2023-12-08T08:39:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.956+0000 7f6f45ffb640 1 -- 172.21.15.16:0/664305921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c030d30 con 0x7f6f480c9660 2023-12-08T08:39:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.957+0000 7f6f4da44640 1 -- 172.21.15.16:0/664305921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f480c9660 msgr2=0x7f6f4806cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.957+0000 7f6f4da44640 1 --2- 172.21.15.16:0/664305921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f480c9660 0x7f6f4806cba0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009870 tx=0x7f6f3c02b980 comp rx=0 tx=0).stop 2023-12-08T08:39:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.957+0000 7f6f4da44640 1 -- 172.21.15.16:0/664305921 shutdown_connections 2023-12-08T08:39:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.957+0000 7f6f4da44640 1 --2- 172.21.15.16:0/664305921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f480c9660 0x7f6f4806cba0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.957+0000 7f6f4da44640 1 --2- 172.21.15.16:0/664305921 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f4806c7c0 0x7f6f480cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.957+0000 7f6f4da44640 1 -- 172.21.15.16:0/664305921 >> 172.21.15.16:0/664305921 conn(0x7f6f480cf010 msgr2=0x7f6f480cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.957+0000 7f6f4da44640 1 -- 172.21.15.16:0/664305921 shutdown_connections 2023-12-08T08:39:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.957+0000 7f6f4da44640 1 -- 172.21.15.16:0/664305921 wait complete. 2023-12-08T08:39:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.958+0000 7f6f4da44640 1 Processor -- start 2023-12-08T08:39:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.958+0000 7f6f4da44640 1 -- start start 2023-12-08T08:39:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.959+0000 7f6f4da44640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f4806c7c0 0x7f6f480b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.959+0000 7f6f4da44640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f4806d0e0 0x7f6f480c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.959+0000 7f6f4da44640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f480c9660 0x7f6f480b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.959+0000 7f6f4da44640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480c5ec0 con 0x7f6f4806c7c0 2023-12-08T08:39:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.959+0000 7f6f4da44640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480b2340 con 0x7f6f480c9660 2023-12-08T08:39:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.959+0000 7f6f4da44640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480b2620 con 0x7f6f4806d0e0 2023-12-08T08:39:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.959+0000 7f6f46ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f4806c7c0 0x7f6f480b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.959+0000 7f6f46ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f4806c7c0 0x7f6f480b1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35018/0 (socket says 172.21.15.16:35018) 2023-12-08T08:39:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.959+0000 7f6f46ffd640 1 -- 172.21.15.16:0/2542698032 learned_addr learned my addr 172.21.15.16:0/2542698032 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.959+0000 7f6f467fc640 1 --2- 172.21.15.16:0/2542698032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f4806d0e0 0x7f6f480c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.959+0000 7f6f477fe640 1 --2- 172.21.15.16:0/2542698032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f480c9660 0x7f6f480b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.960+0000 7f6f46ffd640 1 -- 172.21.15.16:0/2542698032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f4806d0e0 msgr2=0x7f6f480c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.960+0000 7f6f46ffd640 1 --2- 172.21.15.16:0/2542698032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f4806d0e0 0x7f6f480c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.960+0000 7f6f46ffd640 1 -- 172.21.15.16:0/2542698032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f480c9660 msgr2=0x7f6f480b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.960+0000 7f6f46ffd640 1 --2- 172.21.15.16:0/2542698032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f480c9660 0x7f6f480b33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.960+0000 7f6f46ffd640 1 -- 172.21.15.16:0/2542698032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c0094d0 con 0x7f6f4806c7c0 2023-12-08T08:39:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.960+0000 7f6f46ffd640 1 --2- 172.21.15.16:0/2542698032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f4806c7c0 0x7f6f480b1cf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800d7d0 tx=0x7f6f3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.960+0000 7f6f2ffff640 1 -- 172.21.15.16:0/2542698032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38004030 con 0x7f6f4806c7c0 2023-12-08T08:39:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.960+0000 7f6f2ffff640 1 -- 172.21.15.16:0/2542698032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f38004190 con 0x7f6f4806c7c0 2023-12-08T08:39:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.960+0000 7f6f4da44640 1 -- 172.21.15.16:0/2542698032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f480b2900 con 0x7f6f4806c7c0 2023-12-08T08:39:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.960+0000 7f6f2ffff640 1 -- 172.21.15.16:0/2542698032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38004520 con 0x7f6f4806c7c0 2023-12-08T08:39:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.960+0000 7f6f4da44640 1 -- 172.21.15.16:0/2542698032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f480b2ec0 con 0x7f6f4806c7c0 2023-12-08T08:39:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.961+0000 7f6f4da44640 1 -- 172.21.15.16:0/2542698032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f48067430 con 0x7f6f4806c7c0 2023-12-08T08:39:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.962+0000 7f6f2ffff640 1 -- 172.21.15.16:0/2542698032 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6f380096a0 con 0x7f6f4806c7c0 2023-12-08T08:39:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.962+0000 7f6f2ffff640 1 --2- 172.21.15.16:0/2542698032 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6f3406ae70 0x7f6f3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.962+0000 7f6f2ffff640 1 -- 172.21.15.16:0/2542698032 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f6f380899c0 con 0x7f6f4806c7c0 2023-12-08T08:39:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.962+0000 7f6f467fc640 1 --2- 172.21.15.16:0/2542698032 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6f3406ae70 0x7f6f3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.963+0000 7f6f467fc640 1 --2- 172.21.15.16:0/2542698032 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6f3406ae70 0x7f6f3406d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c036000 tx=0x7f6f3c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:09.966+0000 7f6f2ffff640 1 -- 172.21.15.16:0/2542698032 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f380547d0 con 0x7f6f4806c7c0 2023-12-08T08:39:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.142+0000 7f6f4da44640 1 -- 172.21.15.16:0/2542698032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6f480cebe0 con 0x7f6f4806c7c0 2023-12-08T08:39:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.145+0000 7f6f2ffff640 1 -- 172.21.15.16:0/2542698032 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6f38057df0 con 0x7f6f4806c7c0 2023-12-08T08:39:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.147+0000 7f6f4da44640 1 -- 172.21.15.16:0/2542698032 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6f3406ae70 msgr2=0x7f6f3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.147+0000 7f6f4da44640 1 --2- 172.21.15.16:0/2542698032 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6f3406ae70 0x7f6f3406d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c036000 tx=0x7f6f3c008030 comp rx=0 tx=0).stop 2023-12-08T08:39:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.147+0000 7f6f4da44640 1 -- 172.21.15.16:0/2542698032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f4806c7c0 msgr2=0x7f6f480b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.147+0000 7f6f4da44640 1 --2- 172.21.15.16:0/2542698032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f4806c7c0 0x7f6f480b1cf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800d7d0 tx=0x7f6f3800dc90 comp rx=0 tx=0).stop 2023-12-08T08:39:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.147+0000 7f6f4da44640 1 -- 172.21.15.16:0/2542698032 shutdown_connections 2023-12-08T08:39:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.147+0000 7f6f4da44640 1 --2- 172.21.15.16:0/2542698032 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6f3406ae70 0x7f6f3406d330 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.147+0000 7f6f4da44640 1 --2- 172.21.15.16:0/2542698032 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6f480c9660 0x7f6f480b33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.147+0000 7f6f4da44640 1 --2- 172.21.15.16:0/2542698032 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6f4806d0e0 0x7f6f480c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.147+0000 7f6f4da44640 1 --2- 172.21.15.16:0/2542698032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f4806c7c0 0x7f6f480b1cf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.147+0000 7f6f4da44640 1 -- 172.21.15.16:0/2542698032 >> 172.21.15.16:0/2542698032 conn(0x7f6f480cf010 msgr2=0x7f6f480ad410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.147+0000 7f6f4da44640 1 -- 172.21.15.16:0/2542698032 shutdown_connections 2023-12-08T08:39:10.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.147+0000 7f6f4da44640 1 -- 172.21.15.16:0/2542698032 wait complete. 2023-12-08T08:39:10.160 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-08T08:39:10.160 DEBUG:teuthology.orchestra.run.smithi016:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-08T08:39:10.202 DEBUG:teuthology.orchestra.run.smithi112:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-08T08:39:10.247 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-08T08:39:10.247 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:39:10.247 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T08:39:10.263 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:39:10.263 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T08:39:10.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:10.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:39:10.413 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:39:10.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.465+0000 7f64c319e640 1 Processor -- start 2023-12-08T08:39:10.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.465+0000 7f64c319e640 1 -- start start 2023-12-08T08:39:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.466+0000 7f64c319e640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64bc0ad2d0 0x7f64bc0aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.466+0000 7f64c319e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64bc06abe0 0x7f64bc0cb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.466+0000 7f64c319e640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64bc0ca3f0 con 0x7f64bc0af090 2023-12-08T08:39:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.466+0000 7f64c319e640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64bc0caf70 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.466+0000 7f64c319e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc0ad6b0 con 0x7f64bc06abe0 2023-12-08T08:39:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.466+0000 7f64c219c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64bc0ad2d0 0x7f64bc0aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.466+0000 7f64c199b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f64bc0af090 0x7f64bc06a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50654/0 (socket says 172.21.15.16:50654) 2023-12-08T08:39:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.466+0000 7f64c199b640 1 -- 172.21.15.16:0/764870735 learned_addr learned my addr 172.21.15.16:0/764870735 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.466+0000 7f64c299d640 1 --2- 172.21.15.16:0/764870735 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64bc06abe0 0x7f64bc0cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.467+0000 7f64c119a640 1 -- 172.21.15.16:0/764870735 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316118873 0 0) 0x7f64bc0ca3f0 con 0x7f64bc0af090 2023-12-08T08:39:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.467+0000 7f64c119a640 1 -- 172.21.15.16:0/764870735 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64a8003520 con 0x7f64bc0af090 2023-12-08T08:39:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.467+0000 7f64c219c640 1 -- 172.21.15.16:0/764870735 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64bc06abe0 msgr2=0x7f64bc0cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:10.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.467+0000 7f64c219c640 1 --2- 172.21.15.16:0/764870735 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64bc06abe0 0x7f64bc0cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.467+0000 7f64c219c640 1 -- 172.21.15.16:0/764870735 >> v1:172.21.15.16:6789/0 conn(0x7f64bc0af090 legacy=0x7f64bc06a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:10.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.467+0000 7f64c219c640 1 -- 172.21.15.16:0/764870735 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc0ca640 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.467+0000 7f64c299d640 1 --2- 172.21.15.16:0/764870735 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64bc06abe0 0x7f64bc0cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.467+0000 7f64c219c640 1 --2- 172.21.15.16:0/764870735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc0ad2d0 0x7f64bc0aeb50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f64b80097d0 tx=0x7f64b802b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5a735e3b3413c server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.468+0000 7f64c119a640 1 -- 172.21.15.16:0/764870735 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8031770 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.468+0000 7f64c119a640 1 -- 172.21.15.16:0/764870735 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b80318d0 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.468+0000 7f64c119a640 1 -- 172.21.15.16:0/764870735 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8031c60 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.468+0000 7f64c319e640 1 -- 172.21.15.16:0/764870735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc0ad2d0 msgr2=0x7f64bc0aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.468+0000 7f64c319e640 1 --2- 172.21.15.16:0/764870735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc0ad2d0 0x7f64bc0aeb50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f64b80097d0 tx=0x7f64b802b810 comp rx=0 tx=0).stop 2023-12-08T08:39:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.468+0000 7f64c319e640 1 -- 172.21.15.16:0/764870735 shutdown_connections 2023-12-08T08:39:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.468+0000 7f64c319e640 1 --2- 172.21.15.16:0/764870735 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64bc06abe0 0x7f64bc0cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.468+0000 7f64c319e640 1 --2- 172.21.15.16:0/764870735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc0ad2d0 0x7f64bc0aeb50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.468+0000 7f64c319e640 1 -- 172.21.15.16:0/764870735 >> 172.21.15.16:0/764870735 conn(0x7f64bc0cfda0 msgr2=0x7f64bc0d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.468+0000 7f64c319e640 1 -- 172.21.15.16:0/764870735 shutdown_connections 2023-12-08T08:39:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.468+0000 7f64c319e640 1 -- 172.21.15.16:0/764870735 wait complete. 2023-12-08T08:39:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.470+0000 7f64c319e640 1 Processor -- start 2023-12-08T08:39:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.470+0000 7f64c319e640 1 -- start start 2023-12-08T08:39:10.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.470+0000 7f64c319e640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64bc06abe0 0x7f64bc15db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.470+0000 7f64c319e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc0ad2d0 0x7f64bc0c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.470+0000 7f64c319e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64bc0af090 0x7f64bc156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.470+0000 7f64c319e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc0c6ce0 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.470+0000 7f64c319e640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc0c78d0 con 0x7f64bc06abe0 2023-12-08T08:39:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.470+0000 7f64c319e640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc157430 con 0x7f64bc0af090 2023-12-08T08:39:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64c199b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc0ad2d0 0x7f64bc0c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64c199b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc0ad2d0 0x7f64bc0c67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35030/0 (socket says 172.21.15.16:35030) 2023-12-08T08:39:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64c199b640 1 -- 172.21.15.16:0/1815844179 learned_addr learned my addr 172.21.15.16:0/1815844179 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64c299d640 1 --2- 172.21.15.16:0/1815844179 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64bc0af090 0x7f64bc156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64c219c640 1 --2- 172.21.15.16:0/1815844179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64bc06abe0 0x7f64bc15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64c199b640 1 -- 172.21.15.16:0/1815844179 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64bc0af090 msgr2=0x7f64bc156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64c199b640 1 --2- 172.21.15.16:0/1815844179 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64bc0af090 0x7f64bc156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64c199b640 1 -- 172.21.15.16:0/1815844179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64bc06abe0 msgr2=0x7f64bc15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64c199b640 1 --2- 172.21.15.16:0/1815844179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64bc06abe0 0x7f64bc15db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64c199b640 1 -- 172.21.15.16:0/1815844179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8009400 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64c199b640 1 --2- 172.21.15.16:0/1815844179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc0ad2d0 0x7f64bc0c67a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00c500 tx=0x7f64ac00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64b2ffd640 1 -- 172.21.15.16:0/1815844179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac005ce0 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64b2ffd640 1 -- 172.21.15.16:0/1815844179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ac005e40 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64c319e640 1 -- 172.21.15.16:0/1815844179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64bc15e1d0 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64b2ffd640 1 -- 172.21.15.16:0/1815844179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac007330 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.471+0000 7f64c319e640 1 -- 172.21.15.16:0/1815844179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64bc0c7280 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.472+0000 7f64c319e640 1 -- 172.21.15.16:0/1815844179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64bc067ad0 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.473+0000 7f64b2ffd640 1 -- 172.21.15.16:0/1815844179 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f64ac007490 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.473+0000 7f64b2ffd640 1 --2- 172.21.15.16:0/1815844179 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f64a006ae70 0x7f64a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.473+0000 7f64b2ffd640 1 -- 172.21.15.16:0/1815844179 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f64ac088f30 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.473+0000 7f64c219c640 1 --2- 172.21.15.16:0/1815844179 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f64a006ae70 0x7f64a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.474+0000 7f64c219c640 1 --2- 172.21.15.16:0/1815844179 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f64a006ae70 0x7f64a006d330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f64b8036000 tx=0x7f64b803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.477+0000 7f64b2ffd640 1 -- 172.21.15.16:0/1815844179 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f64ac053e00 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.653+0000 7f64c319e640 1 -- 172.21.15.16:0/1815844179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64bc06f010 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.654+0000 7f64b2ffd640 1 -- 172.21.15.16:0/1815844179 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7f64ac017090 con 0x7f64bc0ad2d0 2023-12-08T08:39:10.656 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:10.656 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:10.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.656+0000 7f64c319e640 1 -- 172.21.15.16:0/1815844179 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f64a006ae70 msgr2=0x7f64a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:10.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.656+0000 7f64c319e640 1 --2- 172.21.15.16:0/1815844179 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f64a006ae70 0x7f64a006d330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f64b8036000 tx=0x7f64b803d040 comp rx=0 tx=0).stop 2023-12-08T08:39:10.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.656+0000 7f64c319e640 1 -- 172.21.15.16:0/1815844179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc0ad2d0 msgr2=0x7f64bc0c67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:10.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.656+0000 7f64c319e640 1 --2- 172.21.15.16:0/1815844179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc0ad2d0 0x7f64bc0c67a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00c500 tx=0x7f64ac00c9c0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.656+0000 7f64c319e640 1 -- 172.21.15.16:0/1815844179 shutdown_connections 2023-12-08T08:39:10.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.656+0000 7f64c319e640 1 --2- 172.21.15.16:0/1815844179 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f64a006ae70 0x7f64a006d330 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.656+0000 7f64c319e640 1 --2- 172.21.15.16:0/1815844179 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f64bc0af090 0x7f64bc156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.656+0000 7f64c319e640 1 --2- 172.21.15.16:0/1815844179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc0ad2d0 0x7f64bc0c67a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.656+0000 7f64c319e640 1 --2- 172.21.15.16:0/1815844179 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f64bc06abe0 0x7f64bc15db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.656+0000 7f64c319e640 1 -- 172.21.15.16:0/1815844179 >> 172.21.15.16:0/1815844179 conn(0x7f64bc0cfda0 msgr2=0x7f64bc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:10.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.656+0000 7f64c319e640 1 -- 172.21.15.16:0/1815844179 shutdown_connections 2023-12-08T08:39:10.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.656+0000 7f64c319e640 1 -- 172.21.15.16:0/1815844179 wait complete. 2023-12-08T08:39:10.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:39:10.671 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:10.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.819+0000 7f97f16d3640 1 Processor -- start 2023-12-08T08:39:10.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.819+0000 7f97f16d3640 1 -- start start 2023-12-08T08:39:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.819+0000 7f97f16d3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f97ec165db0 0x7f97ec1681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.819+0000 7f97f16d3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec0705b0 con 0x7f97ec0ce770 2023-12-08T08:39:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.819+0000 7f97f16d3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ec15d640 con 0x7f97ec06c7c0 2023-12-08T08:39:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.819+0000 7f97f16d3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ec0ae940 con 0x7f97ec165db0 2023-12-08T08:39:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.820+0000 7f97eb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f97ec165db0 0x7f97ec1681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:10.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.820+0000 7f97ea7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f97ec0ce770 0x7f97ec06cba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50666/0 (socket says 172.21.15.16:50666) 2023-12-08T08:39:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.820+0000 7f97ea7fc640 1 -- 172.21.15.16:0/3014246318 learned_addr learned my addr 172.21.15.16:0/3014246318 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.820+0000 7f97eb7fe640 1 -- 172.21.15.16:0/3014246318 >> v1:172.21.15.112:6789/0 conn(0x7f97ec06c7c0 legacy=0x7f97ec0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.820+0000 7f97eb7fe640 1 -- 172.21.15.16:0/3014246318 >> v1:172.21.15.16:6789/0 conn(0x7f97ec0ce770 legacy=0x7f97ec06cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:10.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.820+0000 7f97eb7fe640 1 -- 172.21.15.16:0/3014246318 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec0cec30 con 0x7f97ec165db0 2023-12-08T08:39:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.821+0000 7f97eb7fe640 1 --2- 172.21.15.16:0/3014246318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97ec165db0 0x7f97ec1681a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009d90 tx=0x7f97e402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64e4ce3a3fe62a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.821+0000 7f97e9ffb640 1 -- 172.21.15.16:0/3014246318 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e402cd80 con 0x7f97ec165db0 2023-12-08T08:39:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.821+0000 7f97e9ffb640 1 -- 172.21.15.16:0/3014246318 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e4034440 con 0x7f97ec165db0 2023-12-08T08:39:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.821+0000 7f97e9ffb640 1 -- 172.21.15.16:0/3014246318 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e40347d0 con 0x7f97ec165db0 2023-12-08T08:39:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.821+0000 7f97f16d3640 1 -- 172.21.15.16:0/3014246318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97ec165db0 msgr2=0x7f97ec1681a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.821+0000 7f97f16d3640 1 --2- 172.21.15.16:0/3014246318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97ec165db0 0x7f97ec1681a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009d90 tx=0x7f97e402bfd0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.821+0000 7f97f16d3640 1 -- 172.21.15.16:0/3014246318 shutdown_connections 2023-12-08T08:39:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.821+0000 7f97f16d3640 1 --2- 172.21.15.16:0/3014246318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97ec165db0 0x7f97ec1681a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.821+0000 7f97f16d3640 1 -- 172.21.15.16:0/3014246318 >> 172.21.15.16:0/3014246318 conn(0x7f97ec0cf010 msgr2=0x7f97ec0cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.821+0000 7f97f16d3640 1 -- 172.21.15.16:0/3014246318 shutdown_connections 2023-12-08T08:39:10.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.822+0000 7f97f16d3640 1 -- 172.21.15.16:0/3014246318 wait complete. 2023-12-08T08:39:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.823+0000 7f97f16d3640 1 Processor -- start 2023-12-08T08:39:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.823+0000 7f97f16d3640 1 -- start start 2023-12-08T08:39:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97f16d3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ec06c7c0 0x7f97ec0b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97f16d3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97ec0ce770 0x7f97ec15c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97f16d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97ec165db0 0x7f97ec150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97f16d3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0c4390 con 0x7f97ec165db0 2023-12-08T08:39:10.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97f16d3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0c4670 con 0x7f97ec06c7c0 2023-12-08T08:39:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97f16d3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec0c4950 con 0x7f97ec0ce770 2023-12-08T08:39:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97eb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97ec165db0 0x7f97ec150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97eb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97ec165db0 0x7f97ec150f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35042/0 (socket says 172.21.15.16:35042) 2023-12-08T08:39:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97eb7fe640 1 -- 172.21.15.16:0/2572465143 learned_addr learned my addr 172.21.15.16:0/2572465143 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97eaffd640 1 --2- 172.21.15.16:0/2572465143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ec06c7c0 0x7f97ec0b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97ea7fc640 1 --2- 172.21.15.16:0/2572465143 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97ec0ce770 0x7f97ec15c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97eb7fe640 1 -- 172.21.15.16:0/2572465143 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97ec0ce770 msgr2=0x7f97ec15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97eb7fe640 1 --2- 172.21.15.16:0/2572465143 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97ec0ce770 0x7f97ec15c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97eb7fe640 1 -- 172.21.15.16:0/2572465143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ec06c7c0 msgr2=0x7f97ec0b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97eb7fe640 1 --2- 172.21.15.16:0/2572465143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ec06c7c0 0x7f97ec0b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97eb7fe640 1 -- 172.21.15.16:0/2572465143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e4009a70 con 0x7f97ec165db0 2023-12-08T08:39:10.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97eb7fe640 1 --2- 172.21.15.16:0/2572465143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97ec165db0 0x7f97ec150f10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009d90 tx=0x7f97e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:10.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.824+0000 7f97d37fe640 1 -- 172.21.15.16:0/2572465143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4040650 con 0x7f97ec165db0 2023-12-08T08:39:10.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.825+0000 7f97d37fe640 1 -- 172.21.15.16:0/2572465143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e40407b0 con 0x7f97ec165db0 2023-12-08T08:39:10.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.825+0000 7f97f16d3640 1 -- 172.21.15.16:0/2572465143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ec0c4c30 con 0x7f97ec165db0 2023-12-08T08:39:10.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.825+0000 7f97d37fe640 1 -- 172.21.15.16:0/2572465143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4040b60 con 0x7f97ec165db0 2023-12-08T08:39:10.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.825+0000 7f97f16d3640 1 -- 172.21.15.16:0/2572465143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec159460 con 0x7f97ec165db0 2023-12-08T08:39:10.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.825+0000 7f97f16d3640 1 -- 172.21.15.16:0/2572465143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ec0ac340 con 0x7f97ec165db0 2023-12-08T08:39:10.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.826+0000 7f97d37fe640 1 -- 172.21.15.16:0/2572465143 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f97e4040cc0 con 0x7f97ec165db0 2023-12-08T08:39:10.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.826+0000 7f97d37fe640 1 --2- 172.21.15.16:0/2572465143 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f97d806ae70 0x7f97d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:10.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.826+0000 7f97d37fe640 1 -- 172.21.15.16:0/2572465143 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f97e40ac7f0 con 0x7f97ec165db0 2023-12-08T08:39:10.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.827+0000 7f97eaffd640 1 --2- 172.21.15.16:0/2572465143 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f97d806ae70 0x7f97d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:10.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.827+0000 7f97eaffd640 1 --2- 172.21.15.16:0/2572465143 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f97d806ae70 0x7f97d806d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f97dc000f60 tx=0x7f97dc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:10.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:10.831+0000 7f97d37fe640 1 -- 172.21.15.16:0/2572465143 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f97e4077620 con 0x7f97ec165db0 2023-12-08T08:39:11.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.005+0000 7f97f16d3640 1 -- 172.21.15.16:0/2572465143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97ec067430 con 0x7f97ec165db0 2023-12-08T08:39:11.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.006+0000 7f97d37fe640 1 -- 172.21.15.16:0/2572465143 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f97e407ac40 con 0x7f97ec165db0 2023-12-08T08:39:11.008 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.008+0000 7f97f16d3640 1 -- 172.21.15.16:0/2572465143 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f97d806ae70 msgr2=0x7f97d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.008+0000 7f97f16d3640 1 --2- 172.21.15.16:0/2572465143 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f97d806ae70 0x7f97d806d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f97dc000f60 tx=0x7f97dc009040 comp rx=0 tx=0).stop 2023-12-08T08:39:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.008+0000 7f97f16d3640 1 -- 172.21.15.16:0/2572465143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97ec165db0 msgr2=0x7f97ec150f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.008+0000 7f97f16d3640 1 --2- 172.21.15.16:0/2572465143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97ec165db0 0x7f97ec150f10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009d90 tx=0x7f97e4004a40 comp rx=0 tx=0).stop 2023-12-08T08:39:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.008+0000 7f97f16d3640 1 -- 172.21.15.16:0/2572465143 shutdown_connections 2023-12-08T08:39:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.008+0000 7f97f16d3640 1 --2- 172.21.15.16:0/2572465143 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f97d806ae70 0x7f97d806d330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.008+0000 7f97f16d3640 1 --2- 172.21.15.16:0/2572465143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97ec165db0 0x7f97ec150f10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.008+0000 7f97f16d3640 1 --2- 172.21.15.16:0/2572465143 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f97ec0ce770 0x7f97ec15c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.008+0000 7f97f16d3640 1 --2- 172.21.15.16:0/2572465143 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f97ec06c7c0 0x7f97ec0b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.008+0000 7f97f16d3640 1 -- 172.21.15.16:0/2572465143 >> 172.21.15.16:0/2572465143 conn(0x7f97ec0cf010 msgr2=0x7f97ec0ad650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:11.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.008+0000 7f97f16d3640 1 -- 172.21.15.16:0/2572465143 shutdown_connections 2023-12-08T08:39:11.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.008+0000 7f97f16d3640 1 -- 172.21.15.16:0/2572465143 wait complete. 2023-12-08T08:39:11.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:39:11.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.128+0000 7f88a3fde640 1 Processor -- start 2023-12-08T08:39:11.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.128+0000 7f88a3fde640 1 -- start start 2023-12-08T08:39:11.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.129+0000 7f88a3fde640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f889c0cf570 0x7f889c0cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.129+0000 7f88a3fde640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889c0cfef0 con 0x7f889c0b6c50 2023-12-08T08:39:11.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.129+0000 7f88a3fde640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889c0b5f00 con 0x7f889c0b8950 2023-12-08T08:39:11.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.129+0000 7f88a3fde640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f889c0b6170 con 0x7f889c0cf570 2023-12-08T08:39:11.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.129+0000 7f88a257b640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f889c0b6c50 0x7f889c0b70a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50668/0 (socket says 172.21.15.16:50668) 2023-12-08T08:39:11.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.129+0000 7f88a257b640 1 -- 172.21.15.16:0/1722169476 learned_addr learned my addr 172.21.15.16:0/1722169476 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:11.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.129+0000 7f88a1579640 1 --2- 172.21.15.16:0/1722169476 >> v2:172.21.15.112:3300/0 conn(0x7f889c0cf570 0x7f889c0cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.129+0000 7f88a0d78640 1 -- 172.21.15.16:0/1722169476 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236176284 0 0) 0x7f889c0cfef0 con 0x7f889c0b6c50 2023-12-08T08:39:11.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.129+0000 7f88a0d78640 1 -- 172.21.15.16:0/1722169476 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8888003520 con 0x7f889c0b6c50 2023-12-08T08:39:11.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.129+0000 7f88a1579640 1 -- 172.21.15.16:0/1722169476 >> v1:172.21.15.112:6789/0 conn(0x7f889c0b8950 legacy=0x7f889c0ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.129+0000 7f88a1579640 1 -- 172.21.15.16:0/1722169476 >> v1:172.21.15.16:6789/0 conn(0x7f889c0b6c50 legacy=0x7f889c0b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.129+0000 7f88a1579640 1 -- 172.21.15.16:0/1722169476 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889c0b6450 con 0x7f889c0cf570 2023-12-08T08:39:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.130+0000 7f88a1579640 1 --2- 172.21.15.16:0/1722169476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c0cf570 0x7f889c0cf9b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f888c0097a0 tx=0x7f888c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b4389efb11c6f77 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.130+0000 7f88a0d78640 1 -- 172.21.15.16:0/1722169476 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c031830 con 0x7f889c0cf570 2023-12-08T08:39:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.130+0000 7f88a0d78640 1 -- 172.21.15.16:0/1722169476 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c031990 con 0x7f889c0cf570 2023-12-08T08:39:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.130+0000 7f88a0d78640 1 -- 172.21.15.16:0/1722169476 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c031d20 con 0x7f889c0cf570 2023-12-08T08:39:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.131+0000 7f88a3fde640 1 -- 172.21.15.16:0/1722169476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c0cf570 msgr2=0x7f889c0cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.131+0000 7f88a3fde640 1 --2- 172.21.15.16:0/1722169476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c0cf570 0x7f889c0cf9b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f888c0097a0 tx=0x7f888c02b8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.131+0000 7f88a3fde640 1 -- 172.21.15.16:0/1722169476 shutdown_connections 2023-12-08T08:39:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.131+0000 7f88a3fde640 1 --2- 172.21.15.16:0/1722169476 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c0cf570 0x7f889c0cf9b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.131+0000 7f88a3fde640 1 -- 172.21.15.16:0/1722169476 >> 172.21.15.16:0/1722169476 conn(0x7f889c0d3010 msgr2=0x7f889c0d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:11.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.131+0000 7f88a3fde640 1 -- 172.21.15.16:0/1722169476 shutdown_connections 2023-12-08T08:39:11.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.131+0000 7f88a3fde640 1 -- 172.21.15.16:0/1722169476 wait complete. 2023-12-08T08:39:11.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.132+0000 7f88a3fde640 1 Processor -- start 2023-12-08T08:39:11.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.132+0000 7f88a3fde640 1 -- start start 2023-12-08T08:39:11.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a3fde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0b6c50 0x7f889c1f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a3fde640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c0b8950 0x7f889c0c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a3fde640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f889c0cf570 0x7f889c155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a3fde640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0cc730 con 0x7f889c0b6c50 2023-12-08T08:39:11.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a3fde640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0c6ce0 con 0x7f889c0b8950 2023-12-08T08:39:11.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a3fde640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0c6fc0 con 0x7f889c0cf570 2023-12-08T08:39:11.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a1d7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0b6c50 0x7f889c1f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a1d7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0b6c50 0x7f889c1f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35048/0 (socket says 172.21.15.16:35048) 2023-12-08T08:39:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a1d7a640 1 -- 172.21.15.16:0/496881162 learned_addr learned my addr 172.21.15.16:0/496881162 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a1579640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c0b8950 0x7f889c0c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a257b640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f889c0cf570 0x7f889c155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a1d7a640 1 -- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f889c0cf570 msgr2=0x7f889c155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a1d7a640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f889c0cf570 0x7f889c155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a1d7a640 1 -- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c0b8950 msgr2=0x7f889c0c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a1d7a640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c0b8950 0x7f889c0c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a1d7a640 1 -- 172.21.15.16:0/496881162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c009400 con 0x7f889c0b6c50 2023-12-08T08:39:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a257b640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f889c0cf570 0x7f889c155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.133+0000 7f88a1d7a640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0b6c50 0x7f889c1f4140 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f889800c450 tx=0x7f889800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.134+0000 7f88927fc640 1 -- 172.21.15.16:0/496881162 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88980052d0 con 0x7f889c0b6c50 2023-12-08T08:39:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.134+0000 7f88927fc640 1 -- 172.21.15.16:0/496881162 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8898005430 con 0x7f889c0b6c50 2023-12-08T08:39:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.134+0000 7f88a3fde640 1 -- 172.21.15.16:0/496881162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889c0c72a0 con 0x7f889c0b6c50 2023-12-08T08:39:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.134+0000 7f88927fc640 1 -- 172.21.15.16:0/496881162 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8898012870 con 0x7f889c0b6c50 2023-12-08T08:39:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.134+0000 7f88a3fde640 1 -- 172.21.15.16:0/496881162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889c1f4680 con 0x7f889c0b6c50 2023-12-08T08:39:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.135+0000 7f88a3fde640 1 -- 172.21.15.16:0/496881162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f889c0c77c0 con 0x7f889c0b6c50 2023-12-08T08:39:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.135+0000 7f88927fc640 1 -- 172.21.15.16:0/496881162 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f88980129d0 con 0x7f889c0b6c50 2023-12-08T08:39:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.136+0000 7f88927fc640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f888006ae40 0x7f888006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.136+0000 7f88927fc640 1 -- 172.21.15.16:0/496881162 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f8898089460 con 0x7f889c0b6c50 2023-12-08T08:39:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.136+0000 7f88a1579640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f888006ae40 0x7f888006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.136+0000 7f88a1579640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f888006ae40 0x7f888006d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f888c036000 tx=0x7f888c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:11.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.140+0000 7f88927fc640 1 -- 172.21.15.16:0/496881162 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f889c0c77c0 con 0x7f889c0b6c50 2023-12-08T08:39:11.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.314+0000 7f88a3fde640 1 -- 172.21.15.16:0/496881162 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f889c0c77c0 con 0x7f889c0b6c50 2023-12-08T08:39:11.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.315+0000 7f88927fc640 1 -- 172.21.15.16:0/496881162 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f889c0c77c0 con 0x7f889c0b6c50 2023-12-08T08:39:11.317 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.316+0000 7f88a3fde640 1 -- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f888006ae40 msgr2=0x7f888006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.316+0000 7f88a3fde640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f888006ae40 0x7f888006d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f888c036000 tx=0x7f888c03c040 comp rx=0 tx=0).stop 2023-12-08T08:39:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.316+0000 7f88a3fde640 1 -- 172.21.15.16:0/496881162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0b6c50 msgr2=0x7f889c1f4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.316+0000 7f88a3fde640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0b6c50 0x7f889c1f4140 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f889800c450 tx=0x7f889800c910 comp rx=0 tx=0).stop 2023-12-08T08:39:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.316+0000 7f88a3fde640 1 -- 172.21.15.16:0/496881162 shutdown_connections 2023-12-08T08:39:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.316+0000 7f88a3fde640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f888006ae40 0x7f888006d300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.316+0000 7f88a3fde640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f889c0cf570 0x7f889c155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.316+0000 7f88a3fde640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f889c0b8950 0x7f889c0c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.316+0000 7f88a3fde640 1 --2- 172.21.15.16:0/496881162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f889c0b6c50 0x7f889c1f4140 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.316+0000 7f88a3fde640 1 -- 172.21.15.16:0/496881162 >> 172.21.15.16:0/496881162 conn(0x7f889c0d3010 msgr2=0x7f889c0b5250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.317+0000 7f88a3fde640 1 -- 172.21.15.16:0/496881162 shutdown_connections 2023-12-08T08:39:11.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.317+0000 7f88a3fde640 1 -- 172.21.15.16:0/496881162 wait complete. 2023-12-08T08:39:11.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:39:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.438+0000 7fdc31dc9640 1 Processor -- start 2023-12-08T08:39:11.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.438+0000 7fdc31dc9640 1 -- start start 2023-12-08T08:39:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.438+0000 7fdc31dc9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc2c0b4150 0x7fdc2c0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.438+0000 7fdc31dc9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fdc2c0cfe10 0x7fdc2c0cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.438+0000 7fdc31dc9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2c0c6be0 con 0x7fdc2c0cfa30 2023-12-08T08:39:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.438+0000 7fdc31dc9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c0d3ff0 con 0x7fdc2c0cfe10 2023-12-08T08:39:11.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.438+0000 7fdc31dc9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0b4860 con 0x7fdc2c0b4150 2023-12-08T08:39:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.439+0000 7fdc2b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdc2c0cfa30 0x7fdc2c0b2f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50672/0 (socket says 172.21.15.16:50672) 2023-12-08T08:39:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.439+0000 7fdc2b7fe640 1 -- 172.21.15.16:0/2828513068 learned_addr learned my addr 172.21.15.16:0/2828513068 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.439+0000 7fdc2affd640 1 --2- 172.21.15.16:0/2828513068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc2c0b4150 0x7fdc2c0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.439+0000 7fdc2bfff640 1 --2- 172.21.15.16:0/2828513068 >> v2:172.21.15.112:3300/0 conn(0x7fdc2c0cfe10 0x7fdc2c0cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.439+0000 7fdc2a7fc640 1 -- 172.21.15.16:0/2828513068 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3579935420 0 0) 0x7fdc2c0c6be0 con 0x7fdc2c0cfa30 2023-12-08T08:39:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.439+0000 7fdc2a7fc640 1 -- 172.21.15.16:0/2828513068 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc14003520 con 0x7fdc2c0cfa30 2023-12-08T08:39:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.439+0000 7fdc2bfff640 1 -- 172.21.15.16:0/2828513068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc2c0b4150 msgr2=0x7fdc2c0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.439+0000 7fdc2bfff640 1 --2- 172.21.15.16:0/2828513068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc2c0b4150 0x7fdc2c0b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.439+0000 7fdc2bfff640 1 -- 172.21.15.16:0/2828513068 >> v1:172.21.15.16:6789/0 conn(0x7fdc2c0cfa30 legacy=0x7fdc2c0b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.439+0000 7fdc2bfff640 1 -- 172.21.15.16:0/2828513068 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c0d3d50 con 0x7fdc2c0cfe10 2023-12-08T08:39:11.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.439+0000 7fdc2affd640 1 --2- 172.21.15.16:0/2828513068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc2c0b4150 0x7fdc2c0b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:39:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.440+0000 7fdc2bfff640 1 --2- 172.21.15.16:0/2828513068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c0cfe10 0x7fdc2c0cdf50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009d90 tx=0x7fdc2402be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13e1009f625c22a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.440+0000 7fdc2a7fc640 1 -- 172.21.15.16:0/2828513068 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2402cbc0 con 0x7fdc2c0cfe10 2023-12-08T08:39:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.440+0000 7fdc2a7fc640 1 -- 172.21.15.16:0/2828513068 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2402cd20 con 0x7fdc2c0cfe10 2023-12-08T08:39:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.440+0000 7fdc2a7fc640 1 -- 172.21.15.16:0/2828513068 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24034640 con 0x7fdc2c0cfe10 2023-12-08T08:39:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.441+0000 7fdc31dc9640 1 -- 172.21.15.16:0/2828513068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c0cfe10 msgr2=0x7fdc2c0cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.441+0000 7fdc31dc9640 1 --2- 172.21.15.16:0/2828513068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c0cfe10 0x7fdc2c0cdf50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009d90 tx=0x7fdc2402be10 comp rx=0 tx=0).stop 2023-12-08T08:39:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.441+0000 7fdc31dc9640 1 -- 172.21.15.16:0/2828513068 shutdown_connections 2023-12-08T08:39:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.441+0000 7fdc31dc9640 1 --2- 172.21.15.16:0/2828513068 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c0cfe10 0x7fdc2c0cdf50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.441+0000 7fdc31dc9640 1 --2- 172.21.15.16:0/2828513068 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc2c0b4150 0x7fdc2c0b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.441+0000 7fdc31dc9640 1 -- 172.21.15.16:0/2828513068 >> 172.21.15.16:0/2828513068 conn(0x7fdc2c0d3090 msgr2=0x7fdc2c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:11.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.441+0000 7fdc31dc9640 1 -- 172.21.15.16:0/2828513068 shutdown_connections 2023-12-08T08:39:11.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.441+0000 7fdc31dc9640 1 -- 172.21.15.16:0/2828513068 wait complete. 2023-12-08T08:39:11.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.442+0000 7fdc31dc9640 1 Processor -- start 2023-12-08T08:39:11.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.442+0000 7fdc31dc9640 1 -- start start 2023-12-08T08:39:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc31dc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc2c0b4150 0x7fdc2c0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc31dc9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc2c0cfa30 0x7fdc2c1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc31dc9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c0cfe10 0x7fdc2c0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc31dc9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c154df0 con 0x7fdc2c0b4150 2023-12-08T08:39:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc31dc9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0ca990 con 0x7fdc2c0cfe10 2023-12-08T08:39:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc31dc9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0cac70 con 0x7fdc2c0cfa30 2023-12-08T08:39:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc2b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc2c0b4150 0x7fdc2c0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc2b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc2c0b4150 0x7fdc2c0cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35064/0 (socket says 172.21.15.16:35064) 2023-12-08T08:39:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc2b7fe640 1 -- 172.21.15.16:0/3891639720 learned_addr learned my addr 172.21.15.16:0/3891639720 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc2affd640 1 --2- 172.21.15.16:0/3891639720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc2c0cfa30 0x7fdc2c1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc2bfff640 1 --2- 172.21.15.16:0/3891639720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c0cfe10 0x7fdc2c0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc2b7fe640 1 -- 172.21.15.16:0/3891639720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc2c0cfa30 msgr2=0x7fdc2c1548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc2b7fe640 1 --2- 172.21.15.16:0/3891639720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc2c0cfa30 0x7fdc2c1548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc2b7fe640 1 -- 172.21.15.16:0/3891639720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c0cfe10 msgr2=0x7fdc2c0ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc2b7fe640 1 --2- 172.21.15.16:0/3891639720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c0cfe10 0x7fdc2c0ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc2b7fe640 1 -- 172.21.15.16:0/3891639720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc20009400 con 0x7fdc2c0b4150 2023-12-08T08:39:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc2b7fe640 1 --2- 172.21.15.16:0/3891639720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc2c0b4150 0x7fdc2c0cba30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00c390 tx=0x7fdc1c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc13fff640 1 -- 172.21.15.16:0/3891639720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c005ce0 con 0x7fdc2c0b4150 2023-12-08T08:39:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc13fff640 1 -- 172.21.15.16:0/3891639720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c005e40 con 0x7fdc2c0b4150 2023-12-08T08:39:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc31dc9640 1 -- 172.21.15.16:0/3891639720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc24009a70 con 0x7fdc2c0b4150 2023-12-08T08:39:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.443+0000 7fdc13fff640 1 -- 172.21.15.16:0/3891639720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0072f0 con 0x7fdc2c0b4150 2023-12-08T08:39:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.444+0000 7fdc31dc9640 1 -- 172.21.15.16:0/3891639720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c0cb230 con 0x7fdc2c0b4150 2023-12-08T08:39:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.444+0000 7fdc31dc9640 1 -- 172.21.15.16:0/3891639720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2c067430 con 0x7fdc2c0b4150 2023-12-08T08:39:11.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.445+0000 7fdc13fff640 1 -- 172.21.15.16:0/3891639720 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc1c007450 con 0x7fdc2c0b4150 2023-12-08T08:39:11.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.445+0000 7fdc13fff640 1 --2- 172.21.15.16:0/3891639720 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdc1806ae40 0x7fdc1806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.445+0000 7fdc13fff640 1 -- 172.21.15.16:0/3891639720 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fdc1c088fc0 con 0x7fdc2c0b4150 2023-12-08T08:39:11.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.445+0000 7fdc2affd640 1 --2- 172.21.15.16:0/3891639720 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdc1806ae40 0x7fdc1806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.446+0000 7fdc2affd640 1 --2- 172.21.15.16:0/3891639720 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdc1806ae40 0x7fdc1806d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c15c430 tx=0x7fdc20009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.449+0000 7fdc13fff640 1 -- 172.21.15.16:0/3891639720 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc1c053cb0 con 0x7fdc2c0b4150 2023-12-08T08:39:11.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.625+0000 7fdc31dc9640 1 -- 172.21.15.16:0/3891639720 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc2c0743e0 con 0x7fdc2c0b4150 2023-12-08T08:39:11.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.625+0000 7fdc13fff640 1 -- 172.21.15.16:0/3891639720 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc1c0572d0 con 0x7fdc2c0b4150 2023-12-08T08:39:11.627 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:39:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.627+0000 7fdc31dc9640 1 -- 172.21.15.16:0/3891639720 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdc1806ae40 msgr2=0x7fdc1806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.627+0000 7fdc31dc9640 1 --2- 172.21.15.16:0/3891639720 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdc1806ae40 0x7fdc1806d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c15c430 tx=0x7fdc20009190 comp rx=0 tx=0).stop 2023-12-08T08:39:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.627+0000 7fdc31dc9640 1 -- 172.21.15.16:0/3891639720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc2c0b4150 msgr2=0x7fdc2c0cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.627+0000 7fdc31dc9640 1 --2- 172.21.15.16:0/3891639720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc2c0b4150 0x7fdc2c0cba30 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00c390 tx=0x7fdc1c00c850 comp rx=0 tx=0).stop 2023-12-08T08:39:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.627+0000 7fdc31dc9640 1 -- 172.21.15.16:0/3891639720 shutdown_connections 2023-12-08T08:39:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.627+0000 7fdc31dc9640 1 --2- 172.21.15.16:0/3891639720 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdc1806ae40 0x7fdc1806d300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.627+0000 7fdc31dc9640 1 --2- 172.21.15.16:0/3891639720 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdc2c0cfe10 0x7fdc2c0ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.627+0000 7fdc31dc9640 1 --2- 172.21.15.16:0/3891639720 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdc2c0cfa30 0x7fdc2c1548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.627+0000 7fdc31dc9640 1 --2- 172.21.15.16:0/3891639720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc2c0b4150 0x7fdc2c0cba30 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.627+0000 7fdc31dc9640 1 -- 172.21.15.16:0/3891639720 >> 172.21.15.16:0/3891639720 conn(0x7fdc2c0d3090 msgr2=0x7fdc2c155690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.627+0000 7fdc31dc9640 1 -- 172.21.15.16:0/3891639720 shutdown_connections 2023-12-08T08:39:11.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.627+0000 7fdc31dc9640 1 -- 172.21.15.16:0/3891639720 wait complete. 2023-12-08T08:39:11.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:39:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.749+0000 7fa397ddc640 1 Processor -- start 2023-12-08T08:39:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.749+0000 7fa397ddc640 1 -- start start 2023-12-08T08:39:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.750+0000 7fa397ddc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa3900afeb0 0x7fa3900cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.750+0000 7fa397ddc640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa39006d650 con 0x7fa3900ae350 2023-12-08T08:39:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.750+0000 7fa397ddc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3900adbe0 con 0x7fa39015d590 2023-12-08T08:39:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.750+0000 7fa397ddc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39015d970 con 0x7fa3900afeb0 2023-12-08T08:39:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.750+0000 7fa395377640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa3900afeb0 0x7fa3900cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.750+0000 7fa396379640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa3900ae350 0x7fa3900ae7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50678/0 (socket says 172.21.15.16:50678) 2023-12-08T08:39:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.750+0000 7fa396379640 1 -- 172.21.15.16:0/3438405277 learned_addr learned my addr 172.21.15.16:0/3438405277 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.750+0000 7fa395377640 1 -- 172.21.15.16:0/3438405277 >> v1:172.21.15.112:6789/0 conn(0x7fa39015d590 legacy=0x7fa3900ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.750+0000 7fa395377640 1 -- 172.21.15.16:0/3438405277 >> v1:172.21.15.16:6789/0 conn(0x7fa3900ae350 legacy=0x7fa3900ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.750+0000 7fa394b76640 1 -- 172.21.15.16:0/3438405277 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436716831 0 0) 0x7fa388001490 con 0x7fa3900ae350 2023-12-08T08:39:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.750+0000 7fa395377640 1 -- 172.21.15.16:0/3438405277 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39006c810 con 0x7fa3900afeb0 2023-12-08T08:39:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.751+0000 7fa395377640 1 --2- 172.21.15.16:0/3438405277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3900afeb0 0x7fa3900cf420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa3800097a0 tx=0x7fa38002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d92219c53a915 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.751+0000 7fa394b76640 1 -- 172.21.15.16:0/3438405277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380031830 con 0x7fa3900afeb0 2023-12-08T08:39:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.751+0000 7fa394b76640 1 -- 172.21.15.16:0/3438405277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa380031990 con 0x7fa3900afeb0 2023-12-08T08:39:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.751+0000 7fa394b76640 1 -- 172.21.15.16:0/3438405277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380031d20 con 0x7fa3900afeb0 2023-12-08T08:39:11.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.751+0000 7fa397ddc640 1 -- 172.21.15.16:0/3438405277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3900afeb0 msgr2=0x7fa3900cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.751+0000 7fa397ddc640 1 --2- 172.21.15.16:0/3438405277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3900afeb0 0x7fa3900cf420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa3800097a0 tx=0x7fa38002b8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.752+0000 7fa397ddc640 1 -- 172.21.15.16:0/3438405277 shutdown_connections 2023-12-08T08:39:11.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.752+0000 7fa397ddc640 1 --2- 172.21.15.16:0/3438405277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3900afeb0 0x7fa3900cf420 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.752+0000 7fa397ddc640 1 -- 172.21.15.16:0/3438405277 >> 172.21.15.16:0/3438405277 conn(0x7fa3900d0e10 msgr2=0x7fa3900d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:11.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.752+0000 7fa397ddc640 1 -- 172.21.15.16:0/3438405277 shutdown_connections 2023-12-08T08:39:11.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.752+0000 7fa397ddc640 1 -- 172.21.15.16:0/3438405277 wait complete. 2023-12-08T08:39:11.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.753+0000 7fa397ddc640 1 Processor -- start 2023-12-08T08:39:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.753+0000 7fa397ddc640 1 -- start start 2023-12-08T08:39:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa397ddc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3900ae350 0x7fa3900cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa397ddc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3900afeb0 0x7fa3900ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa397ddc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa39015d590 0x7fa39014f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa397ddc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa390149f80 con 0x7fa3900ae350 2023-12-08T08:39:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa397ddc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900ca8e0 con 0x7fa3900afeb0 2023-12-08T08:39:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa397ddc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900ca600 con 0x7fa39015d590 2023-12-08T08:39:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa395b78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3900ae350 0x7fa3900cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa395b78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3900ae350 0x7fa3900cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35090/0 (socket says 172.21.15.16:35090) 2023-12-08T08:39:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa395b78640 1 -- 172.21.15.16:0/1783716335 learned_addr learned my addr 172.21.15.16:0/1783716335 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa395377640 1 --2- 172.21.15.16:0/1783716335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3900afeb0 0x7fa3900ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa396379640 1 --2- 172.21.15.16:0/1783716335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa39015d590 0x7fa39014f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa395b78640 1 -- 172.21.15.16:0/1783716335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa39015d590 msgr2=0x7fa39014f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa395b78640 1 --2- 172.21.15.16:0/1783716335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa39015d590 0x7fa39014f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa395b78640 1 -- 172.21.15.16:0/1783716335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3900afeb0 msgr2=0x7fa3900ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa395b78640 1 --2- 172.21.15.16:0/1783716335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3900afeb0 0x7fa3900ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa395b78640 1 -- 172.21.15.16:0/1783716335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa380009400 con 0x7fa3900ae350 2023-12-08T08:39:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa395b78640 1 --2- 172.21.15.16:0/1783716335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3900ae350 0x7fa3900cddb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00c400 tx=0x7fa38c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa3867fc640 1 -- 172.21.15.16:0/1783716335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c0052d0 con 0x7fa3900ae350 2023-12-08T08:39:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa3867fc640 1 -- 172.21.15.16:0/1783716335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa38c005430 con 0x7fa3900ae350 2023-12-08T08:39:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa397ddc640 1 -- 172.21.15.16:0/1783716335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39014a320 con 0x7fa3900ae350 2023-12-08T08:39:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa3867fc640 1 -- 172.21.15.16:0/1783716335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38c003830 con 0x7fa3900ae350 2023-12-08T08:39:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.754+0000 7fa397ddc640 1 -- 172.21.15.16:0/1783716335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39014a8e0 con 0x7fa3900ae350 2023-12-08T08:39:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.755+0000 7fa397ddc640 1 -- 172.21.15.16:0/1783716335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa390067430 con 0x7fa3900ae350 2023-12-08T08:39:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.756+0000 7fa3867fc640 1 -- 172.21.15.16:0/1783716335 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa38c003990 con 0x7fa3900ae350 2023-12-08T08:39:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.756+0000 7fa3867fc640 1 --2- 172.21.15.16:0/1783716335 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa37406ae40 0x7fa37406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.756+0000 7fa3867fc640 1 -- 172.21.15.16:0/1783716335 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fa38c088ec0 con 0x7fa3900ae350 2023-12-08T08:39:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.756+0000 7fa395377640 1 --2- 172.21.15.16:0/1783716335 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa37406ae40 0x7fa37406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.757+0000 7fa395377640 1 --2- 172.21.15.16:0/1783716335 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa37406ae40 0x7fa37406d300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa380036000 tx=0x7fa38003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.761+0000 7fa3867fc640 1 -- 172.21.15.16:0/1783716335 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa38c053c40 con 0x7fa3900ae350 2023-12-08T08:39:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.938+0000 7fa397ddc640 1 -- 172.21.15.16:0/1783716335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa3900b6d90 con 0x7fa3900ae350 2023-12-08T08:39:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.939+0000 7fa3867fc640 1 -- 172.21.15.16:0/1783716335 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa38c0170a0 con 0x7fa3900ae350 2023-12-08T08:39:11.941 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.941+0000 7fa397ddc640 1 -- 172.21.15.16:0/1783716335 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa37406ae40 msgr2=0x7fa37406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.941+0000 7fa397ddc640 1 --2- 172.21.15.16:0/1783716335 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa37406ae40 0x7fa37406d300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa380036000 tx=0x7fa38003c040 comp rx=0 tx=0).stop 2023-12-08T08:39:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.941+0000 7fa397ddc640 1 -- 172.21.15.16:0/1783716335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3900ae350 msgr2=0x7fa3900cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.941+0000 7fa397ddc640 1 --2- 172.21.15.16:0/1783716335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3900ae350 0x7fa3900cddb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa38c00c400 tx=0x7fa38c00c8c0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.941+0000 7fa397ddc640 1 -- 172.21.15.16:0/1783716335 shutdown_connections 2023-12-08T08:39:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.941+0000 7fa397ddc640 1 --2- 172.21.15.16:0/1783716335 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa37406ae40 0x7fa37406d300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.941+0000 7fa397ddc640 1 --2- 172.21.15.16:0/1783716335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa39015d590 0x7fa39014f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.941+0000 7fa397ddc640 1 --2- 172.21.15.16:0/1783716335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa3900afeb0 0x7fa3900ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.941+0000 7fa397ddc640 1 --2- 172.21.15.16:0/1783716335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3900ae350 0x7fa3900cddb0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:11.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.941+0000 7fa397ddc640 1 -- 172.21.15.16:0/1783716335 >> 172.21.15.16:0/1783716335 conn(0x7fa3900d0e10 msgr2=0x7fa39015ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:11.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.941+0000 7fa397ddc640 1 -- 172.21.15.16:0/1783716335 shutdown_connections 2023-12-08T08:39:11.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:11.941+0000 7fa397ddc640 1 -- 172.21.15.16:0/1783716335 wait complete. 2023-12-08T08:39:11.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-08T08:39:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.059+0000 7f7e13b68640 1 Processor -- start 2023-12-08T08:39:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.059+0000 7f7e13b68640 1 -- start start 2023-12-08T08:39:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.060+0000 7f7e13b68640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e0c0ad290 0x7f7e0c06ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.060+0000 7f7e13b68640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0c06aee0 con 0x7f7e0c160ca0 2023-12-08T08:39:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.060+0000 7f7e13b68640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0c06bd40 con 0x7f7e0c0ad710 2023-12-08T08:39:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.060+0000 7f7e13b68640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0c06b130 con 0x7f7e0c0ad290 2023-12-08T08:39:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.060+0000 7f7e11103640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e0c0ad290 0x7f7e0c06ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.060+0000 7f7e11904640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7e0c160ca0 0x7f7e0c161080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50680/0 (socket says 172.21.15.16:50680) 2023-12-08T08:39:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.060+0000 7f7e11103640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e0c0ad290 0x7f7e0c06ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35102/0 (socket says 172.21.15.16:35102) 2023-12-08T08:39:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.060+0000 7f7e11904640 1 -- 172.21.15.16:0/1105330968 learned_addr learned my addr 172.21.15.16:0/1105330968 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.060+0000 7f7e11103640 1 -- 172.21.15.16:0/1105330968 >> v1:172.21.15.112:6789/0 conn(0x7f7e0c0ad710 legacy=0x7f7e0c06d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.060+0000 7f7e11103640 1 -- 172.21.15.16:0/1105330968 >> v1:172.21.15.16:6789/0 conn(0x7f7e0c160ca0 legacy=0x7f7e0c161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.060+0000 7f7e10902640 1 -- 172.21.15.16:0/1105330968 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1827877506 0 0) 0x7f7e0c06aee0 con 0x7f7e0c160ca0 2023-12-08T08:39:12.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.060+0000 7f7e11103640 1 -- 172.21.15.16:0/1105330968 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c06b410 con 0x7f7e0c0ad290 2023-12-08T08:39:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.061+0000 7f7e11103640 1 --2- 172.21.15.16:0/1105330968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0ad290 0x7f7e0c06ce10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0097a0 tx=0x7f7dfc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eac1cc2f07ac96 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.061+0000 7f7e10902640 1 -- 172.21.15.16:0/1105330968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc031830 con 0x7f7e0c0ad290 2023-12-08T08:39:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.061+0000 7f7e10902640 1 -- 172.21.15.16:0/1105330968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc031990 con 0x7f7e0c0ad290 2023-12-08T08:39:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.061+0000 7f7e10902640 1 -- 172.21.15.16:0/1105330968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc031d20 con 0x7f7e0c0ad290 2023-12-08T08:39:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.062+0000 7f7e13b68640 1 -- 172.21.15.16:0/1105330968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0ad290 msgr2=0x7f7e0c06ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.062+0000 7f7e13b68640 1 --2- 172.21.15.16:0/1105330968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0ad290 0x7f7e0c06ce10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0097a0 tx=0x7f7dfc02b8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.062+0000 7f7e13b68640 1 -- 172.21.15.16:0/1105330968 shutdown_connections 2023-12-08T08:39:12.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.062+0000 7f7e13b68640 1 --2- 172.21.15.16:0/1105330968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0ad290 0x7f7e0c06ce10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.062+0000 7f7e13b68640 1 -- 172.21.15.16:0/1105330968 >> 172.21.15.16:0/1105330968 conn(0x7f7e0c0cf470 msgr2=0x7f7e0c0cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.062+0000 7f7e13b68640 1 -- 172.21.15.16:0/1105330968 shutdown_connections 2023-12-08T08:39:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.062+0000 7f7e13b68640 1 -- 172.21.15.16:0/1105330968 wait complete. 2023-12-08T08:39:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.063+0000 7f7e13b68640 1 Processor -- start 2023-12-08T08:39:12.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.063+0000 7f7e13b68640 1 -- start start 2023-12-08T08:39:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e13b68640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e0c0ad290 0x7f7e0c0cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e13b68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0ad710 0x7f7e0c13ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e13b68640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e0c160ca0 0x7f7e0c14c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e13b68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c0cd740 con 0x7f7e0c0ad710 2023-12-08T08:39:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e13b68640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c14ffe0 con 0x7f7e0c160ca0 2023-12-08T08:39:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e13b68640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c14b500 con 0x7f7e0c0ad290 2023-12-08T08:39:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e11103640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0ad710 0x7f7e0c13ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e11103640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0ad710 0x7f7e0c13ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35116/0 (socket says 172.21.15.16:35116) 2023-12-08T08:39:12.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e11103640 1 -- 172.21.15.16:0/1447739056 learned_addr learned my addr 172.21.15.16:0/1447739056 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e11904640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e0c0ad290 0x7f7e0c0cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e12105640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e0c160ca0 0x7f7e0c14c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e11103640 1 -- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e0c0ad290 msgr2=0x7f7e0c0cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e11103640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e0c0ad290 0x7f7e0c0cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e11103640 1 -- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e0c160ca0 msgr2=0x7f7e0c14c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e11103640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e0c160ca0 0x7f7e0c14c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e11103640 1 -- 172.21.15.16:0/1447739056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc009400 con 0x7f7e0c0ad710 2023-12-08T08:39:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e11904640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e0c0ad290 0x7f7e0c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.064+0000 7f7e11103640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0ad710 0x7f7e0c13ecd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0098c0 tx=0x7f7dfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.065+0000 7f7e027fc640 1 -- 172.21.15.16:0/1447739056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc03ba70 con 0x7f7e0c0ad710 2023-12-08T08:39:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.065+0000 7f7e027fc640 1 -- 172.21.15.16:0/1447739056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc03bbf0 con 0x7f7e0c0ad710 2023-12-08T08:39:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.065+0000 7f7e13b68640 1 -- 172.21.15.16:0/1447739056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0c14b220 con 0x7f7e0c0ad710 2023-12-08T08:39:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.065+0000 7f7e027fc640 1 -- 172.21.15.16:0/1447739056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc0453f0 con 0x7f7e0c0ad710 2023-12-08T08:39:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.065+0000 7f7e13b68640 1 -- 172.21.15.16:0/1447739056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c14d5c0 con 0x7f7e0c0ad710 2023-12-08T08:39:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.066+0000 7f7e13b68640 1 -- 172.21.15.16:0/1447739056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c067430 con 0x7f7e0c0ad710 2023-12-08T08:39:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.066+0000 7f7e027fc640 1 -- 172.21.15.16:0/1447739056 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7dfc04e450 con 0x7f7e0c0ad710 2023-12-08T08:39:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.067+0000 7f7e027fc640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7df006ae70 0x7f7df006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.067+0000 7f7e027fc640 1 -- 172.21.15.16:0/1447739056 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f7dfc0abbd0 con 0x7f7e0c0ad710 2023-12-08T08:39:12.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.067+0000 7f7e11904640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7df006ae70 0x7f7df006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.067+0000 7f7e11904640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7df006ae70 0x7f7df006d330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7e08000f60 tx=0x7f7e08009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:12.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.071+0000 7f7e027fc640 1 -- 172.21.15.16:0/1447739056 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7dfc076a50 con 0x7f7e0c0ad710 2023-12-08T08:39:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.262+0000 7f7e13b68640 1 -- 172.21.15.16:0/1447739056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7e0c073ce0 con 0x7f7e0c0ad710 2023-12-08T08:39:12.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.263+0000 7f7e027fc640 1 -- 172.21.15.16:0/1447739056 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7dfc07a070 con 0x7f7e0c0ad710 2023-12-08T08:39:12.265 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-12-08T08:39:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.265+0000 7f7e13b68640 1 -- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7df006ae70 msgr2=0x7f7df006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.265+0000 7f7e13b68640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7df006ae70 0x7f7df006d330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7e08000f60 tx=0x7f7e08009040 comp rx=0 tx=0).stop 2023-12-08T08:39:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.265+0000 7f7e13b68640 1 -- 172.21.15.16:0/1447739056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0ad710 msgr2=0x7f7e0c13ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.265+0000 7f7e13b68640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0ad710 0x7f7e0c13ecd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0098c0 tx=0x7f7dfc0046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.265+0000 7f7e13b68640 1 -- 172.21.15.16:0/1447739056 shutdown_connections 2023-12-08T08:39:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.265+0000 7f7e13b68640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7df006ae70 0x7f7df006d330 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.265+0000 7f7e13b68640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7e0c160ca0 0x7f7e0c14c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.265+0000 7f7e13b68640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e0c0ad710 0x7f7e0c13ecd0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.265+0000 7f7e13b68640 1 --2- 172.21.15.16:0/1447739056 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7e0c0ad290 0x7f7e0c0cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.265+0000 7f7e13b68640 1 -- 172.21.15.16:0/1447739056 >> 172.21.15.16:0/1447739056 conn(0x7f7e0c0cf470 msgr2=0x7f7e0c0af190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:12.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.265+0000 7f7e13b68640 1 -- 172.21.15.16:0/1447739056 shutdown_connections 2023-12-08T08:39:12.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.265+0000 7f7e13b68640 1 -- 172.21.15.16:0/1447739056 wait complete. 2023-12-08T08:39:12.278 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:39:12.278 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T08:39:12.296 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:12.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.440+0000 7f5e34d89640 1 Processor -- start 2023-12-08T08:39:12.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.440+0000 7f5e34d89640 1 -- start start 2023-12-08T08:39:12.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.441+0000 7f5e34d89640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e300cbd10 0x7f5e30070e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.441+0000 7f5e34d89640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e300720d0 con 0x7f5e30158ee0 2023-12-08T08:39:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.441+0000 7f5e34d89640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e300cc890 con 0x7f5e3015b750 2023-12-08T08:39:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.441+0000 7f5e34d89640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e300714c0 con 0x7f5e300cbd10 2023-12-08T08:39:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.441+0000 7f5e27fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e300cbd10 0x7f5e30070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.441+0000 7f5e2e59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5e30158ee0 0x7f5e300cb600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50694/0 (socket says 172.21.15.16:50694) 2023-12-08T08:39:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.441+0000 7f5e2e59c640 1 -- 172.21.15.16:0/3648938488 learned_addr learned my addr 172.21.15.16:0/3648938488 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.441+0000 7f5e27fff640 1 -- 172.21.15.16:0/3648938488 >> v1:172.21.15.112:6789/0 conn(0x7f5e3015b750 legacy=0x7f5e3006c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.441+0000 7f5e27fff640 1 -- 172.21.15.16:0/3648938488 >> v1:172.21.15.16:6789/0 conn(0x7f5e30158ee0 legacy=0x7f5e300cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.441+0000 7f5e2dd9b640 1 -- 172.21.15.16:0/3648938488 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839990800 0 0) 0x7f5e300720d0 con 0x7f5e30158ee0 2023-12-08T08:39:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.441+0000 7f5e27fff640 1 -- 172.21.15.16:0/3648938488 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e300717a0 con 0x7f5e300cbd10 2023-12-08T08:39:12.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.442+0000 7f5e27fff640 1 --2- 172.21.15.16:0/3648938488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e300cbd10 0x7f5e30070e30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5e180097a0 tx=0x7f5e1802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8973ac856cba87a0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:12.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.442+0000 7f5e2dd9b640 1 -- 172.21.15.16:0/3648938488 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1802e070 con 0x7f5e300cbd10 2023-12-08T08:39:12.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.442+0000 7f5e2dd9b640 1 -- 172.21.15.16:0/3648938488 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e180315a0 con 0x7f5e300cbd10 2023-12-08T08:39:12.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.442+0000 7f5e2dd9b640 1 -- 172.21.15.16:0/3648938488 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e18031930 con 0x7f5e300cbd10 2023-12-08T08:39:12.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.443+0000 7f5e34d89640 1 -- 172.21.15.16:0/3648938488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e300cbd10 msgr2=0x7f5e30070e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.443+0000 7f5e34d89640 1 --2- 172.21.15.16:0/3648938488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e300cbd10 0x7f5e30070e30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5e180097a0 tx=0x7f5e1802b840 comp rx=0 tx=0).stop 2023-12-08T08:39:12.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.443+0000 7f5e34d89640 1 -- 172.21.15.16:0/3648938488 shutdown_connections 2023-12-08T08:39:12.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.443+0000 7f5e34d89640 1 --2- 172.21.15.16:0/3648938488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e300cbd10 0x7f5e30070e30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.443+0000 7f5e34d89640 1 -- 172.21.15.16:0/3648938488 >> 172.21.15.16:0/3648938488 conn(0x7f5e300b09f0 msgr2=0x7f5e300b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:12.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.443+0000 7f5e34d89640 1 -- 172.21.15.16:0/3648938488 shutdown_connections 2023-12-08T08:39:12.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.443+0000 7f5e34d89640 1 -- 172.21.15.16:0/3648938488 wait complete. 2023-12-08T08:39:12.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.444+0000 7f5e34d89640 1 Processor -- start 2023-12-08T08:39:12.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.444+0000 7f5e34d89640 1 -- start start 2023-12-08T08:39:12.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e34d89640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e300cbd10 0x7f5e30099c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e34d89640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e30158ee0 0x7f5e301536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e34d89640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e3015b750 0x7f5e30147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e34d89640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300ce840 con 0x7f5e300cbd10 2023-12-08T08:39:12.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e34d89640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300ceb20 con 0x7f5e3015b750 2023-12-08T08:39:12.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e34d89640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e300cee00 con 0x7f5e30158ee0 2023-12-08T08:39:12.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e2e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e300cbd10 0x7f5e30099c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e2e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e300cbd10 0x7f5e30099c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35140/0 (socket says 172.21.15.16:35140) 2023-12-08T08:39:12.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e2e59c640 1 -- 172.21.15.16:0/2889419793 learned_addr learned my addr 172.21.15.16:0/2889419793 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:12.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e2ed9d640 1 --2- 172.21.15.16:0/2889419793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e3015b750 0x7f5e30147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e27fff640 1 --2- 172.21.15.16:0/2889419793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e30158ee0 0x7f5e301536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e2e59c640 1 -- 172.21.15.16:0/2889419793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e30158ee0 msgr2=0x7f5e301536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e2e59c640 1 --2- 172.21.15.16:0/2889419793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e30158ee0 0x7f5e301536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e2e59c640 1 -- 172.21.15.16:0/2889419793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e3015b750 msgr2=0x7f5e30147560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e2e59c640 1 --2- 172.21.15.16:0/2889419793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e3015b750 0x7f5e30147560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.445+0000 7f5e2e59c640 1 -- 172.21.15.16:0/2889419793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e18009400 con 0x7f5e300cbd10 2023-12-08T08:39:12.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.446+0000 7f5e2e59c640 1 --2- 172.21.15.16:0/2889419793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e300cbd10 0x7f5e30099c40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000c4d0 tx=0x7f5e2000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:12.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.446+0000 7f5e26ffd640 1 -- 172.21.15.16:0/2889419793 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e200074e0 con 0x7f5e300cbd10 2023-12-08T08:39:12.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.446+0000 7f5e26ffd640 1 -- 172.21.15.16:0/2889419793 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e20007660 con 0x7f5e300cbd10 2023-12-08T08:39:12.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.446+0000 7f5e26ffd640 1 -- 172.21.15.16:0/2889419793 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20004a20 con 0x7f5e300cbd10 2023-12-08T08:39:12.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.446+0000 7f5e34d89640 1 -- 172.21.15.16:0/2889419793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e300cf0e0 con 0x7f5e300cbd10 2023-12-08T08:39:12.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.446+0000 7f5e34d89640 1 -- 172.21.15.16:0/2889419793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3009a180 con 0x7f5e300cbd10 2023-12-08T08:39:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.447+0000 7f5e34d89640 1 -- 172.21.15.16:0/2889419793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e300cf600 con 0x7f5e300cbd10 2023-12-08T08:39:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.447+0000 7f5e26ffd640 1 -- 172.21.15.16:0/2889419793 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5e20004b80 con 0x7f5e300cbd10 2023-12-08T08:39:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.448+0000 7f5e26ffd640 1 --2- 172.21.15.16:0/2889419793 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5e1006ae40 0x7f5e1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.448+0000 7f5e26ffd640 1 -- 172.21.15.16:0/2889419793 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f5e20088c80 con 0x7f5e300cbd10 2023-12-08T08:39:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.448+0000 7f5e27fff640 1 --2- 172.21.15.16:0/2889419793 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5e1006ae40 0x7f5e1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.448+0000 7f5e27fff640 1 --2- 172.21.15.16:0/2889419793 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5e1006ae40 0x7f5e1006d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5e18036000 tx=0x7f5e1803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:12.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.452+0000 7f5e26ffd640 1 -- 172.21.15.16:0/2889419793 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5e300cf600 con 0x7f5e300cbd10 2023-12-08T08:39:12.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.627+0000 7f5e34d89640 1 -- 172.21.15.16:0/2889419793 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e300cf600 con 0x7f5e300cbd10 2023-12-08T08:39:12.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.628+0000 7f5e26ffd640 1 -- 172.21.15.16:0/2889419793 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7f5e20018020 con 0x7f5e300cbd10 2023-12-08T08:39:12.631 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:12.631 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.630+0000 7f5e34d89640 1 -- 172.21.15.16:0/2889419793 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5e1006ae40 msgr2=0x7f5e1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.630+0000 7f5e34d89640 1 --2- 172.21.15.16:0/2889419793 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5e1006ae40 0x7f5e1006d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5e18036000 tx=0x7f5e1803d040 comp rx=0 tx=0).stop 2023-12-08T08:39:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.630+0000 7f5e34d89640 1 -- 172.21.15.16:0/2889419793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e300cbd10 msgr2=0x7f5e30099c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.630+0000 7f5e34d89640 1 --2- 172.21.15.16:0/2889419793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e300cbd10 0x7f5e30099c40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000c4d0 tx=0x7f5e2000c990 comp rx=0 tx=0).stop 2023-12-08T08:39:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.631+0000 7f5e34d89640 1 -- 172.21.15.16:0/2889419793 shutdown_connections 2023-12-08T08:39:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.631+0000 7f5e34d89640 1 --2- 172.21.15.16:0/2889419793 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5e1006ae40 0x7f5e1006d300 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.631+0000 7f5e34d89640 1 --2- 172.21.15.16:0/2889419793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e3015b750 0x7f5e30147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.631+0000 7f5e34d89640 1 --2- 172.21.15.16:0/2889419793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e30158ee0 0x7f5e301536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.631+0000 7f5e34d89640 1 --2- 172.21.15.16:0/2889419793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e300cbd10 0x7f5e30099c40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.631+0000 7f5e34d89640 1 -- 172.21.15.16:0/2889419793 >> 172.21.15.16:0/2889419793 conn(0x7f5e300b09f0 msgr2=0x7f5e3006dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.631+0000 7f5e34d89640 1 -- 172.21.15.16:0/2889419793 shutdown_connections 2023-12-08T08:39:12.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.631+0000 7f5e34d89640 1 -- 172.21.15.16:0/2889419793 wait complete. 2023-12-08T08:39:12.644 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:39:12.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:12.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.752+0000 7f6cd7d41640 1 Processor -- start 2023-12-08T08:39:12.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.752+0000 7f6cd7d41640 1 -- start start 2023-12-08T08:39:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.752+0000 7f6cd7d41640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6cd015d4f0 0x7f6cd00b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.753+0000 7f6cd7d41640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6cd00b0920 0x7f6cd00b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.753+0000 7f6cd7d41640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd00b2280 con 0x7f6cd00adb50 2023-12-08T08:39:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.753+0000 7f6cd7d41640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd006bb40 con 0x7f6cd015d4f0 2023-12-08T08:39:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.753+0000 7f6cd7d41640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00b2710 con 0x7f6cd00b0920 2023-12-08T08:39:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.753+0000 7f6cd5add640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6cd015d4f0 0x7f6cd00b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.753+0000 7f6cd62de640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6cd00adb50 0x7f6cd00adfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50702/0 (socket says 172.21.15.16:50702) 2023-12-08T08:39:12.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.753+0000 7f6cd62de640 1 -- 172.21.15.16:0/1158339258 learned_addr learned my addr 172.21.15.16:0/1158339258 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:12.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.753+0000 7f6cd52dc640 1 --2- 172.21.15.16:0/1158339258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6cd00b0920 0x7f6cd00b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.753+0000 7f6cd5add640 1 -- 172.21.15.16:0/1158339258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6cd00b0920 msgr2=0x7f6cd00b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:12.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.753+0000 7f6cd5add640 1 --2- 172.21.15.16:0/1158339258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6cd00b0920 0x7f6cd00b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.753+0000 7f6cd5add640 1 -- 172.21.15.16:0/1158339258 >> v1:172.21.15.16:6789/0 conn(0x7f6cd00adb50 legacy=0x7f6cd00adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.753+0000 7f6cd4adb640 1 -- 172.21.15.16:0/1158339258 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128015852 0 0) 0x7f6cd00b2280 con 0x7f6cd00adb50 2023-12-08T08:39:12.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.753+0000 7f6cd5add640 1 -- 172.21.15.16:0/1158339258 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd00b1690 con 0x7f6cd015d4f0 2023-12-08T08:39:12.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.754+0000 7f6cd5add640 1 --2- 172.21.15.16:0/1158339258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd015d4f0 0x7f6cd00b03e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0097a0 tx=0x7f6ccc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d7003bf74620ea server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:12.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.754+0000 7f6cd4adb640 1 -- 172.21.15.16:0/1158339258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc031830 con 0x7f6cd015d4f0 2023-12-08T08:39:12.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.754+0000 7f6cd4adb640 1 -- 172.21.15.16:0/1158339258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc031990 con 0x7f6cd015d4f0 2023-12-08T08:39:12.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.754+0000 7f6cd4adb640 1 -- 172.21.15.16:0/1158339258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc031d20 con 0x7f6cd015d4f0 2023-12-08T08:39:12.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.754+0000 7f6cd7d41640 1 -- 172.21.15.16:0/1158339258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd015d4f0 msgr2=0x7f6cd00b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.754+0000 7f6cd7d41640 1 --2- 172.21.15.16:0/1158339258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd015d4f0 0x7f6cd00b03e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0097a0 tx=0x7f6ccc02b8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.755+0000 7f6cd7d41640 1 -- 172.21.15.16:0/1158339258 shutdown_connections 2023-12-08T08:39:12.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.755+0000 7f6cd7d41640 1 --2- 172.21.15.16:0/1158339258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6cd00b0920 0x7f6cd00b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.755+0000 7f6cd7d41640 1 --2- 172.21.15.16:0/1158339258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd015d4f0 0x7f6cd00b03e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.755+0000 7f6cd7d41640 1 -- 172.21.15.16:0/1158339258 >> 172.21.15.16:0/1158339258 conn(0x7f6cd00d0e00 msgr2=0x7f6cd00d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:12.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.755+0000 7f6cd7d41640 1 -- 172.21.15.16:0/1158339258 shutdown_connections 2023-12-08T08:39:12.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.755+0000 7f6cd7d41640 1 -- 172.21.15.16:0/1158339258 wait complete. 2023-12-08T08:39:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.756+0000 7f6cd7d41640 1 Processor -- start 2023-12-08T08:39:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.756+0000 7f6cd7d41640 1 -- start start 2023-12-08T08:39:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd7d41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd00adb50 0x7f6cd00cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd7d41640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6cd00b0920 0x7f6cd00c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd7d41640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd015d4f0 0x7f6cd00c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd7d41640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd01f4300 con 0x7f6cd00adb50 2023-12-08T08:39:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd7d41640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd01f4020 con 0x7f6cd015d4f0 2023-12-08T08:39:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd7d41640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd00cdf90 con 0x7f6cd00b0920 2023-12-08T08:39:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd5add640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd00adb50 0x7f6cd00cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd5add640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd00adb50 0x7f6cd00cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35166/0 (socket says 172.21.15.16:35166) 2023-12-08T08:39:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd5add640 1 -- 172.21.15.16:0/25728168 learned_addr learned my addr 172.21.15.16:0/25728168 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd52dc640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6cd00b0920 0x7f6cd00c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd62de640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd015d4f0 0x7f6cd00c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd5add640 1 -- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6cd00b0920 msgr2=0x7f6cd00c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd5add640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6cd00b0920 0x7f6cd00c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd5add640 1 -- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd015d4f0 msgr2=0x7f6cd00c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd5add640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd015d4f0 0x7f6cd00c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd5add640 1 -- 172.21.15.16:0/25728168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc009400 con 0x7f6cd00adb50 2023-12-08T08:39:12.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd62de640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd015d4f0 0x7f6cd00c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:12.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd5add640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd00adb50 0x7f6cd00cb410 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc000c00 tx=0x7f6ccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:12.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cc67fc640 1 -- 172.21.15.16:0/25728168 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc004230 con 0x7f6cd00adb50 2023-12-08T08:39:12.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cc67fc640 1 -- 172.21.15.16:0/25728168 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc0043b0 con 0x7f6cd00adb50 2023-12-08T08:39:12.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cc67fc640 1 -- 172.21.15.16:0/25728168 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc004520 con 0x7f6cd00adb50 2023-12-08T08:39:12.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd7d41640 1 -- 172.21.15.16:0/25728168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd00ce270 con 0x7f6cd00adb50 2023-12-08T08:39:12.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.757+0000 7f6cd7d41640 1 -- 172.21.15.16:0/25728168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd00c7690 con 0x7f6cd00adb50 2023-12-08T08:39:12.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.758+0000 7f6cd7d41640 1 -- 172.21.15.16:0/25728168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd00ce820 con 0x7f6cd00adb50 2023-12-08T08:39:12.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.759+0000 7f6cc67fc640 1 -- 172.21.15.16:0/25728168 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ccc03a870 con 0x7f6cd00adb50 2023-12-08T08:39:12.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.759+0000 7f6cc67fc640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6cb406ae40 0x7f6cb406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:12.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.759+0000 7f6cc67fc640 1 -- 172.21.15.16:0/25728168 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f6ccc0abba0 con 0x7f6cd00adb50 2023-12-08T08:39:12.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.760+0000 7f6cd52dc640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6cb406ae40 0x7f6cb406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:12.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.760+0000 7f6cd52dc640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6cb406ae40 0x7f6cb406d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0153580 tx=0x7f6cc0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:12.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.764+0000 7f6cc67fc640 1 -- 172.21.15.16:0/25728168 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ccc076900 con 0x7f6cd00adb50 2023-12-08T08:39:12.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.941+0000 7f6cd7d41640 1 -- 172.21.15.16:0/25728168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6cd0067430 con 0x7f6cd00adb50 2023-12-08T08:39:12.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.942+0000 7f6cc67fc640 1 -- 172.21.15.16:0/25728168 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ccc079f20 con 0x7f6cd00adb50 2023-12-08T08:39:12.944 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.944+0000 7f6cd7d41640 1 -- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6cb406ae40 msgr2=0x7f6cb406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.944+0000 7f6cd7d41640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6cb406ae40 0x7f6cb406d300 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0153580 tx=0x7f6cc0008040 comp rx=0 tx=0).stop 2023-12-08T08:39:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.944+0000 7f6cd7d41640 1 -- 172.21.15.16:0/25728168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd00adb50 msgr2=0x7f6cd00cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.944+0000 7f6cd7d41640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd00adb50 0x7f6cd00cb410 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc000c00 tx=0x7f6ccc0046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.944+0000 7f6cd7d41640 1 -- 172.21.15.16:0/25728168 shutdown_connections 2023-12-08T08:39:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.944+0000 7f6cd7d41640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6cb406ae40 0x7f6cb406d300 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.944+0000 7f6cd7d41640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6cd015d4f0 0x7f6cd00c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.944+0000 7f6cd7d41640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6cd00b0920 0x7f6cd00c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.944+0000 7f6cd7d41640 1 --2- 172.21.15.16:0/25728168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6cd00adb50 0x7f6cd00cb410 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:12.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.944+0000 7f6cd7d41640 1 -- 172.21.15.16:0/25728168 >> 172.21.15.16:0/25728168 conn(0x7f6cd00d0e00 msgr2=0x7f6cd006e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:12.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.944+0000 7f6cd7d41640 1 -- 172.21.15.16:0/25728168 shutdown_connections 2023-12-08T08:39:12.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:12.944+0000 7f6cd7d41640 1 -- 172.21.15.16:0/25728168 wait complete. 2023-12-08T08:39:12.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:39:13.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.064+0000 7fda156bb640 1 Processor -- start 2023-12-08T08:39:13.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.064+0000 7fda156bb640 1 -- start start 2023-12-08T08:39:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.065+0000 7fda156bb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda1006b6e0 0x7fda1006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.065+0000 7fda156bb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda100b1260 con 0x7fda10160980 2023-12-08T08:39:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.065+0000 7fda156bb640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda100b0930 con 0x7fda100adae0 2023-12-08T08:39:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.065+0000 7fda156bb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda100d04f0 con 0x7fda1006b6e0 2023-12-08T08:39:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.065+0000 7fda0effd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fda10160980 0x7fda100cfac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50716/0 (socket says 172.21.15.16:50716) 2023-12-08T08:39:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.065+0000 7fda0effd640 1 -- 172.21.15.16:0/3760878270 learned_addr learned my addr 172.21.15.16:0/3760878270 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.066+0000 7fda0e7fc640 1 --2- 172.21.15.16:0/3760878270 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda1006b6e0 0x7fda1006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.066+0000 7fda0dffb640 1 -- 172.21.15.16:0/3760878270 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948790975 0 0) 0x7fda100b1260 con 0x7fda10160980 2023-12-08T08:39:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.066+0000 7fda0dffb640 1 -- 172.21.15.16:0/3760878270 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9fc003520 con 0x7fda10160980 2023-12-08T08:39:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.066+0000 7fda0dffb640 1 -- 172.21.15.16:0/3760878270 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988433009 0 0) 0x7fda100b0930 con 0x7fda100adae0 2023-12-08T08:39:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.066+0000 7fda0dffb640 1 -- 172.21.15.16:0/3760878270 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda100b1260 con 0x7fda100adae0 2023-12-08T08:39:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.066+0000 7fda0dffb640 1 -- 172.21.15.16:0/3760878270 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3023364911 0 0) 0x7fd9fc003520 con 0x7fda10160980 2023-12-08T08:39:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.066+0000 7fda0e7fc640 1 -- 172.21.15.16:0/3760878270 >> v1:172.21.15.112:6789/0 conn(0x7fda100adae0 legacy=0x7fda100b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.066+0000 7fda0e7fc640 1 -- 172.21.15.16:0/3760878270 >> v1:172.21.15.16:6789/0 conn(0x7fda10160980 legacy=0x7fda100cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.066+0000 7fda0e7fc640 1 -- 172.21.15.16:0/3760878270 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda100b0c10 con 0x7fda1006b6e0 2023-12-08T08:39:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.067+0000 7fda0e7fc640 1 --2- 172.21.15.16:0/3760878270 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda1006b6e0 0x7fda1006bca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80097d0 tx=0x7fd9f802b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f872f210a31c7ce6 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.067+0000 7fda0dffb640 1 -- 172.21.15.16:0/3760878270 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08001670 con 0x7fda1006b6e0 2023-12-08T08:39:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.067+0000 7fda0dffb640 1 -- 172.21.15.16:0/3760878270 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8031580 con 0x7fda1006b6e0 2023-12-08T08:39:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.067+0000 7fda0dffb640 1 -- 172.21.15.16:0/3760878270 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8031910 con 0x7fda1006b6e0 2023-12-08T08:39:13.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.067+0000 7fda156bb640 1 -- 172.21.15.16:0/3760878270 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda1006b6e0 msgr2=0x7fda1006bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.067+0000 7fda156bb640 1 --2- 172.21.15.16:0/3760878270 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda1006b6e0 0x7fda1006bca0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80097d0 tx=0x7fd9f802b8a0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.067+0000 7fda156bb640 1 -- 172.21.15.16:0/3760878270 shutdown_connections 2023-12-08T08:39:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.067+0000 7fda156bb640 1 --2- 172.21.15.16:0/3760878270 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda1006b6e0 0x7fda1006bca0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.067+0000 7fda156bb640 1 -- 172.21.15.16:0/3760878270 >> 172.21.15.16:0/3760878270 conn(0x7fda100d0e30 msgr2=0x7fda100d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.068+0000 7fda156bb640 1 -- 172.21.15.16:0/3760878270 shutdown_connections 2023-12-08T08:39:13.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.068+0000 7fda156bb640 1 -- 172.21.15.16:0/3760878270 wait complete. 2023-12-08T08:39:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.069+0000 7fda156bb640 1 Processor -- start 2023-12-08T08:39:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.069+0000 7fda156bb640 1 -- start start 2023-12-08T08:39:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda156bb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda1006b6e0 0x7fda1009dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda156bb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda100adae0 0x7fda1009e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda156bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda10160980 0x7fda1014c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda156bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1009e660 con 0x7fda10160980 2023-12-08T08:39:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda156bb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda101599b0 con 0x7fda1006b6e0 2023-12-08T08:39:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda156bb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1014a0f0 con 0x7fda100adae0 2023-12-08T08:39:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda0f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda10160980 0x7fda1014c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda0f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda10160980 0x7fda1014c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35174/0 (socket says 172.21.15.16:35174) 2023-12-08T08:39:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda0f7fe640 1 -- 172.21.15.16:0/1396748949 learned_addr learned my addr 172.21.15.16:0/1396748949 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda0e7fc640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda100adae0 0x7fda1009e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda0effd640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda1006b6e0 0x7fda1009dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda0f7fe640 1 -- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda100adae0 msgr2=0x7fda1009e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda0f7fe640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda100adae0 0x7fda1009e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda0f7fe640 1 -- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda1006b6e0 msgr2=0x7fda1009dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda0f7fe640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda1006b6e0 0x7fda1009dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda0f7fe640 1 -- 172.21.15.16:0/1396748949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f8009400 con 0x7fda10160980 2023-12-08T08:39:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda0effd640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda1006b6e0 0x7fda1009dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda0f7fe640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda10160980 0x7fda1014c0a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c7d0 tx=0x7fda0000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.070+0000 7fda077fe640 1 -- 172.21.15.16:0/1396748949 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00003b70 con 0x7fda10160980 2023-12-08T08:39:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.071+0000 7fda077fe640 1 -- 172.21.15.16:0/1396748949 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda00003cd0 con 0x7fda10160980 2023-12-08T08:39:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.071+0000 7fda156bb640 1 -- 172.21.15.16:0/1396748949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda100c6e90 con 0x7fda10160980 2023-12-08T08:39:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.071+0000 7fda077fe640 1 -- 172.21.15.16:0/1396748949 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0000aa90 con 0x7fda10160980 2023-12-08T08:39:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.071+0000 7fda156bb640 1 -- 172.21.15.16:0/1396748949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1014a640 con 0x7fda10160980 2023-12-08T08:39:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.071+0000 7fda156bb640 1 -- 172.21.15.16:0/1396748949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda1014a830 con 0x7fda10160980 2023-12-08T08:39:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.072+0000 7fda077fe640 1 -- 172.21.15.16:0/1396748949 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fda0000abf0 con 0x7fda10160980 2023-12-08T08:39:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.072+0000 7fda077fe640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd9f006ae40 0x7fd9f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.072+0000 7fda077fe640 1 -- 172.21.15.16:0/1396748949 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fda000898f0 con 0x7fda10160980 2023-12-08T08:39:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.073+0000 7fda0effd640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd9f006ae40 0x7fd9f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.073+0000 7fda0effd640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd9f006ae40 0x7fd9f006d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fda08000f60 tx=0x7fda08009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.077+0000 7fda077fe640 1 -- 172.21.15.16:0/1396748949 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fda000544b0 con 0x7fda10160980 2023-12-08T08:39:13.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.252+0000 7fda156bb640 1 -- 172.21.15.16:0/1396748949 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda10067ad0 con 0x7fda10160980 2023-12-08T08:39:13.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.252+0000 7fda077fe640 1 -- 172.21.15.16:0/1396748949 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda00087e60 con 0x7fda10160980 2023-12-08T08:39:13.255 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.254+0000 7fda156bb640 1 -- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd9f006ae40 msgr2=0x7fd9f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.254+0000 7fda156bb640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd9f006ae40 0x7fd9f006d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fda08000f60 tx=0x7fda08009040 comp rx=0 tx=0).stop 2023-12-08T08:39:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.254+0000 7fda156bb640 1 -- 172.21.15.16:0/1396748949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda10160980 msgr2=0x7fda1014c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.254+0000 7fda156bb640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda10160980 0x7fda1014c0a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fda0000c7d0 tx=0x7fda0000cc90 comp rx=0 tx=0).stop 2023-12-08T08:39:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.254+0000 7fda156bb640 1 -- 172.21.15.16:0/1396748949 shutdown_connections 2023-12-08T08:39:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.254+0000 7fda156bb640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd9f006ae40 0x7fd9f006d300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.254+0000 7fda156bb640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fda10160980 0x7fda1014c0a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.254+0000 7fda156bb640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fda100adae0 0x7fda1009e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.254+0000 7fda156bb640 1 --2- 172.21.15.16:0/1396748949 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fda1006b6e0 0x7fda1009dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.254+0000 7fda156bb640 1 -- 172.21.15.16:0/1396748949 >> 172.21.15.16:0/1396748949 conn(0x7fda100d0e30 msgr2=0x7fda100adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.254+0000 7fda156bb640 1 -- 172.21.15.16:0/1396748949 shutdown_connections 2023-12-08T08:39:13.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.254+0000 7fda156bb640 1 -- 172.21.15.16:0/1396748949 wait complete. 2023-12-08T08:39:13.267 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:39:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.374+0000 7f86dff05640 1 Processor -- start 2023-12-08T08:39:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.374+0000 7f86dff05640 1 -- start start 2023-12-08T08:39:13.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.375+0000 7f86dff05640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86d815e6a0 0x7f86d80ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.375+0000 7f86dff05640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f86d80b3c80 0x7f86d80b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.375+0000 7f86dff05640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d80ce790 con 0x7f86d80d1950 2023-12-08T08:39:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.375+0000 7f86dff05640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d80ceba0 con 0x7f86d815e6a0 2023-12-08T08:39:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.375+0000 7f86dff05640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d80cee80 con 0x7f86d80b3c80 2023-12-08T08:39:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.375+0000 7f86ddca1640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f86d815e6a0 0x7f86d80ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.375+0000 7f86dd4a0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f86d80d1950 0x7f86d80cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50724/0 (socket says 172.21.15.16:50724) 2023-12-08T08:39:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.375+0000 7f86dd4a0640 1 -- 172.21.15.16:0/276867207 learned_addr learned my addr 172.21.15.16:0/276867207 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.376+0000 7f86de4a2640 1 --2- 172.21.15.16:0/276867207 >> v2:172.21.15.112:3300/0 conn(0x7f86d80b3c80 0x7f86d80b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.376+0000 7f86dcc9f640 1 -- 172.21.15.16:0/276867207 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799491147 0 0) 0x7f86d80ce790 con 0x7f86d80d1950 2023-12-08T08:39:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.376+0000 7f86dcc9f640 1 -- 172.21.15.16:0/276867207 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c0003520 con 0x7f86d80d1950 2023-12-08T08:39:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.376+0000 7f86ddca1640 1 -- 172.21.15.16:0/276867207 >> v2:172.21.15.112:3300/0 conn(0x7f86d80b3c80 msgr2=0x7f86d80b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.376+0000 7f86ddca1640 1 --2- 172.21.15.16:0/276867207 >> v2:172.21.15.112:3300/0 conn(0x7f86d80b3c80 0x7f86d80b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.376+0000 7f86ddca1640 1 -- 172.21.15.16:0/276867207 >> v1:172.21.15.16:6789/0 conn(0x7f86d80d1950 legacy=0x7f86d80cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.376+0000 7f86ddca1640 1 -- 172.21.15.16:0/276867207 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d80cbcb0 con 0x7f86d815e6a0 2023-12-08T08:39:13.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.376+0000 7f86de4a2640 1 --2- 172.21.15.16:0/276867207 >> v2:172.21.15.112:3300/0 conn(0x7f86d80b3c80 0x7f86d80b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:13.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.377+0000 7f86ddca1640 1 --2- 172.21.15.16:0/276867207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d815e6a0 0x7f86d80ce250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f86d8072690 tx=0x7f86d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d16457a2e279f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:13.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.377+0000 7f86dcc9f640 1 -- 172.21.15.16:0/276867207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0031830 con 0x7f86d815e6a0 2023-12-08T08:39:13.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.377+0000 7f86dcc9f640 1 -- 172.21.15.16:0/276867207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d0031990 con 0x7f86d815e6a0 2023-12-08T08:39:13.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.377+0000 7f86dcc9f640 1 -- 172.21.15.16:0/276867207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d0031d20 con 0x7f86d815e6a0 2023-12-08T08:39:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.377+0000 7f86dff05640 1 -- 172.21.15.16:0/276867207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d815e6a0 msgr2=0x7f86d80ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.377+0000 7f86dff05640 1 --2- 172.21.15.16:0/276867207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d815e6a0 0x7f86d80ce250 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f86d8072690 tx=0x7f86d002b8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.377+0000 7f86dff05640 1 -- 172.21.15.16:0/276867207 shutdown_connections 2023-12-08T08:39:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.377+0000 7f86dff05640 1 --2- 172.21.15.16:0/276867207 >> v2:172.21.15.112:3300/0 conn(0x7f86d80b3c80 0x7f86d80b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.377+0000 7f86dff05640 1 --2- 172.21.15.16:0/276867207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d815e6a0 0x7f86d80ce250 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.377+0000 7f86dff05640 1 -- 172.21.15.16:0/276867207 >> 172.21.15.16:0/276867207 conn(0x7f86d80d0e30 msgr2=0x7f86d80d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.377+0000 7f86dff05640 1 -- 172.21.15.16:0/276867207 shutdown_connections 2023-12-08T08:39:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.377+0000 7f86dff05640 1 -- 172.21.15.16:0/276867207 wait complete. 2023-12-08T08:39:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.378+0000 7f86dff05640 1 Processor -- start 2023-12-08T08:39:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.378+0000 7f86dff05640 1 -- start start 2023-12-08T08:39:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.379+0000 7f86dff05640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86d80b3c80 0x7f86d81543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.379+0000 7f86dff05640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d80d1950 0x7f86d81f1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.379+0000 7f86dff05640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86d815e6a0 0x7f86d81554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.379+0000 7f86dff05640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d81f0980 con 0x7f86d80d1950 2023-12-08T08:39:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.379+0000 7f86dff05640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d81564d0 con 0x7f86d80b3c80 2023-12-08T08:39:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.379+0000 7f86dff05640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d8154920 con 0x7f86d815e6a0 2023-12-08T08:39:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.379+0000 7f86dd4a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d80d1950 0x7f86d81f1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.379+0000 7f86dd4a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d80d1950 0x7f86d81f1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35202/0 (socket says 172.21.15.16:35202) 2023-12-08T08:39:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.379+0000 7f86dd4a0640 1 -- 172.21.15.16:0/3929869874 learned_addr learned my addr 172.21.15.16:0/3929869874 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.379+0000 7f86ddca1640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86d80b3c80 0x7f86d81543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.379+0000 7f86de4a2640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86d815e6a0 0x7f86d81554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.380+0000 7f86dd4a0640 1 -- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86d815e6a0 msgr2=0x7f86d81554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.380+0000 7f86dd4a0640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86d815e6a0 0x7f86d81554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.380+0000 7f86dd4a0640 1 -- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86d80b3c80 msgr2=0x7f86d81543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.380+0000 7f86dd4a0640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86d80b3c80 0x7f86d81543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.380+0000 7f86dd4a0640 1 -- 172.21.15.16:0/3929869874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d0009400 con 0x7f86d80d1950 2023-12-08T08:39:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.380+0000 7f86ddca1640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86d80b3c80 0x7f86d81543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.380+0000 7f86dd4a0640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d80d1950 0x7f86d81f1370 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00c410 tx=0x7f86cc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.380+0000 7f86d67fc640 1 -- 172.21.15.16:0/3929869874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc005ce0 con 0x7f86d80d1950 2023-12-08T08:39:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.380+0000 7f86d67fc640 1 -- 172.21.15.16:0/3929869874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc005e40 con 0x7f86d80d1950 2023-12-08T08:39:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.380+0000 7f86dff05640 1 -- 172.21.15.16:0/3929869874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d8155f10 con 0x7f86d80d1950 2023-12-08T08:39:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.380+0000 7f86d67fc640 1 -- 172.21.15.16:0/3929869874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc0072f0 con 0x7f86d80d1950 2023-12-08T08:39:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.380+0000 7f86dff05640 1 -- 172.21.15.16:0/3929869874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d81f0b70 con 0x7f86d80d1950 2023-12-08T08:39:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.381+0000 7f86dff05640 1 -- 172.21.15.16:0/3929869874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d8067ad0 con 0x7f86d80d1950 2023-12-08T08:39:13.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.381+0000 7f86d67fc640 1 -- 172.21.15.16:0/3929869874 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f86cc007510 con 0x7f86d80d1950 2023-12-08T08:39:13.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.382+0000 7f86d67fc640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f86c406ae40 0x7f86c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.382+0000 7f86d67fc640 1 -- 172.21.15.16:0/3929869874 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f86cc0890a0 con 0x7f86d80d1950 2023-12-08T08:39:13.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.382+0000 7f86ddca1640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f86c406ae40 0x7f86c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.383+0000 7f86ddca1640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f86c406ae40 0x7f86c406d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f86d0036000 tx=0x7f86d003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:13.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.386+0000 7f86d67fc640 1 -- 172.21.15.16:0/3929869874 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f86cc053e30 con 0x7f86d80d1950 2023-12-08T08:39:13.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.562+0000 7f86dff05640 1 -- 172.21.15.16:0/3929869874 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f86d806dbe0 con 0x7f86d80d1950 2023-12-08T08:39:13.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.562+0000 7f86d67fc640 1 -- 172.21.15.16:0/3929869874 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f86cc017020 con 0x7f86d80d1950 2023-12-08T08:39:13.565 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:39:13.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.564+0000 7f86dff05640 1 -- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f86c406ae40 msgr2=0x7f86c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.564+0000 7f86dff05640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f86c406ae40 0x7f86c406d300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f86d0036000 tx=0x7f86d003c040 comp rx=0 tx=0).stop 2023-12-08T08:39:13.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.564+0000 7f86dff05640 1 -- 172.21.15.16:0/3929869874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d80d1950 msgr2=0x7f86d81f1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.564+0000 7f86dff05640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d80d1950 0x7f86d81f1370 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f86cc00c410 tx=0x7f86cc00c8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.564+0000 7f86dff05640 1 -- 172.21.15.16:0/3929869874 shutdown_connections 2023-12-08T08:39:13.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.564+0000 7f86dff05640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f86c406ae40 0x7f86c406d300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.564+0000 7f86dff05640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f86d815e6a0 0x7f86d81554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.564+0000 7f86dff05640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86d80d1950 0x7f86d81f1370 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.564+0000 7f86dff05640 1 --2- 172.21.15.16:0/3929869874 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f86d80b3c80 0x7f86d81543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.564+0000 7f86dff05640 1 -- 172.21.15.16:0/3929869874 >> 172.21.15.16:0/3929869874 conn(0x7f86d80d0e30 msgr2=0x7f86d80cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:13.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.564+0000 7f86dff05640 1 -- 172.21.15.16:0/3929869874 shutdown_connections 2023-12-08T08:39:13.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.564+0000 7f86dff05640 1 -- 172.21.15.16:0/3929869874 wait complete. 2023-12-08T08:39:13.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:39:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.684+0000 7fba743c9640 1 Processor -- start 2023-12-08T08:39:13.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.684+0000 7fba743c9640 1 -- start start 2023-12-08T08:39:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.685+0000 7fba743c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fba6c0cfe10 0x7fba6c0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.685+0000 7fba743c9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba6c0c6be0 con 0x7fba6c0cfa30 2023-12-08T08:39:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.685+0000 7fba743c9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba6c0d3ff0 con 0x7fba6c0b4150 2023-12-08T08:39:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.685+0000 7fba743c9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6c0b4860 con 0x7fba6c0cfe10 2023-12-08T08:39:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.685+0000 7fba72966640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fba6c0cfe10 0x7fba6c0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.685+0000 7fba72165640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fba6c0cfa30 0x7fba6c0b2f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50740/0 (socket says 172.21.15.16:50740) 2023-12-08T08:39:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.686+0000 7fba72165640 1 -- 172.21.15.16:0/3748928665 learned_addr learned my addr 172.21.15.16:0/3748928665 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.686+0000 7fba72966640 1 -- 172.21.15.16:0/3748928665 >> v1:172.21.15.112:6789/0 conn(0x7fba6c0b4150 legacy=0x7fba6c0cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.686+0000 7fba72966640 1 -- 172.21.15.16:0/3748928665 >> v1:172.21.15.16:6789/0 conn(0x7fba6c0cfa30 legacy=0x7fba6c0b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.686+0000 7fba72966640 1 -- 172.21.15.16:0/3748928665 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c0d3d50 con 0x7fba6c0cfe10 2023-12-08T08:39:13.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.686+0000 7fba71163640 1 -- 172.21.15.16:0/3748928665 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731867183 0 0) 0x7fba6c0c6be0 con 0x7fba6c0cfa30 2023-12-08T08:39:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.687+0000 7fba72966640 1 --2- 172.21.15.16:0/3748928665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0cfe10 0x7fba6c0b3630 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fba64009d90 tx=0x7fba6402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d66db37a1e96600 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.687+0000 7fba71163640 1 -- 172.21.15.16:0/3748928665 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6402cbd0 con 0x7fba6c0cfe10 2023-12-08T08:39:13.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.687+0000 7fba71163640 1 -- 172.21.15.16:0/3748928665 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6402cd30 con 0x7fba6c0cfe10 2023-12-08T08:39:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.687+0000 7fba71163640 1 -- 172.21.15.16:0/3748928665 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6402cea0 con 0x7fba6c0cfe10 2023-12-08T08:39:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.687+0000 7fba743c9640 1 -- 172.21.15.16:0/3748928665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0cfe10 msgr2=0x7fba6c0b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.687+0000 7fba743c9640 1 --2- 172.21.15.16:0/3748928665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0cfe10 0x7fba6c0b3630 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fba64009d90 tx=0x7fba6402be20 comp rx=0 tx=0).stop 2023-12-08T08:39:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.687+0000 7fba743c9640 1 -- 172.21.15.16:0/3748928665 shutdown_connections 2023-12-08T08:39:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.687+0000 7fba743c9640 1 --2- 172.21.15.16:0/3748928665 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0cfe10 0x7fba6c0b3630 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.687+0000 7fba743c9640 1 -- 172.21.15.16:0/3748928665 >> 172.21.15.16:0/3748928665 conn(0x7fba6c0d3090 msgr2=0x7fba6c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.687+0000 7fba743c9640 1 -- 172.21.15.16:0/3748928665 shutdown_connections 2023-12-08T08:39:13.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.687+0000 7fba743c9640 1 -- 172.21.15.16:0/3748928665 wait complete. 2023-12-08T08:39:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.689+0000 7fba743c9640 1 Processor -- start 2023-12-08T08:39:13.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.689+0000 7fba743c9640 1 -- start start 2023-12-08T08:39:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.689+0000 7fba743c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0b4150 0x7fba6c15edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.689+0000 7fba743c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba6c0cfa30 0x7fba6c155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.689+0000 7fba743c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6c0cfe10 0x7fba6c1536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.689+0000 7fba743c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c15ffa0 con 0x7fba6c0b4150 2023-12-08T08:39:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.689+0000 7fba743c9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c153c00 con 0x7fba6c0cfe10 2023-12-08T08:39:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.689+0000 7fba743c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c15f390 con 0x7fba6c0cfa30 2023-12-08T08:39:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba72165640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0b4150 0x7fba6c15edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba72165640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0b4150 0x7fba6c15edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35208/0 (socket says 172.21.15.16:35208) 2023-12-08T08:39:13.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba72165640 1 -- 172.21.15.16:0/1093154868 learned_addr learned my addr 172.21.15.16:0/1093154868 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba72966640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6c0cfe10 0x7fba6c1536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba72165640 1 -- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba6c0cfa30 msgr2=0x7fba6c155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba71964640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba6c0cfa30 0x7fba6c155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba72165640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba6c0cfa30 0x7fba6c155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba72165640 1 -- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6c0cfe10 msgr2=0x7fba6c1536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba72165640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6c0cfe10 0x7fba6c1536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba72165640 1 -- 172.21.15.16:0/1093154868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba64009a70 con 0x7fba6c0b4150 2023-12-08T08:39:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba72966640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6c0cfe10 0x7fba6c1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba71964640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba6c0cfa30 0x7fba6c155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba72165640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0b4150 0x7fba6c15edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba6800c4d0 tx=0x7fba6800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba62ffd640 1 -- 172.21.15.16:0/1093154868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba680074e0 con 0x7fba6c0b4150 2023-12-08T08:39:13.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba62ffd640 1 -- 172.21.15.16:0/1093154868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba68007660 con 0x7fba6c0b4150 2023-12-08T08:39:13.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba743c9640 1 -- 172.21.15.16:0/1093154868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba6c15f670 con 0x7fba6c0b4150 2023-12-08T08:39:13.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba62ffd640 1 -- 172.21.15.16:0/1093154868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba68004a20 con 0x7fba6c0b4150 2023-12-08T08:39:13.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.690+0000 7fba743c9640 1 -- 172.21.15.16:0/1093154868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6c15fc30 con 0x7fba6c0b4150 2023-12-08T08:39:13.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.691+0000 7fba743c9640 1 -- 172.21.15.16:0/1093154868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba6c15d0c0 con 0x7fba6c0b4150 2023-12-08T08:39:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.692+0000 7fba62ffd640 1 -- 172.21.15.16:0/1093154868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba68004b80 con 0x7fba6c0b4150 2023-12-08T08:39:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.692+0000 7fba62ffd640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fba5006ae40 0x7fba5006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.692+0000 7fba62ffd640 1 -- 172.21.15.16:0/1093154868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fba68088f50 con 0x7fba6c0b4150 2023-12-08T08:39:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.692+0000 7fba71964640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fba5006ae40 0x7fba5006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.693+0000 7fba71964640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fba5006ae40 0x7fba5006d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0c9550 tx=0x7fba5c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.696+0000 7fba62ffd640 1 -- 172.21.15.16:0/1093154868 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba68053c20 con 0x7fba6c0b4150 2023-12-08T08:39:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.873+0000 7fba743c9640 1 -- 172.21.15.16:0/1093154868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fba6c067430 con 0x7fba6c0b4150 2023-12-08T08:39:13.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.873+0000 7fba62ffd640 1 -- 172.21.15.16:0/1093154868 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fba68018020 con 0x7fba6c0b4150 2023-12-08T08:39:13.876 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.876+0000 7fba743c9640 1 -- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fba5006ae40 msgr2=0x7fba5006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.876+0000 7fba743c9640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fba5006ae40 0x7fba5006d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0c9550 tx=0x7fba5c008040 comp rx=0 tx=0).stop 2023-12-08T08:39:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.876+0000 7fba743c9640 1 -- 172.21.15.16:0/1093154868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0b4150 msgr2=0x7fba6c15edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.876+0000 7fba743c9640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0b4150 0x7fba6c15edb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fba6800c4d0 tx=0x7fba6800c990 comp rx=0 tx=0).stop 2023-12-08T08:39:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.876+0000 7fba743c9640 1 -- 172.21.15.16:0/1093154868 shutdown_connections 2023-12-08T08:39:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.876+0000 7fba743c9640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fba5006ae40 0x7fba5006d300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.876+0000 7fba743c9640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fba6c0cfe10 0x7fba6c1536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.876+0000 7fba743c9640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fba6c0cfa30 0x7fba6c155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.876+0000 7fba743c9640 1 --2- 172.21.15.16:0/1093154868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba6c0b4150 0x7fba6c15edb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.876+0000 7fba743c9640 1 -- 172.21.15.16:0/1093154868 >> 172.21.15.16:0/1093154868 conn(0x7fba6c0d3090 msgr2=0x7fba6c0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:13.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.876+0000 7fba743c9640 1 -- 172.21.15.16:0/1093154868 shutdown_connections 2023-12-08T08:39:13.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.876+0000 7fba743c9640 1 -- 172.21.15.16:0/1093154868 wait complete. 2023-12-08T08:39:13.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-08T08:39:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.994+0000 7fced33b0640 1 Processor -- start 2023-12-08T08:39:13.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.994+0000 7fced33b0640 1 -- start start 2023-12-08T08:39:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.995+0000 7fced33b0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcecc0cbd10 0x7fcecc070e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.995+0000 7fced33b0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcecc15b750 0x7fcecc06c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:13.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.995+0000 7fced33b0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcecc0720d0 con 0x7fcecc158ee0 2023-12-08T08:39:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.995+0000 7fced33b0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcecc0cc890 con 0x7fcecc15b750 2023-12-08T08:39:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.995+0000 7fced33b0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc0714c0 con 0x7fcecc0cbd10 2023-12-08T08:39:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.995+0000 7fced114c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcecc158ee0 0x7fcecc0cb600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50752/0 (socket says 172.21.15.16:50752) 2023-12-08T08:39:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.995+0000 7fced114c640 1 -- 172.21.15.16:0/2838872835 learned_addr learned my addr 172.21.15.16:0/2838872835 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.995+0000 7fced094b640 1 --2- 172.21.15.16:0/2838872835 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcecc0cbd10 0x7fcecc070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.995+0000 7fced194d640 1 --2- 172.21.15.16:0/2838872835 >> v2:172.21.15.112:3300/0 conn(0x7fcecc15b750 0x7fcecc06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.995+0000 7fcecbfff640 1 -- 172.21.15.16:0/2838872835 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3314582113 0 0) 0x7fcecc0720d0 con 0x7fcecc158ee0 2023-12-08T08:39:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.995+0000 7fcecbfff640 1 -- 172.21.15.16:0/2838872835 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fceb4003520 con 0x7fcecc158ee0 2023-12-08T08:39:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.996+0000 7fced094b640 1 -- 172.21.15.16:0/2838872835 >> v2:172.21.15.112:3300/0 conn(0x7fcecc15b750 msgr2=0x7fcecc06c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.996+0000 7fced094b640 1 --2- 172.21.15.16:0/2838872835 >> v2:172.21.15.112:3300/0 conn(0x7fcecc15b750 0x7fcecc06c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.996+0000 7fcecbfff640 1 -- 172.21.15.16:0/2838872835 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1957742595 0 0) 0x7fceb4003520 con 0x7fcecc158ee0 2023-12-08T08:39:13.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.996+0000 7fced094b640 1 -- 172.21.15.16:0/2838872835 >> v1:172.21.15.16:6789/0 conn(0x7fcecc158ee0 legacy=0x7fcecc0cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:13.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.996+0000 7fced094b640 1 -- 172.21.15.16:0/2838872835 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc0717a0 con 0x7fcecc0cbd10 2023-12-08T08:39:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.996+0000 7fced094b640 1 --2- 172.21.15.16:0/2838872835 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcecc0cbd10 0x7fcecc070e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcec00097a0 tx=0x7fcec002b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee71275ed3dec52e server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.997+0000 7fcecbfff640 1 -- 172.21.15.16:0/2838872835 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4001670 con 0x7fcecc0cbd10 2023-12-08T08:39:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.997+0000 7fcecbfff640 1 -- 172.21.15.16:0/2838872835 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec002e050 con 0x7fcecc0cbd10 2023-12-08T08:39:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.997+0000 7fcecbfff640 1 -- 172.21.15.16:0/2838872835 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec0030730 con 0x7fcecc0cbd10 2023-12-08T08:39:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.997+0000 7fced33b0640 1 -- 172.21.15.16:0/2838872835 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcecc0cbd10 msgr2=0x7fcecc070e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.997+0000 7fced33b0640 1 --2- 172.21.15.16:0/2838872835 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcecc0cbd10 0x7fcecc070e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcec00097a0 tx=0x7fcec002b710 comp rx=0 tx=0).stop 2023-12-08T08:39:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.997+0000 7fced33b0640 1 -- 172.21.15.16:0/2838872835 shutdown_connections 2023-12-08T08:39:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.997+0000 7fced33b0640 1 --2- 172.21.15.16:0/2838872835 >> v2:172.21.15.112:3300/0 conn(0x7fcecc15b750 0x7fcecc06c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.997+0000 7fced33b0640 1 --2- 172.21.15.16:0/2838872835 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcecc0cbd10 0x7fcecc070e30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.997+0000 7fced33b0640 1 -- 172.21.15.16:0/2838872835 >> 172.21.15.16:0/2838872835 conn(0x7fcecc0b09f0 msgr2=0x7fcecc0b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.997+0000 7fced33b0640 1 -- 172.21.15.16:0/2838872835 shutdown_connections 2023-12-08T08:39:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.997+0000 7fced33b0640 1 -- 172.21.15.16:0/2838872835 wait complete. 2023-12-08T08:39:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.998+0000 7fced33b0640 1 Processor -- start 2023-12-08T08:39:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.999+0000 7fced33b0640 1 -- start start 2023-12-08T08:39:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.999+0000 7fced33b0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcecc0cbd10 0x7fcecc0b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.999+0000 7fced33b0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcecc158ee0 0x7fcecc151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.999+0000 7fced33b0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcecc15b750 0x7fcecc09b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.999+0000 7fced33b0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc09b8c0 con 0x7fcecc0cbd10 2023-12-08T08:39:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.999+0000 7fced33b0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc0b3e20 con 0x7fcecc158ee0 2023-12-08T08:39:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.999+0000 7fced33b0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc09a5c0 con 0x7fcecc15b750 2023-12-08T08:39:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.999+0000 7fced114c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcecc0cbd10 0x7fcecc0b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.999+0000 7fced114c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcecc0cbd10 0x7fcecc0b2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35210/0 (socket says 172.21.15.16:35210) 2023-12-08T08:39:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.999+0000 7fced114c640 1 -- 172.21.15.16:0/2348570875 learned_addr learned my addr 172.21.15.16:0/2348570875 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.999+0000 7fced194d640 1 --2- 172.21.15.16:0/2348570875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcecc15b750 0x7fcecc09b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:13.999+0000 7fced094b640 1 --2- 172.21.15.16:0/2348570875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcecc158ee0 0x7fcecc151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.000+0000 7fced114c640 1 -- 172.21.15.16:0/2348570875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcecc15b750 msgr2=0x7fcecc09b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.000+0000 7fced114c640 1 --2- 172.21.15.16:0/2348570875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcecc15b750 0x7fcecc09b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.000+0000 7fced114c640 1 -- 172.21.15.16:0/2348570875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcecc158ee0 msgr2=0x7fcecc151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.000+0000 7fced114c640 1 --2- 172.21.15.16:0/2348570875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcecc158ee0 0x7fcecc151640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.000+0000 7fced114c640 1 -- 172.21.15.16:0/2348570875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec0009400 con 0x7fcecc0cbd10 2023-12-08T08:39:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.000+0000 7fced114c640 1 --2- 172.21.15.16:0/2348570875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcecc0cbd10 0x7fcecc0b2e70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcec400c710 tx=0x7fcec400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.000+0000 7fcec9ffb640 1 -- 172.21.15.16:0/2348570875 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4005ad0 con 0x7fcecc0cbd10 2023-12-08T08:39:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.000+0000 7fcec9ffb640 1 -- 172.21.15.16:0/2348570875 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec4005c30 con 0x7fcecc0cbd10 2023-12-08T08:39:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.000+0000 7fcec9ffb640 1 -- 172.21.15.16:0/2348570875 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec4007070 con 0x7fcecc0cbd10 2023-12-08T08:39:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.000+0000 7fced33b0640 1 -- 172.21.15.16:0/2348570875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcecc09a8a0 con 0x7fcecc0cbd10 2023-12-08T08:39:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.000+0000 7fced33b0640 1 -- 172.21.15.16:0/2348570875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcecc09ae60 con 0x7fcecc0cbd10 2023-12-08T08:39:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.001+0000 7fced33b0640 1 -- 172.21.15.16:0/2348570875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcecc071030 con 0x7fcecc0cbd10 2023-12-08T08:39:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.002+0000 7fcec9ffb640 1 -- 172.21.15.16:0/2348570875 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcec40071f0 con 0x7fcecc0cbd10 2023-12-08T08:39:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.002+0000 7fcec9ffb640 1 --2- 172.21.15.16:0/2348570875 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fceb806ae40 0x7fceb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.002+0000 7fcec9ffb640 1 -- 172.21.15.16:0/2348570875 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fcec4089610 con 0x7fcecc0cbd10 2023-12-08T08:39:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.002+0000 7fced094b640 1 --2- 172.21.15.16:0/2348570875 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fceb806ae40 0x7fceb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.003+0000 7fced094b640 1 --2- 172.21.15.16:0/2348570875 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fceb806ae40 0x7fceb806d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcec0034000 tx=0x7fcec0008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.007+0000 7fcec9ffb640 1 -- 172.21.15.16:0/2348570875 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcec4054390 con 0x7fcecc0cbd10 2023-12-08T08:39:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.194+0000 7fced33b0640 1 -- 172.21.15.16:0/2348570875 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcecc072430 con 0x7fcecc0cbd10 2023-12-08T08:39:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.195+0000 7fcec9ffb640 1 -- 172.21.15.16:0/2348570875 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcec40579b0 con 0x7fcecc0cbd10 2023-12-08T08:39:14.197 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-12-08T08:39:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.197+0000 7fced33b0640 1 -- 172.21.15.16:0/2348570875 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fceb806ae40 msgr2=0x7fceb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.197+0000 7fced33b0640 1 --2- 172.21.15.16:0/2348570875 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fceb806ae40 0x7fceb806d300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcec0034000 tx=0x7fcec0008030 comp rx=0 tx=0).stop 2023-12-08T08:39:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.197+0000 7fced33b0640 1 -- 172.21.15.16:0/2348570875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcecc0cbd10 msgr2=0x7fcecc0b2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.197+0000 7fced33b0640 1 --2- 172.21.15.16:0/2348570875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcecc0cbd10 0x7fcecc0b2e70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcec400c710 tx=0x7fcec400cbd0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.197+0000 7fced33b0640 1 -- 172.21.15.16:0/2348570875 shutdown_connections 2023-12-08T08:39:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.197+0000 7fced33b0640 1 --2- 172.21.15.16:0/2348570875 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fceb806ae40 0x7fceb806d300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.197+0000 7fced33b0640 1 --2- 172.21.15.16:0/2348570875 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcecc15b750 0x7fcecc09b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.197+0000 7fced33b0640 1 --2- 172.21.15.16:0/2348570875 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcecc158ee0 0x7fcecc151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.197+0000 7fced33b0640 1 --2- 172.21.15.16:0/2348570875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcecc0cbd10 0x7fcecc0b2e70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.197+0000 7fced33b0640 1 -- 172.21.15.16:0/2348570875 >> 172.21.15.16:0/2348570875 conn(0x7fcecc0b09f0 msgr2=0x7fcecc0ad6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.197+0000 7fced33b0640 1 -- 172.21.15.16:0/2348570875 shutdown_connections 2023-12-08T08:39:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.197+0000 7fced33b0640 1 -- 172.21.15.16:0/2348570875 wait complete. 2023-12-08T08:39:14.210 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-12-08T08:39:14.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:14.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-08T08:39:14.241 INFO:teuthology.orchestra.run:waiting for 300 2023-12-08T08:39:14.241 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-12-08T08:39:14.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:14.242 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-08T08:39:14.297 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:39:14.297 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T08:39:14.316 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:39:14.317 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T08:39:14.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.480+0000 7fd63bc9b640 1 Processor -- start 2023-12-08T08:39:14.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.480+0000 7fd63bc9b640 1 -- start start 2023-12-08T08:39:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.481+0000 7fd63bc9b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6340ce3e0 0x7fd6340b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.481+0000 7fd63bc9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd634071e50 0x7fd6340b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.481+0000 7fd63bc9b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6340b29b0 con 0x7fd634140a10 2023-12-08T08:39:14.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.481+0000 7fd63bc9b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd634070ba0 con 0x7fd634071e50 2023-12-08T08:39:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.481+0000 7fd63bc9b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd634070db0 con 0x7fd6340ce3e0 2023-12-08T08:39:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.481+0000 7fd63a238640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd634071e50 0x7fd6340b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.481+0000 7fd63a238640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd634071e50 0x7fd6340b2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35222/0 (socket says 172.21.15.16:35222) 2023-12-08T08:39:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.481+0000 7fd63a238640 1 -- 172.21.15.16:0/563071891 learned_addr learned my addr 172.21.15.16:0/563071891 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.481+0000 7fd639236640 1 --2- 172.21.15.16:0/563071891 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6340ce3e0 0x7fd6340b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.481+0000 7fd63a238640 1 -- 172.21.15.16:0/563071891 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6340ce3e0 msgr2=0x7fd6340b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.481+0000 7fd63a238640 1 --2- 172.21.15.16:0/563071891 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6340ce3e0 0x7fd6340b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.482+0000 7fd63a238640 1 -- 172.21.15.16:0/563071891 >> v1:172.21.15.112:6789/0 conn(0x7fd634140a10 legacy=0x7fd6340cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:14.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.482+0000 7fd63a238640 1 -- 172.21.15.16:0/563071891 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd634071090 con 0x7fd634071e50 2023-12-08T08:39:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.482+0000 7fd63a238640 1 --2- 172.21.15.16:0/563071891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd634071e50 0x7fd6340b2470 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd624009e50 tx=0x7fd62402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e566976fcf2c22b server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.482+0000 7fd638a35640 1 -- 172.21.15.16:0/563071891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62402cb00 con 0x7fd634071e50 2023-12-08T08:39:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.482+0000 7fd638a35640 1 -- 172.21.15.16:0/563071891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62402cc60 con 0x7fd634071e50 2023-12-08T08:39:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.482+0000 7fd638a35640 1 -- 172.21.15.16:0/563071891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6240343f0 con 0x7fd634071e50 2023-12-08T08:39:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.483+0000 7fd63bc9b640 1 -- 172.21.15.16:0/563071891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd634071e50 msgr2=0x7fd6340b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.483+0000 7fd63bc9b640 1 --2- 172.21.15.16:0/563071891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd634071e50 0x7fd6340b2470 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd624009e50 tx=0x7fd62402bf60 comp rx=0 tx=0).stop 2023-12-08T08:39:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.483+0000 7fd63bc9b640 1 -- 172.21.15.16:0/563071891 shutdown_connections 2023-12-08T08:39:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.483+0000 7fd63bc9b640 1 --2- 172.21.15.16:0/563071891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd634071e50 0x7fd6340b2470 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.483+0000 7fd63bc9b640 1 --2- 172.21.15.16:0/563071891 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd6340ce3e0 0x7fd6340b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.483+0000 7fd63bc9b640 1 -- 172.21.15.16:0/563071891 >> 172.21.15.16:0/563071891 conn(0x7fd6340b09f0 msgr2=0x7fd6340b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.483+0000 7fd63bc9b640 1 -- 172.21.15.16:0/563071891 shutdown_connections 2023-12-08T08:39:14.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.483+0000 7fd63bc9b640 1 -- 172.21.15.16:0/563071891 wait complete. 2023-12-08T08:39:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.484+0000 7fd63bc9b640 1 Processor -- start 2023-12-08T08:39:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.484+0000 7fd63bc9b640 1 -- start start 2023-12-08T08:39:14.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd63bc9b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634071e50 0x7fd63409c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd63bc9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6340ce3e0 0x7fd6341f0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd63bc9b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd634140a10 0x7fd634159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd63bc9b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63415a280 con 0x7fd6340ce3e0 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd63bc9b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6340ac050 con 0x7fd634071e50 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd63bc9b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6341f13a0 con 0x7fd634140a10 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd639236640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6340ce3e0 0x7fd6341f0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd639236640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6340ce3e0 0x7fd6341f0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35228/0 (socket says 172.21.15.16:35228) 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd639236640 1 -- 172.21.15.16:0/502971515 learned_addr learned my addr 172.21.15.16:0/502971515 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd63a238640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd634140a10 0x7fd634159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd639a37640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634071e50 0x7fd63409c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd639236640 1 -- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd634140a10 msgr2=0x7fd634159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd639236640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd634140a10 0x7fd634159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd639236640 1 -- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634071e50 msgr2=0x7fd63409c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd639236640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634071e50 0x7fd63409c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd639236640 1 -- 172.21.15.16:0/502971515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd624009b30 con 0x7fd6340ce3e0 2023-12-08T08:39:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd63a238640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd634140a10 0x7fd634159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.485+0000 7fd639a37640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634071e50 0x7fd63409c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.486+0000 7fd639236640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6340ce3e0 0x7fd6341f0ca0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c7f0 tx=0x7fd62800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.486+0000 7fd6327fc640 1 -- 172.21.15.16:0/502971515 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628004220 con 0x7fd6340ce3e0 2023-12-08T08:39:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.486+0000 7fd6327fc640 1 -- 172.21.15.16:0/502971515 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd628004380 con 0x7fd6340ce3e0 2023-12-08T08:39:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.486+0000 7fd63bc9b640 1 -- 172.21.15.16:0/502971515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6341f1680 con 0x7fd6340ce3e0 2023-12-08T08:39:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.486+0000 7fd6327fc640 1 -- 172.21.15.16:0/502971515 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628007840 con 0x7fd6340ce3e0 2023-12-08T08:39:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.486+0000 7fd63bc9b640 1 -- 172.21.15.16:0/502971515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6341f1c40 con 0x7fd6340ce3e0 2023-12-08T08:39:14.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.487+0000 7fd63bc9b640 1 -- 172.21.15.16:0/502971515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6340ac200 con 0x7fd6340ce3e0 2023-12-08T08:39:14.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.487+0000 7fd6327fc640 1 -- 172.21.15.16:0/502971515 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd628007bb0 con 0x7fd6340ce3e0 2023-12-08T08:39:14.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.488+0000 7fd6327fc640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd62006ae70 0x7fd62006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.488+0000 7fd6327fc640 1 -- 172.21.15.16:0/502971515 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fd628089860 con 0x7fd6340ce3e0 2023-12-08T08:39:14.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.488+0000 7fd639a37640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd62006ae70 0x7fd62006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:14.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.488+0000 7fd639a37640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd62006ae70 0x7fd62006d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006fd0 tx=0x7fd62c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:14.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.492+0000 7fd6327fc640 1 -- 172.21.15.16:0/502971515 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd6340ac200 con 0x7fd6340ce3e0 2023-12-08T08:39:14.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.669+0000 7fd63bc9b640 1 -- 172.21.15.16:0/502971515 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6340ac1c0 con 0x7fd6340ce3e0 2023-12-08T08:39:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.671+0000 7fd6327fc640 1 -- 172.21.15.16:0/502971515 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7fd62801e090 con 0x7fd6340ce3e0 2023-12-08T08:39:14.673 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:14.673 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.672+0000 7fd63bc9b640 1 -- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd62006ae70 msgr2=0x7fd62006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.672+0000 7fd63bc9b640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd62006ae70 0x7fd62006d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006fd0 tx=0x7fd62c008040 comp rx=0 tx=0).stop 2023-12-08T08:39:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.672+0000 7fd63bc9b640 1 -- 172.21.15.16:0/502971515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6340ce3e0 msgr2=0x7fd6341f0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.672+0000 7fd63bc9b640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6340ce3e0 0x7fd6341f0ca0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd62800c7f0 tx=0x7fd62800ccb0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.672+0000 7fd63bc9b640 1 -- 172.21.15.16:0/502971515 shutdown_connections 2023-12-08T08:39:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.672+0000 7fd63bc9b640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd62006ae70 0x7fd62006d330 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.672+0000 7fd63bc9b640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd634140a10 0x7fd634159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.672+0000 7fd63bc9b640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6340ce3e0 0x7fd6341f0ca0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.672+0000 7fd63bc9b640 1 --2- 172.21.15.16:0/502971515 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd634071e50 0x7fd63409c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.672+0000 7fd63bc9b640 1 -- 172.21.15.16:0/502971515 >> 172.21.15.16:0/502971515 conn(0x7fd6340b09f0 msgr2=0x7fd63406d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.673+0000 7fd63bc9b640 1 -- 172.21.15.16:0/502971515 shutdown_connections 2023-12-08T08:39:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.673+0000 7fd63bc9b640 1 -- 172.21.15.16:0/502971515 wait complete. 2023-12-08T08:39:14.686 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:39:14.686 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:14.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.793+0000 7f0200f2c640 1 Processor -- start 2023-12-08T08:39:14.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.793+0000 7f0200f2c640 1 -- start start 2023-12-08T08:39:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.794+0000 7f0200f2c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f01fc160980 0x7f01fc0cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.794+0000 7f0200f2c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01fc0b1260 con 0x7f01fc06c3b0 2023-12-08T08:39:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.794+0000 7f0200f2c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01fc0b0930 con 0x7f01fc06b6e0 2023-12-08T08:39:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.794+0000 7f0200f2c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01fc0d04f0 con 0x7f01fc160980 2023-12-08T08:39:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.794+0000 7f01fa59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f01fc160980 0x7f01fc0cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.794+0000 7f01fad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f01fc06c3b0 0x7f01fc0b16f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50762/0 (socket says 172.21.15.16:50762) 2023-12-08T08:39:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.794+0000 7f01fad9d640 1 -- 172.21.15.16:0/913419415 learned_addr learned my addr 172.21.15.16:0/913419415 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.795+0000 7f01fa59c640 1 -- 172.21.15.16:0/913419415 >> v1:172.21.15.112:6789/0 conn(0x7f01fc06b6e0 legacy=0x7f01fc06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.795+0000 7f01fa59c640 1 -- 172.21.15.16:0/913419415 >> v1:172.21.15.16:6789/0 conn(0x7f01fc06c3b0 legacy=0x7f01fc0b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.795+0000 7f01f959a640 1 -- 172.21.15.16:0/913419415 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2374566977 0 0) 0x7f01fc0b1260 con 0x7f01fc06c3b0 2023-12-08T08:39:14.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.795+0000 7f01fa59c640 1 -- 172.21.15.16:0/913419415 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc0b0c10 con 0x7f01fc160980 2023-12-08T08:39:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.795+0000 7f01fa59c640 1 --2- 172.21.15.16:0/913419415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc160980 0x7f01fc0cfac0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009d90 tx=0x7f01ec02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846a5529b0532ddd server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.796+0000 7f01f959a640 1 -- 172.21.15.16:0/913419415 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec02cd00 con 0x7f01fc160980 2023-12-08T08:39:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.796+0000 7f01f959a640 1 -- 172.21.15.16:0/913419415 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec02ce60 con 0x7f01fc160980 2023-12-08T08:39:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.796+0000 7f01f959a640 1 -- 172.21.15.16:0/913419415 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec034660 con 0x7f01fc160980 2023-12-08T08:39:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.796+0000 7f0200f2c640 1 -- 172.21.15.16:0/913419415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc160980 msgr2=0x7f01fc0cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.796+0000 7f0200f2c640 1 --2- 172.21.15.16:0/913419415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc160980 0x7f01fc0cfac0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009d90 tx=0x7f01ec02bf50 comp rx=0 tx=0).stop 2023-12-08T08:39:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.796+0000 7f0200f2c640 1 -- 172.21.15.16:0/913419415 shutdown_connections 2023-12-08T08:39:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.796+0000 7f0200f2c640 1 --2- 172.21.15.16:0/913419415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc160980 0x7f01fc0cfac0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.796+0000 7f0200f2c640 1 -- 172.21.15.16:0/913419415 >> 172.21.15.16:0/913419415 conn(0x7f01fc0d0e30 msgr2=0x7f01fc0d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.796+0000 7f0200f2c640 1 -- 172.21.15.16:0/913419415 shutdown_connections 2023-12-08T08:39:14.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.796+0000 7f0200f2c640 1 -- 172.21.15.16:0/913419415 wait complete. 2023-12-08T08:39:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.798+0000 7f0200f2c640 1 Processor -- start 2023-12-08T08:39:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.798+0000 7f0200f2c640 1 -- start start 2023-12-08T08:39:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.798+0000 7f0200f2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01fc06b6e0 0x7f01fc0b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.798+0000 7f0200f2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc06c3b0 0x7f01fc0c73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.798+0000 7f0200f2c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01fc160980 0x7f01fc0af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.798+0000 7f0200f2c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0c6350 con 0x7f01fc06c3b0 2023-12-08T08:39:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.798+0000 7f0200f2c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0b6d10 con 0x7f01fc06b6e0 2023-12-08T08:39:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.798+0000 7f0200f2c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc0c65f0 con 0x7f01fc160980 2023-12-08T08:39:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.799+0000 7f01f9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc06c3b0 0x7f01fc0c73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.799+0000 7f01f9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc06c3b0 0x7f01fc0c73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35242/0 (socket says 172.21.15.16:35242) 2023-12-08T08:39:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.799+0000 7f01f9d9b640 1 -- 172.21.15.16:0/1663206374 learned_addr learned my addr 172.21.15.16:0/1663206374 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.799+0000 7f01f9d9b640 1 -- 172.21.15.16:0/1663206374 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01fc160980 msgr2=0x7f01fc0af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.799+0000 7f01f9d9b640 1 --2- 172.21.15.16:0/1663206374 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01fc160980 0x7f01fc0af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.799+0000 7f01f9d9b640 1 -- 172.21.15.16:0/1663206374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01fc06b6e0 msgr2=0x7f01fc0b67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.799+0000 7f01f9d9b640 1 --2- 172.21.15.16:0/1663206374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01fc06b6e0 0x7f01fc0b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.799+0000 7f01f9d9b640 1 -- 172.21.15.16:0/1663206374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec009a70 con 0x7f01fc06c3b0 2023-12-08T08:39:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.799+0000 7f01f9d9b640 1 --2- 172.21.15.16:0/1663206374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc06c3b0 0x7f01fc0c73b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c430 tx=0x7f01e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.799+0000 7f01f2ffd640 1 -- 172.21.15.16:0/1663206374 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4011790 con 0x7f01fc06c3b0 2023-12-08T08:39:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.799+0000 7f01f2ffd640 1 -- 172.21.15.16:0/1663206374 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e40118f0 con 0x7f01fc06c3b0 2023-12-08T08:39:14.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.799+0000 7f0200f2c640 1 -- 172.21.15.16:0/1663206374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01fc0c68d0 con 0x7f01fc06c3b0 2023-12-08T08:39:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.799+0000 7f01f2ffd640 1 -- 172.21.15.16:0/1663206374 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4011ca0 con 0x7f01fc06c3b0 2023-12-08T08:39:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.799+0000 7f0200f2c640 1 -- 172.21.15.16:0/1663206374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01fc0c6e90 con 0x7f01fc06c3b0 2023-12-08T08:39:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.800+0000 7f0200f2c640 1 -- 172.21.15.16:0/1663206374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01fc067ad0 con 0x7f01fc06c3b0 2023-12-08T08:39:14.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.801+0000 7f01f2ffd640 1 -- 172.21.15.16:0/1663206374 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f01e401c460 con 0x7f01fc06c3b0 2023-12-08T08:39:14.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.801+0000 7f01f2ffd640 1 --2- 172.21.15.16:0/1663206374 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f01dc06ae40 0x7f01dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:14.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.801+0000 7f01f2ffd640 1 -- 172.21.15.16:0/1663206374 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f01e4089910 con 0x7f01fc06c3b0 2023-12-08T08:39:14.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.801+0000 7f01fa59c640 1 --2- 172.21.15.16:0/1663206374 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f01dc06ae40 0x7f01dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:14.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.802+0000 7f01fa59c640 1 --2- 172.21.15.16:0/1663206374 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f01dc06ae40 0x7f01dc06d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f01ec036000 tx=0x7f01ec02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.805+0000 7f01f2ffd640 1 -- 172.21.15.16:0/1663206374 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01e4054670 con 0x7f01fc06c3b0 2023-12-08T08:39:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.980+0000 7f0200f2c640 1 -- 172.21.15.16:0/1663206374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01fc06dbe0 con 0x7f01fc06c3b0 2023-12-08T08:39:14.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.980+0000 7f01f2ffd640 1 -- 172.21.15.16:0/1663206374 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01e4057c90 con 0x7f01fc06c3b0 2023-12-08T08:39:14.983 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.982+0000 7f0200f2c640 1 -- 172.21.15.16:0/1663206374 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f01dc06ae40 msgr2=0x7f01dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.982+0000 7f0200f2c640 1 --2- 172.21.15.16:0/1663206374 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f01dc06ae40 0x7f01dc06d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f01ec036000 tx=0x7f01ec02e000 comp rx=0 tx=0).stop 2023-12-08T08:39:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.982+0000 7f0200f2c640 1 -- 172.21.15.16:0/1663206374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc06c3b0 msgr2=0x7f01fc0c73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.982+0000 7f0200f2c640 1 --2- 172.21.15.16:0/1663206374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc06c3b0 0x7f01fc0c73b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f01e400c430 tx=0x7f01e400c8f0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.982+0000 7f0200f2c640 1 -- 172.21.15.16:0/1663206374 shutdown_connections 2023-12-08T08:39:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.982+0000 7f0200f2c640 1 --2- 172.21.15.16:0/1663206374 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f01dc06ae40 0x7f01dc06d300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.982+0000 7f0200f2c640 1 --2- 172.21.15.16:0/1663206374 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f01fc160980 0x7f01fc0af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.982+0000 7f0200f2c640 1 --2- 172.21.15.16:0/1663206374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01fc06c3b0 0x7f01fc0c73b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.982+0000 7f0200f2c640 1 --2- 172.21.15.16:0/1663206374 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f01fc06b6e0 0x7f01fc0b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.982+0000 7f0200f2c640 1 -- 172.21.15.16:0/1663206374 >> 172.21.15.16:0/1663206374 conn(0x7f01fc0d0e30 msgr2=0x7f01fc0661f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.982+0000 7f0200f2c640 1 -- 172.21.15.16:0/1663206374 shutdown_connections 2023-12-08T08:39:14.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:14.982+0000 7f0200f2c640 1 -- 172.21.15.16:0/1663206374 wait complete. 2023-12-08T08:39:14.995 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:39:15.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.103+0000 7ff5049a9640 1 Processor -- start 2023-12-08T08:39:15.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.103+0000 7ff5049a9640 1 -- start start 2023-12-08T08:39:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.104+0000 7ff5049a9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff50006ab20 0x7ff5000ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.104+0000 7ff5049a9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50015f860 con 0x7ff50006a740 2023-12-08T08:39:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.104+0000 7ff5049a9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50015ec50 con 0x7ff5001518a0 2023-12-08T08:39:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.104+0000 7ff5049a9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff500152400 con 0x7ff50006ab20 2023-12-08T08:39:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.104+0000 7ff4fe59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff50006a740 0x7ff50006f240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50774/0 (socket says 172.21.15.16:50774) 2023-12-08T08:39:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.104+0000 7ff4fe59c640 1 -- 172.21.15.16:0/3925230724 learned_addr learned my addr 172.21.15.16:0/3925230724 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fdd9b640 1 --2- 172.21.15.16:0/3925230724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff50006ab20 0x7ff5000ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 -- 172.21.15.16:0/3925230724 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129188423 0 0) 0x7ff50015f860 con 0x7ff50006a740 2023-12-08T08:39:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 -- 172.21.15.16:0/3925230724 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4e8003520 con 0x7ff50006a740 2023-12-08T08:39:15.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 -- 172.21.15.16:0/3925230724 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 886570002 0 0) 0x7ff4e8003520 con 0x7ff50006a740 2023-12-08T08:39:15.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 -- 172.21.15.16:0/3925230724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff50006ab20 msgr2=0x7ff5000ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:15.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 --2- 172.21.15.16:0/3925230724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff50006ab20 0x7ff5000ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 -- 172.21.15.16:0/3925230724 >> v1:172.21.15.112:6789/0 conn(0x7ff5001518a0 legacy=0x7ff500151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 -- 172.21.15.16:0/3925230724 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50015ef30 con 0x7ff50006a740 2023-12-08T08:39:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 -- 172.21.15.16:0/3925230724 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7ff4f8001670 con 0x7ff50006a740 2023-12-08T08:39:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 -- 172.21.15.16:0/3925230724 >> v1:172.21.15.16:6789/0 conn(0x7ff50006a740 legacy=0x7ff50006f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 --2- 172.21.15.16:0/3925230724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e80039a0 0x7ff4e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 --2- 172.21.15.16:0/3925230724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff50006ab20 0x7ff4e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 --2- 172.21.15.16:0/3925230724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4e8004d00 0x7ff4e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 -- 172.21.15.16:0/3925230724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e800ba30 con 0x7ff4e80039a0 2023-12-08T08:39:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 -- 172.21.15.16:0/3925230724 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e800bb70 con 0x7ff4e8004d00 2023-12-08T08:39:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fd59a640 1 -- 172.21.15.16:0/3925230724 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e800bd30 con 0x7ff50006ab20 2023-12-08T08:39:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fe59c640 1 --2- 172.21.15.16:0/3925230724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e80039a0 0x7ff4e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fdd9b640 1 --2- 172.21.15.16:0/3925230724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff50006ab20 0x7ff4e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fed9d640 1 --2- 172.21.15.16:0/3925230724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4e8004d00 0x7ff4e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fe59c640 1 -- 172.21.15.16:0/3925230724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff50006ab20 msgr2=0x7ff4e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fe59c640 1 --2- 172.21.15.16:0/3925230724 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff50006ab20 0x7ff4e80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fe59c640 1 -- 172.21.15.16:0/3925230724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4e8004d00 msgr2=0x7ff4e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fe59c640 1 --2- 172.21.15.16:0/3925230724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4e8004d00 0x7ff4e8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fe59c640 1 -- 172.21.15.16:0/3925230724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e8003520 con 0x7ff4e80039a0 2023-12-08T08:39:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.105+0000 7ff4fed9d640 1 --2- 172.21.15.16:0/3925230724 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff4e8004d00 0x7ff4e8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.106+0000 7ff4fe59c640 1 --2- 172.21.15.16:0/3925230724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e80039a0 0x7ff4e8003dc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800c210 tx=0x7ff4e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545d14f584a30c99 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.106+0000 7ff4fd59a640 1 -- 172.21.15.16:0/3925230724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8005ad0 con 0x7ff4e80039a0 2023-12-08T08:39:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.106+0000 7ff4fd59a640 1 -- 172.21.15.16:0/3925230724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f8005c50 con 0x7ff4e80039a0 2023-12-08T08:39:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.106+0000 7ff4fd59a640 1 -- 172.21.15.16:0/3925230724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f802c890 con 0x7ff4e80039a0 2023-12-08T08:39:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.107+0000 7ff5049a9640 1 -- 172.21.15.16:0/3925230724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e80039a0 msgr2=0x7ff4e8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.107+0000 7ff5049a9640 1 --2- 172.21.15.16:0/3925230724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4e80039a0 0x7ff4e8003dc0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800c210 tx=0x7ff4e80057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.107+0000 7ff4fed9d640 1 -- 172.21.15.16:0/3925230724 reap_dead start 2023-12-08T08:39:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.107+0000 7ff5049a9640 1 -- 172.21.15.16:0/3925230724 shutdown_connections 2023-12-08T08:39:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.107+0000 7ff5049a9640 1 -- 172.21.15.16:0/3925230724 >> 172.21.15.16:0/3925230724 conn(0x7ff5000cfcd0 msgr2=0x7ff5000d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.107+0000 7ff5049a9640 1 -- 172.21.15.16:0/3925230724 shutdown_connections 2023-12-08T08:39:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.107+0000 7ff5049a9640 1 -- 172.21.15.16:0/3925230724 wait complete. 2023-12-08T08:39:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.108+0000 7ff5049a9640 1 Processor -- start 2023-12-08T08:39:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.108+0000 7ff5049a9640 1 -- start start 2023-12-08T08:39:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff5049a9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4e80039a0 0x7ff5001f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff5049a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff500159ec0 0x7ff50015a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff5049a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5000cdae0 0x7ff5000cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff5049a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5000cf4e0 con 0x7ff5000cdae0 2023-12-08T08:39:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff5049a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5000cf900 con 0x7ff500159ec0 2023-12-08T08:39:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff5049a9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50015a860 con 0x7ff4e80039a0 2023-12-08T08:39:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff4fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5000cdae0 0x7ff5000cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff4fed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5000cdae0 0x7ff5000cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35256/0 (socket says 172.21.15.16:35256) 2023-12-08T08:39:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff4fed9d640 1 -- 172.21.15.16:0/1879997940 learned_addr learned my addr 172.21.15.16:0/1879997940 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff4fe59c640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4e80039a0 0x7ff5001f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff4fdd9b640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff500159ec0 0x7ff50015a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff4fed9d640 1 -- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4e80039a0 msgr2=0x7ff5001f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff4fed9d640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4e80039a0 0x7ff5001f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff4fed9d640 1 -- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff500159ec0 msgr2=0x7ff50015a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff4fed9d640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff500159ec0 0x7ff50015a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff4fed9d640 1 -- 172.21.15.16:0/1879997940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5000ce860 con 0x7ff5000cdae0 2023-12-08T08:39:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff4fe59c640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4e80039a0 0x7ff5001f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.109+0000 7ff4fdd9b640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff500159ec0 0x7ff50015a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.110+0000 7ff4fed9d640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5000cdae0 0x7ff5000cdf60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff500072510 tx=0x7ff4f000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.110+0000 7ff4eeffd640 1 -- 172.21.15.16:0/1879997940 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0017070 con 0x7ff5000cdae0 2023-12-08T08:39:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.110+0000 7ff4eeffd640 1 -- 172.21.15.16:0/1879997940 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f0006b50 con 0x7ff5000cdae0 2023-12-08T08:39:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.110+0000 7ff5049a9640 1 -- 172.21.15.16:0/1879997940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5000ceb40 con 0x7ff5000cdae0 2023-12-08T08:39:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.110+0000 7ff4eeffd640 1 -- 172.21.15.16:0/1879997940 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0006d00 con 0x7ff5000cdae0 2023-12-08T08:39:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.110+0000 7ff5049a9640 1 -- 172.21.15.16:0/1879997940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5000cf100 con 0x7ff5000cdae0 2023-12-08T08:39:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.111+0000 7ff5049a9640 1 -- 172.21.15.16:0/1879997940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5000cf2c0 con 0x7ff5000cdae0 2023-12-08T08:39:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.111+0000 7ff4eeffd640 1 -- 172.21.15.16:0/1879997940 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff4f0003970 con 0x7ff5000cdae0 2023-12-08T08:39:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.112+0000 7ff4eeffd640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4e006ae70 0x7ff4e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.112+0000 7ff4eeffd640 1 -- 172.21.15.16:0/1879997940 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7ff4f0089040 con 0x7ff5000cdae0 2023-12-08T08:39:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.112+0000 7ff4fe59c640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4e006ae70 0x7ff4e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.113+0000 7ff4fe59c640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4e006ae70 0x7ff4e006d330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8034000 tx=0x7ff4f8009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.116+0000 7ff4eeffd640 1 -- 172.21.15.16:0/1879997940 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4f0053d10 con 0x7ff5000cdae0 2023-12-08T08:39:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.290+0000 7ff5049a9640 1 -- 172.21.15.16:0/1879997940 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff500067ad0 con 0x7ff5000cdae0 2023-12-08T08:39:15.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.291+0000 7ff4eeffd640 1 -- 172.21.15.16:0/1879997940 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff4f0057330 con 0x7ff5000cdae0 2023-12-08T08:39:15.293 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.293+0000 7ff5049a9640 1 -- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4e006ae70 msgr2=0x7ff4e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.293+0000 7ff5049a9640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4e006ae70 0x7ff4e006d330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8034000 tx=0x7ff4f8009030 comp rx=0 tx=0).stop 2023-12-08T08:39:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.293+0000 7ff5049a9640 1 -- 172.21.15.16:0/1879997940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5000cdae0 msgr2=0x7ff5000cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.293+0000 7ff5049a9640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5000cdae0 0x7ff5000cdf60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff500072510 tx=0x7ff4f000eb90 comp rx=0 tx=0).stop 2023-12-08T08:39:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.293+0000 7ff5049a9640 1 -- 172.21.15.16:0/1879997940 shutdown_connections 2023-12-08T08:39:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.293+0000 7ff5049a9640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff4e006ae70 0x7ff4e006d330 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.293+0000 7ff5049a9640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5000cdae0 0x7ff5000cdf60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.293+0000 7ff5049a9640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff500159ec0 0x7ff50015a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.293+0000 7ff5049a9640 1 --2- 172.21.15.16:0/1879997940 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff4e80039a0 0x7ff5001f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.293+0000 7ff5049a9640 1 -- 172.21.15.16:0/1879997940 >> 172.21.15.16:0/1879997940 conn(0x7ff5000cfcd0 msgr2=0x7ff5000ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.293+0000 7ff5049a9640 1 -- 172.21.15.16:0/1879997940 shutdown_connections 2023-12-08T08:39:15.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.293+0000 7ff5049a9640 1 -- 172.21.15.16:0/1879997940 wait complete. 2023-12-08T08:39:15.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:39:15.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.413+0000 7faf5adc9640 1 Processor -- start 2023-12-08T08:39:15.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.413+0000 7faf5adc9640 1 -- start start 2023-12-08T08:39:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.414+0000 7faf5adc9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7faf540b0af0 0x7faf540b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.414+0000 7faf5adc9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf540cef90 0x7faf540cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.414+0000 7faf5adc9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf540b2280 con 0x7faf5415d4f0 2023-12-08T08:39:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.414+0000 7faf5adc9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf5406bb40 con 0x7faf540cef90 2023-12-08T08:39:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.414+0000 7faf5adc9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf540b1430 con 0x7faf540b0af0 2023-12-08T08:39:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.414+0000 7faf59366640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf540cef90 0x7faf540cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.414+0000 7faf59366640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf540cef90 0x7faf540cf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35266/0 (socket says 172.21.15.16:35266) 2023-12-08T08:39:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.414+0000 7faf59366640 1 -- 172.21.15.16:0/459511646 learned_addr learned my addr 172.21.15.16:0/459511646 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.414+0000 7faf53fff640 1 --2- 172.21.15.16:0/459511646 >> v2:172.21.15.112:3300/0 conn(0x7faf540b0af0 0x7faf540b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.414+0000 7faf59366640 1 -- 172.21.15.16:0/459511646 >> v2:172.21.15.112:3300/0 conn(0x7faf540b0af0 msgr2=0x7faf540b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.414+0000 7faf59366640 1 --2- 172.21.15.16:0/459511646 >> v2:172.21.15.112:3300/0 conn(0x7faf540b0af0 0x7faf540b0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.414+0000 7faf59366640 1 -- 172.21.15.16:0/459511646 >> v1:172.21.15.112:6789/0 conn(0x7faf5415d4f0 legacy=0x7faf540b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:15.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.414+0000 7faf59366640 1 -- 172.21.15.16:0/459511646 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf540b2710 con 0x7faf540cef90 2023-12-08T08:39:15.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.415+0000 7faf59366640 1 --2- 172.21.15.16:0/459511646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf540cef90 0x7faf540cf400 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7faf54071250 tx=0x7faf3c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f437a2fb11cd7f server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.415+0000 7faf537fe640 1 -- 172.21.15.16:0/459511646 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c031870 con 0x7faf540cef90 2023-12-08T08:39:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.415+0000 7faf537fe640 1 -- 172.21.15.16:0/459511646 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c0319d0 con 0x7faf540cef90 2023-12-08T08:39:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.415+0000 7faf537fe640 1 -- 172.21.15.16:0/459511646 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c031d60 con 0x7faf540cef90 2023-12-08T08:39:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.416+0000 7faf5adc9640 1 -- 172.21.15.16:0/459511646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf540cef90 msgr2=0x7faf540cf400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.416+0000 7faf5adc9640 1 --2- 172.21.15.16:0/459511646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf540cef90 0x7faf540cf400 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7faf54071250 tx=0x7faf3c02b910 comp rx=0 tx=0).stop 2023-12-08T08:39:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.416+0000 7faf5adc9640 1 -- 172.21.15.16:0/459511646 shutdown_connections 2023-12-08T08:39:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.416+0000 7faf5adc9640 1 --2- 172.21.15.16:0/459511646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf540cef90 0x7faf540cf400 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.416+0000 7faf5adc9640 1 --2- 172.21.15.16:0/459511646 >> v2:172.21.15.112:3300/0 conn(0x7faf540b0af0 0x7faf540b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.416+0000 7faf5adc9640 1 -- 172.21.15.16:0/459511646 >> 172.21.15.16:0/459511646 conn(0x7faf540d0e00 msgr2=0x7faf540d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.416+0000 7faf5adc9640 1 -- 172.21.15.16:0/459511646 shutdown_connections 2023-12-08T08:39:15.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.416+0000 7faf5adc9640 1 -- 172.21.15.16:0/459511646 wait complete. 2023-12-08T08:39:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.417+0000 7faf5adc9640 1 Processor -- start 2023-12-08T08:39:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.417+0000 7faf5adc9640 1 -- start start 2023-12-08T08:39:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf5adc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf540b0af0 0x7faf5415b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf5adc9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf540cef90 0x7faf540c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf5adc9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf5415d4f0 0x7faf540c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf5adc9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5415a400 con 0x7faf540b0af0 2023-12-08T08:39:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf5adc9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5415a6e0 con 0x7faf5415d4f0 2023-12-08T08:39:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf5adc9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5415a9c0 con 0x7faf540cef90 2023-12-08T08:39:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf58b65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf540b0af0 0x7faf5415b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf58b65640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf540b0af0 0x7faf5415b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35278/0 (socket says 172.21.15.16:35278) 2023-12-08T08:39:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf58b65640 1 -- 172.21.15.16:0/1610023341 learned_addr learned my addr 172.21.15.16:0/1610023341 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf53fff640 1 --2- 172.21.15.16:0/1610023341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf540cef90 0x7faf540c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf59366640 1 --2- 172.21.15.16:0/1610023341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf5415d4f0 0x7faf540c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf58b65640 1 -- 172.21.15.16:0/1610023341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf540cef90 msgr2=0x7faf540c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf58b65640 1 --2- 172.21.15.16:0/1610023341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf540cef90 0x7faf540c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf58b65640 1 -- 172.21.15.16:0/1610023341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf5415d4f0 msgr2=0x7faf540c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf58b65640 1 --2- 172.21.15.16:0/1610023341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf5415d4f0 0x7faf540c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.418+0000 7faf58b65640 1 -- 172.21.15.16:0/1610023341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0094d0 con 0x7faf540b0af0 2023-12-08T08:39:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.419+0000 7faf58b65640 1 --2- 172.21.15.16:0/1610023341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf540b0af0 0x7faf5415b4a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00b3e0 tx=0x7faf4c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.419+0000 7faf517fa640 1 -- 172.21.15.16:0/1610023341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c011830 con 0x7faf540b0af0 2023-12-08T08:39:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.419+0000 7faf517fa640 1 -- 172.21.15.16:0/1610023341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4c0119b0 con 0x7faf540b0af0 2023-12-08T08:39:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.419+0000 7faf5adc9640 1 -- 172.21.15.16:0/1610023341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5415aca0 con 0x7faf540b0af0 2023-12-08T08:39:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.419+0000 7faf517fa640 1 -- 172.21.15.16:0/1610023341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c011d60 con 0x7faf540b0af0 2023-12-08T08:39:15.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.419+0000 7faf5adc9640 1 -- 172.21.15.16:0/1610023341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf54159e90 con 0x7faf540b0af0 2023-12-08T08:39:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.420+0000 7faf5adc9640 1 -- 172.21.15.16:0/1610023341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5415b1c0 con 0x7faf540b0af0 2023-12-08T08:39:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.420+0000 7faf517fa640 1 -- 172.21.15.16:0/1610023341 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf4c01a400 con 0x7faf540b0af0 2023-12-08T08:39:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.421+0000 7faf517fa640 1 --2- 172.21.15.16:0/1610023341 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7faf3406ae40 0x7faf3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.421+0000 7faf517fa640 1 -- 172.21.15.16:0/1610023341 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7faf4c089b70 con 0x7faf540b0af0 2023-12-08T08:39:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.421+0000 7faf53fff640 1 --2- 172.21.15.16:0/1610023341 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7faf3406ae40 0x7faf3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.421+0000 7faf53fff640 1 --2- 172.21.15.16:0/1610023341 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7faf3406ae40 0x7faf3406d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faf440070b0 tx=0x7faf44007040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:15.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.425+0000 7faf517fa640 1 -- 172.21.15.16:0/1610023341 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf5415b1c0 con 0x7faf540b0af0 2023-12-08T08:39:15.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.601+0000 7faf5adc9640 1 -- 172.21.15.16:0/1610023341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faf5415b1c0 con 0x7faf540b0af0 2023-12-08T08:39:15.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.601+0000 7faf517fa640 1 -- 172.21.15.16:0/1610023341 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7faf5415b1c0 con 0x7faf540b0af0 2023-12-08T08:39:15.603 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:39:15.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.603+0000 7faf5adc9640 1 -- 172.21.15.16:0/1610023341 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7faf3406ae40 msgr2=0x7faf3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.603+0000 7faf5adc9640 1 --2- 172.21.15.16:0/1610023341 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7faf3406ae40 0x7faf3406d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faf440070b0 tx=0x7faf44007040 comp rx=0 tx=0).stop 2023-12-08T08:39:15.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.603+0000 7faf5adc9640 1 -- 172.21.15.16:0/1610023341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf540b0af0 msgr2=0x7faf5415b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.603+0000 7faf5adc9640 1 --2- 172.21.15.16:0/1610023341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf540b0af0 0x7faf5415b4a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00b3e0 tx=0x7faf4c00b8a0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.603+0000 7faf5adc9640 1 -- 172.21.15.16:0/1610023341 shutdown_connections 2023-12-08T08:39:15.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.603+0000 7faf5adc9640 1 --2- 172.21.15.16:0/1610023341 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7faf3406ae40 0x7faf3406d300 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.603+0000 7faf5adc9640 1 --2- 172.21.15.16:0/1610023341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7faf5415d4f0 0x7faf540c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.603+0000 7faf5adc9640 1 --2- 172.21.15.16:0/1610023341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7faf540cef90 0x7faf540c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.603+0000 7faf5adc9640 1 --2- 172.21.15.16:0/1610023341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf540b0af0 0x7faf5415b4a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.603+0000 7faf5adc9640 1 -- 172.21.15.16:0/1610023341 >> 172.21.15.16:0/1610023341 conn(0x7faf540d0e00 msgr2=0x7faf540af020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:15.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.603+0000 7faf5adc9640 1 -- 172.21.15.16:0/1610023341 shutdown_connections 2023-12-08T08:39:15.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.603+0000 7faf5adc9640 1 -- 172.21.15.16:0/1610023341 wait complete. 2023-12-08T08:39:15.616 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:39:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.724+0000 7f19676b2640 1 Processor -- start 2023-12-08T08:39:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.724+0000 7f19676b2640 1 -- start start 2023-12-08T08:39:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.725+0000 7f19676b2640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f196015d030 0x7f1960071940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.725+0000 7f19676b2640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196006be30 con 0x7f19600ac880 2023-12-08T08:39:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.725+0000 7f19676b2640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1960070e10 con 0x7f1960070a00 2023-12-08T08:39:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.725+0000 7f19676b2640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f196015d410 con 0x7f196015d030 2023-12-08T08:39:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.725+0000 7f1964c4d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f19600ac880 0x7f19600702f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50776/0 (socket says 172.21.15.16:50776) 2023-12-08T08:39:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.725+0000 7f1964c4d640 1 -- 172.21.15.16:0/3148112299 learned_addr learned my addr 172.21.15.16:0/3148112299 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.725+0000 7f196544e640 1 --2- 172.21.15.16:0/3148112299 >> v2:172.21.15.112:3300/0 conn(0x7f196015d030 0x7f1960071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.725+0000 7f195ffff640 1 -- 172.21.15.16:0/3148112299 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386326720 0 0) 0x7f196006be30 con 0x7f19600ac880 2023-12-08T08:39:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.725+0000 7f195ffff640 1 -- 172.21.15.16:0/3148112299 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1948003520 con 0x7f19600ac880 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f195ffff640 1 -- 172.21.15.16:0/3148112299 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3207989311 0 0) 0x7f1948003520 con 0x7f19600ac880 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f195ffff640 1 -- 172.21.15.16:0/3148112299 >> v2:172.21.15.112:3300/0 conn(0x7f196015d030 msgr2=0x7f1960071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f195ffff640 1 --2- 172.21.15.16:0/3148112299 >> v2:172.21.15.112:3300/0 conn(0x7f196015d030 0x7f1960071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f195ffff640 1 -- 172.21.15.16:0/3148112299 >> v1:172.21.15.112:6789/0 conn(0x7f1960070a00 legacy=0x7f1960166bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f195ffff640 1 -- 172.21.15.16:0/3148112299 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1960071090 con 0x7f19600ac880 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f195ffff640 1 -- 172.21.15.16:0/3148112299 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f1954001670 con 0x7f19600ac880 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f195ffff640 1 -- 172.21.15.16:0/3148112299 >> v1:172.21.15.16:6789/0 conn(0x7f19600ac880 legacy=0x7f19600702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f195ffff640 1 --2- 172.21.15.16:0/3148112299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1948003970 0x7f1948003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f195ffff640 1 --2- 172.21.15.16:0/3148112299 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19480047c0 0x7f1948004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f195ffff640 1 --2- 172.21.15.16:0/3148112299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1948005180 0x7f194800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f195ffff640 1 -- 172.21.15.16:0/3148112299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196015d410 con 0x7f1948003970 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f195ffff640 1 -- 172.21.15.16:0/3148112299 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194800c020 con 0x7f1948005180 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f195ffff640 1 -- 172.21.15.16:0/3148112299 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194800c1e0 con 0x7f19480047c0 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f196544e640 1 --2- 172.21.15.16:0/3148112299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1948003970 0x7f1948003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f1965c4f640 1 --2- 172.21.15.16:0/3148112299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1948005180 0x7f194800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f1964c4d640 1 --2- 172.21.15.16:0/3148112299 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19480047c0 0x7f1948004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f196544e640 1 -- 172.21.15.16:0/3148112299 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19480047c0 msgr2=0x7f1948004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f196544e640 1 --2- 172.21.15.16:0/3148112299 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19480047c0 0x7f1948004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f196544e640 1 -- 172.21.15.16:0/3148112299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1948005180 msgr2=0x7f194800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f196544e640 1 --2- 172.21.15.16:0/3148112299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1948005180 0x7f194800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f196544e640 1 -- 172.21.15.16:0/3148112299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1948003520 con 0x7f1948003970 2023-12-08T08:39:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f1965c4f640 1 --2- 172.21.15.16:0/3148112299 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1948005180 0x7f194800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.726+0000 7f1965c4f640 1 -- 172.21.15.16:0/3148112299 reap_dead start 2023-12-08T08:39:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.727+0000 7f196544e640 1 --2- 172.21.15.16:0/3148112299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1948003970 0x7f1948003d90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1958009ac0 tx=0x7f19480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3e6496cb861f2a server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.727+0000 7f195ffff640 1 -- 172.21.15.16:0/3148112299 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195802c6b0 con 0x7f1948003970 2023-12-08T08:39:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.727+0000 7f195ffff640 1 -- 172.21.15.16:0/3148112299 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195802c810 con 0x7f1948003970 2023-12-08T08:39:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.727+0000 7f195ffff640 1 -- 172.21.15.16:0/3148112299 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195802cba0 con 0x7f1948003970 2023-12-08T08:39:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.728+0000 7f19676b2640 1 -- 172.21.15.16:0/3148112299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1948003970 msgr2=0x7f1948003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.728+0000 7f19676b2640 1 --2- 172.21.15.16:0/3148112299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1948003970 0x7f1948003d90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1958009ac0 tx=0x7f19480057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.728+0000 7f19676b2640 1 -- 172.21.15.16:0/3148112299 shutdown_connections 2023-12-08T08:39:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.728+0000 7f19676b2640 1 --2- 172.21.15.16:0/3148112299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1948003970 0x7f1948003d90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.728+0000 7f19676b2640 1 -- 172.21.15.16:0/3148112299 >> 172.21.15.16:0/3148112299 conn(0x7f19600b0850 msgr2=0x7f19600b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.728+0000 7f19676b2640 1 -- 172.21.15.16:0/3148112299 shutdown_connections 2023-12-08T08:39:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.728+0000 7f19676b2640 1 -- 172.21.15.16:0/3148112299 wait complete. 2023-12-08T08:39:15.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.729+0000 7f19676b2640 1 Processor -- start 2023-12-08T08:39:15.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.729+0000 7f19676b2640 1 -- start start 2023-12-08T08:39:15.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f19676b2640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1948003970 0x7f1960147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f19676b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601f2970 0x7f196014ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f19676b2640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19600afbe0 0x7f19600b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f19676b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19600b05a0 con 0x7f19601f2970 2023-12-08T08:39:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f19676b2640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19600c3df0 con 0x7f1948003970 2023-12-08T08:39:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f19676b2640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196014b320 con 0x7f19600afbe0 2023-12-08T08:39:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f1964c4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601f2970 0x7f196014ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f1964c4d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601f2970 0x7f196014ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35294/0 (socket says 172.21.15.16:35294) 2023-12-08T08:39:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f1964c4d640 1 -- 172.21.15.16:0/1937796097 learned_addr learned my addr 172.21.15.16:0/1937796097 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f196544e640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1948003970 0x7f1960147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f1964c4d640 1 -- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19600afbe0 msgr2=0x7f19600b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f1965c4f640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19600afbe0 0x7f19600b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f1964c4d640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19600afbe0 0x7f19600b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f1964c4d640 1 -- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1948003970 msgr2=0x7f1960147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f1964c4d640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1948003970 0x7f1960147cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f1964c4d640 1 -- 172.21.15.16:0/1937796097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19601f2ec0 con 0x7f19601f2970 2023-12-08T08:39:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f196544e640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1948003970 0x7f1960147cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f1965c4f640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19600afbe0 0x7f19600b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f1964c4d640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601f2970 0x7f196014ade0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f195400e7f0 tx=0x7f195400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f195dffb640 1 -- 172.21.15.16:0/1937796097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195400c8d0 con 0x7f19601f2970 2023-12-08T08:39:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.730+0000 7f195dffb640 1 -- 172.21.15.16:0/1937796097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195400ca50 con 0x7f19601f2970 2023-12-08T08:39:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.731+0000 7f19676b2640 1 -- 172.21.15.16:0/1937796097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19601f31a0 con 0x7f19601f2970 2023-12-08T08:39:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.731+0000 7f195dffb640 1 -- 172.21.15.16:0/1937796097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195400ce00 con 0x7f19601f2970 2023-12-08T08:39:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.731+0000 7f19676b2640 1 -- 172.21.15.16:0/1937796097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19601f3760 con 0x7f19601f2970 2023-12-08T08:39:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.732+0000 7f19676b2640 1 -- 172.21.15.16:0/1937796097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19601f3920 con 0x7f19601f2970 2023-12-08T08:39:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.732+0000 7f195dffb640 1 -- 172.21.15.16:0/1937796097 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f195401d460 con 0x7f19601f2970 2023-12-08T08:39:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.732+0000 7f195dffb640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f194c06ae70 0x7f194c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.733+0000 7f195dffb640 1 -- 172.21.15.16:0/1937796097 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f19540899d0 con 0x7f19601f2970 2023-12-08T08:39:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.733+0000 7f196544e640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f194c06ae70 0x7f194c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:15.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.733+0000 7f196544e640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f194c06ae70 0x7f194c06d330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1958036000 tx=0x7f1958030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:15.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.737+0000 7f195dffb640 1 -- 172.21.15.16:0/1937796097 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f19540546a0 con 0x7f19601f2970 2023-12-08T08:39:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.912+0000 7f19676b2640 1 -- 172.21.15.16:0/1937796097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1960067430 con 0x7f19601f2970 2023-12-08T08:39:15.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.913+0000 7f195dffb640 1 -- 172.21.15.16:0/1937796097 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1954057cc0 con 0x7f19601f2970 2023-12-08T08:39:15.915 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.915+0000 7f19676b2640 1 -- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f194c06ae70 msgr2=0x7f194c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.915+0000 7f19676b2640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f194c06ae70 0x7f194c06d330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1958036000 tx=0x7f1958030000 comp rx=0 tx=0).stop 2023-12-08T08:39:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.915+0000 7f19676b2640 1 -- 172.21.15.16:0/1937796097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601f2970 msgr2=0x7f196014ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.915+0000 7f19676b2640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601f2970 0x7f196014ade0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f195400e7f0 tx=0x7f195400ecb0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.915+0000 7f19676b2640 1 -- 172.21.15.16:0/1937796097 shutdown_connections 2023-12-08T08:39:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.915+0000 7f19676b2640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f194c06ae70 0x7f194c06d330 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.915+0000 7f19676b2640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f19600afbe0 0x7f19600b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.915+0000 7f19676b2640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19601f2970 0x7f196014ade0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.915+0000 7f19676b2640 1 --2- 172.21.15.16:0/1937796097 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1948003970 0x7f1960147cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:15.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.915+0000 7f19676b2640 1 -- 172.21.15.16:0/1937796097 >> 172.21.15.16:0/1937796097 conn(0x7f19600b0850 msgr2=0x7f19600b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:15.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.915+0000 7f19676b2640 1 -- 172.21.15.16:0/1937796097 shutdown_connections 2023-12-08T08:39:15.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:15.915+0000 7f19676b2640 1 -- 172.21.15.16:0/1937796097 wait complete. 2023-12-08T08:39:15.928 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-08T08:39:16.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.033+0000 7f960469b640 1 Processor -- start 2023-12-08T08:39:16.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.033+0000 7f960469b640 1 -- start start 2023-12-08T08:39:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.034+0000 7f960469b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95fc1608a0 0x7f95fc0d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.034+0000 7f960469b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95fc0d20b0 con 0x7f95fc0b0cf0 2023-12-08T08:39:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.034+0000 7f960469b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95fc0d1b80 con 0x7f95fc0b1d70 2023-12-08T08:39:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.034+0000 7f960469b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc0d0620 con 0x7f95fc1608a0 2023-12-08T08:39:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.035+0000 7f9602437640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f95fc1608a0 0x7f95fc0d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.035+0000 7f9602c38640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f95fc0b0cf0 0x7f95fc0cdc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50784/0 (socket says 172.21.15.16:50784) 2023-12-08T08:39:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.035+0000 7f9602c38640 1 -- 172.21.15.16:0/3617989761 learned_addr learned my addr 172.21.15.16:0/3617989761 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.035+0000 7f9602437640 1 -- 172.21.15.16:0/3617989761 >> v1:172.21.15.112:6789/0 conn(0x7f95fc0b1d70 legacy=0x7f95fc0d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.035+0000 7f9602437640 1 -- 172.21.15.16:0/3617989761 >> v1:172.21.15.16:6789/0 conn(0x7f95fc0b0cf0 legacy=0x7f95fc0cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.035+0000 7f9602437640 1 -- 172.21.15.16:0/3617989761 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc0d1dd0 con 0x7f95fc1608a0 2023-12-08T08:39:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.035+0000 7f9601435640 1 -- 172.21.15.16:0/3617989761 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390971725 0 0) 0x7f95fc0d20b0 con 0x7f95fc0b0cf0 2023-12-08T08:39:16.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.036+0000 7f9602437640 1 --2- 172.21.15.16:0/3617989761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95fc1608a0 0x7f95fc0d0f30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f95f80097a0 tx=0x7f95f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae025a878138d35 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:16.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.036+0000 7f9601435640 1 -- 172.21.15.16:0/3617989761 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8031830 con 0x7f95fc1608a0 2023-12-08T08:39:16.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.036+0000 7f9601435640 1 -- 172.21.15.16:0/3617989761 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f8031990 con 0x7f95fc1608a0 2023-12-08T08:39:16.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.036+0000 7f9601435640 1 -- 172.21.15.16:0/3617989761 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f8031d20 con 0x7f95fc1608a0 2023-12-08T08:39:16.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.036+0000 7f960469b640 1 -- 172.21.15.16:0/3617989761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95fc1608a0 msgr2=0x7f95fc0d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.036+0000 7f960469b640 1 --2- 172.21.15.16:0/3617989761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95fc1608a0 0x7f95fc0d0f30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f95f80097a0 tx=0x7f95f802b8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.036+0000 7f960469b640 1 -- 172.21.15.16:0/3617989761 shutdown_connections 2023-12-08T08:39:16.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.036+0000 7f960469b640 1 --2- 172.21.15.16:0/3617989761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95fc1608a0 0x7f95fc0d0f30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.036+0000 7f960469b640 1 -- 172.21.15.16:0/3617989761 >> 172.21.15.16:0/3617989761 conn(0x7f95fc0d3030 msgr2=0x7f95fc0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:16.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.036+0000 7f960469b640 1 -- 172.21.15.16:0/3617989761 shutdown_connections 2023-12-08T08:39:16.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.037+0000 7f960469b640 1 -- 172.21.15.16:0/3617989761 wait complete. 2023-12-08T08:39:16.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.038+0000 7f960469b640 1 Processor -- start 2023-12-08T08:39:16.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.038+0000 7f960469b640 1 -- start start 2023-12-08T08:39:16.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.038+0000 7f960469b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95fc0b0cf0 0x7f95fc15c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.038+0000 7f960469b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95fc0b1d70 0x7f95fc0b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.038+0000 7f960469b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc1608a0 0x7f95fc0b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.038+0000 7f960469b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0b5c20 con 0x7f95fc0b1d70 2023-12-08T08:39:16.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.038+0000 7f960469b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0c8b10 con 0x7f95fc1608a0 2023-12-08T08:39:16.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.038+0000 7f960469b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc0c7f00 con 0x7f95fc0b0cf0 2023-12-08T08:39:16.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.038+0000 7f9601c36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95fc0b1d70 0x7f95fc0b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.038+0000 7f9601c36640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95fc0b1d70 0x7f95fc0b7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35310/0 (socket says 172.21.15.16:35310) 2023-12-08T08:39:16.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.038+0000 7f9601c36640 1 -- 172.21.15.16:0/1031354268 learned_addr learned my addr 172.21.15.16:0/1031354268 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:16.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.039+0000 7f9602437640 1 --2- 172.21.15.16:0/1031354268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95fc0b0cf0 0x7f95fc15c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.039+0000 7f9602c38640 1 --2- 172.21.15.16:0/1031354268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc1608a0 0x7f95fc0b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.039+0000 7f9601c36640 1 -- 172.21.15.16:0/1031354268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95fc0b0cf0 msgr2=0x7f95fc15c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.039+0000 7f9601c36640 1 --2- 172.21.15.16:0/1031354268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95fc0b0cf0 0x7f95fc15c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.039+0000 7f9601c36640 1 -- 172.21.15.16:0/1031354268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc1608a0 msgr2=0x7f95fc0b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.039+0000 7f9601c36640 1 --2- 172.21.15.16:0/1031354268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc1608a0 0x7f95fc0b81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.039+0000 7f9601c36640 1 -- 172.21.15.16:0/1031354268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f8009400 con 0x7f95fc0b1d70 2023-12-08T08:39:16.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.039+0000 7f9601c36640 1 --2- 172.21.15.16:0/1031354268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95fc0b1d70 0x7f95fc0b7cb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c450 tx=0x7f95ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:16.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.039+0000 7f95f2ffd640 1 -- 172.21.15.16:0/1031354268 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec0052d0 con 0x7f95fc0b1d70 2023-12-08T08:39:16.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.039+0000 7f95f2ffd640 1 -- 172.21.15.16:0/1031354268 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ec005430 con 0x7f95fc0b1d70 2023-12-08T08:39:16.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.039+0000 7f960469b640 1 -- 172.21.15.16:0/1031354268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc0c81e0 con 0x7f95fc0b1d70 2023-12-08T08:39:16.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.039+0000 7f95f2ffd640 1 -- 172.21.15.16:0/1031354268 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec012870 con 0x7f95fc0b1d70 2023-12-08T08:39:16.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.039+0000 7f960469b640 1 -- 172.21.15.16:0/1031354268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc0c87a0 con 0x7f95fc0b1d70 2023-12-08T08:39:16.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.040+0000 7f960469b640 1 -- 172.21.15.16:0/1031354268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95fc067430 con 0x7f95fc0b1d70 2023-12-08T08:39:16.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.041+0000 7f95f2ffd640 1 -- 172.21.15.16:0/1031354268 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f95ec0129d0 con 0x7f95fc0b1d70 2023-12-08T08:39:16.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.041+0000 7f95f2ffd640 1 --2- 172.21.15.16:0/1031354268 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f95e006ae40 0x7f95e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.041+0000 7f95f2ffd640 1 -- 172.21.15.16:0/1031354268 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f95ec089360 con 0x7f95fc0b1d70 2023-12-08T08:39:16.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.041+0000 7f9602437640 1 --2- 172.21.15.16:0/1031354268 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f95e006ae40 0x7f95e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.042+0000 7f9602437640 1 --2- 172.21.15.16:0/1031354268 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f95e006ae40 0x7f95e006d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f95f8036000 tx=0x7f95f803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:16.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.045+0000 7f95f2ffd640 1 -- 172.21.15.16:0/1031354268 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95ec054170 con 0x7f95fc0b1d70 2023-12-08T08:39:16.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.235+0000 7f960469b640 1 -- 172.21.15.16:0/1031354268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f95fc0cae60 con 0x7f95fc0b1d70 2023-12-08T08:39:16.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.235+0000 7f95f2ffd640 1 -- 172.21.15.16:0/1031354268 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f95ec057790 con 0x7f95fc0b1d70 2023-12-08T08:39:16.237 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-12-08T08:39:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.237+0000 7f960469b640 1 -- 172.21.15.16:0/1031354268 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f95e006ae40 msgr2=0x7f95e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.237+0000 7f960469b640 1 --2- 172.21.15.16:0/1031354268 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f95e006ae40 0x7f95e006d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f95f8036000 tx=0x7f95f803c040 comp rx=0 tx=0).stop 2023-12-08T08:39:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.237+0000 7f960469b640 1 -- 172.21.15.16:0/1031354268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95fc0b1d70 msgr2=0x7f95fc0b7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.237+0000 7f960469b640 1 --2- 172.21.15.16:0/1031354268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95fc0b1d70 0x7f95fc0b7cb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f95ec00c450 tx=0x7f95ec00c910 comp rx=0 tx=0).stop 2023-12-08T08:39:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.237+0000 7f960469b640 1 -- 172.21.15.16:0/1031354268 shutdown_connections 2023-12-08T08:39:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.237+0000 7f960469b640 1 --2- 172.21.15.16:0/1031354268 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f95e006ae40 0x7f95e006d300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.237+0000 7f960469b640 1 --2- 172.21.15.16:0/1031354268 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f95fc1608a0 0x7f95fc0b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.237+0000 7f960469b640 1 --2- 172.21.15.16:0/1031354268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f95fc0b1d70 0x7f95fc0b7cb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.237+0000 7f960469b640 1 --2- 172.21.15.16:0/1031354268 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f95fc0b0cf0 0x7f95fc15c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.237+0000 7f960469b640 1 -- 172.21.15.16:0/1031354268 >> 172.21.15.16:0/1031354268 conn(0x7f95fc0d3030 msgr2=0x7f95fc0d22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.237+0000 7f960469b640 1 -- 172.21.15.16:0/1031354268 shutdown_connections 2023-12-08T08:39:16.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.237+0000 7f960469b640 1 -- 172.21.15.16:0/1031354268 wait complete. 2023-12-08T08:39:16.250 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:39:16.250 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T08:39:16.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:16.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.374+0000 7f7444f2c640 1 Processor -- start 2023-12-08T08:39:16.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.374+0000 7f7444f2c640 1 -- start start 2023-12-08T08:39:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.375+0000 7f7444f2c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f74400b6ac0 0x7f74400cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.375+0000 7f7444f2c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f74400cd660 0x7f74400cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.375+0000 7f7444f2c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74400d0070 con 0x7f74400cdff0 2023-12-08T08:39:16.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.375+0000 7f7444f2c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7440160cb0 con 0x7f74400b6ac0 2023-12-08T08:39:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.375+0000 7f7444f2c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74400cd200 con 0x7f74400cd660 2023-12-08T08:39:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.375+0000 7f743e59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f74400b6ac0 0x7f74400cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.375+0000 7f743e59c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f74400b6ac0 0x7f74400cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35326/0 (socket says 172.21.15.16:35326) 2023-12-08T08:39:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.375+0000 7f743e59c640 1 -- 172.21.15.16:0/2952144 learned_addr learned my addr 172.21.15.16:0/2952144 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.375+0000 7f743dd9b640 1 --2- 172.21.15.16:0/2952144 >> v2:172.21.15.112:3300/0 conn(0x7f74400cd660 0x7f74400cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.376+0000 7f743e59c640 1 -- 172.21.15.16:0/2952144 >> v2:172.21.15.112:3300/0 conn(0x7f74400cd660 msgr2=0x7f74400cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.376+0000 7f743e59c640 1 --2- 172.21.15.16:0/2952144 >> v2:172.21.15.112:3300/0 conn(0x7f74400cd660 0x7f74400cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.376+0000 7f743e59c640 1 -- 172.21.15.16:0/2952144 >> v1:172.21.15.112:6789/0 conn(0x7f74400cdff0 legacy=0x7f744006b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:16.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.376+0000 7f743e59c640 1 -- 172.21.15.16:0/2952144 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74400ce440 con 0x7f74400b6ac0 2023-12-08T08:39:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.376+0000 7f743e59c640 1 --2- 172.21.15.16:0/2952144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74400b6ac0 0x7f74400cfb30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f744006e0d0 tx=0x7f743002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c80231faf049aac server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.376+0000 7f743d59a640 1 -- 172.21.15.16:0/2952144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743002c960 con 0x7f74400b6ac0 2023-12-08T08:39:16.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.377+0000 7f743d59a640 1 -- 172.21.15.16:0/2952144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743002cac0 con 0x7f74400b6ac0 2023-12-08T08:39:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.377+0000 7f743d59a640 1 -- 172.21.15.16:0/2952144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743002ce50 con 0x7f74400b6ac0 2023-12-08T08:39:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.377+0000 7f7444f2c640 1 -- 172.21.15.16:0/2952144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74400b6ac0 msgr2=0x7f74400cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.377+0000 7f7444f2c640 1 --2- 172.21.15.16:0/2952144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74400b6ac0 0x7f74400cfb30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f744006e0d0 tx=0x7f743002bdc0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.377+0000 7f7444f2c640 1 -- 172.21.15.16:0/2952144 shutdown_connections 2023-12-08T08:39:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.377+0000 7f7444f2c640 1 --2- 172.21.15.16:0/2952144 >> v2:172.21.15.112:3300/0 conn(0x7f74400cd660 0x7f74400cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.377+0000 7f7444f2c640 1 --2- 172.21.15.16:0/2952144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74400b6ac0 0x7f74400cfb30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.377+0000 7f7444f2c640 1 -- 172.21.15.16:0/2952144 >> 172.21.15.16:0/2952144 conn(0x7f74400d0de0 msgr2=0x7f74400d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.377+0000 7f7444f2c640 1 -- 172.21.15.16:0/2952144 shutdown_connections 2023-12-08T08:39:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.377+0000 7f7444f2c640 1 -- 172.21.15.16:0/2952144 wait complete. 2023-12-08T08:39:16.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.379+0000 7f7444f2c640 1 Processor -- start 2023-12-08T08:39:16.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.379+0000 7f7444f2c640 1 -- start start 2023-12-08T08:39:16.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.379+0000 7f7444f2c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74400b6ac0 0x7f74401f0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.379+0000 7f7444f2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74400cd660 0x7f7440158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.379+0000 7f7444f2c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74400cdff0 0x7f744015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.379+0000 7f7444f2c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7440159670 con 0x7f74400cd660 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.379+0000 7f7444f2c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7440159950 con 0x7f74400b6ac0 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.379+0000 7f7444f2c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7440157100 con 0x7f74400cdff0 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.379+0000 7f743dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74400cd660 0x7f7440158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.379+0000 7f743dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74400cd660 0x7f7440158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35336/0 (socket says 172.21.15.16:35336) 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.379+0000 7f743dd9b640 1 -- 172.21.15.16:0/348392540 learned_addr learned my addr 172.21.15.16:0/348392540 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f743e59c640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74400b6ac0 0x7f74401f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f743ed9d640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74400cdff0 0x7f744015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f743dd9b640 1 -- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74400cdff0 msgr2=0x7f744015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f743dd9b640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74400cdff0 0x7f744015b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f743dd9b640 1 -- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74400b6ac0 msgr2=0x7f74401f0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f743dd9b640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74400b6ac0 0x7f74401f0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f743dd9b640 1 -- 172.21.15.16:0/348392540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7430009b30 con 0x7f74400cd660 2023-12-08T08:39:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f743ed9d640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74400cdff0 0x7f744015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f743e59c640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74400b6ac0 0x7f74401f0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f743dd9b640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74400cd660 0x7f7440158d70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f742800d5f0 tx=0x7f742800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f7436ffd640 1 -- 172.21.15.16:0/348392540 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280125f0 con 0x7f74400cd660 2023-12-08T08:39:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f7436ffd640 1 -- 172.21.15.16:0/348392540 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7428012750 con 0x7f74400cd660 2023-12-08T08:39:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f7444f2c640 1 -- 172.21.15.16:0/348392540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74401573e0 con 0x7f74400cd660 2023-12-08T08:39:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f7436ffd640 1 -- 172.21.15.16:0/348392540 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428012b00 con 0x7f74400cd660 2023-12-08T08:39:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.380+0000 7f7444f2c640 1 -- 172.21.15.16:0/348392540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7440159c30 con 0x7f74400cd660 2023-12-08T08:39:16.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.381+0000 7f7444f2c640 1 -- 172.21.15.16:0/348392540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74400745c0 con 0x7f74400cd660 2023-12-08T08:39:16.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.382+0000 7f7436ffd640 1 -- 172.21.15.16:0/348392540 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7428012c60 con 0x7f74400cd660 2023-12-08T08:39:16.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.382+0000 7f7436ffd640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f742006ae40 0x7f742006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.382+0000 7f7436ffd640 1 -- 172.21.15.16:0/348392540 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f7428089ec0 con 0x7f74400cd660 2023-12-08T08:39:16.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.382+0000 7f743e59c640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f742006ae40 0x7f742006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.383+0000 7f743e59c640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f742006ae40 0x7f742006d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7430035000 tx=0x7f7430030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:16.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.386+0000 7f7436ffd640 1 -- 172.21.15.16:0/348392540 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7428054b90 con 0x7f74400cd660 2023-12-08T08:39:16.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.564+0000 7f7444f2c640 1 -- 172.21.15.16:0/348392540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7440067430 con 0x7f74400cd660 2023-12-08T08:39:16.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.565+0000 7f7436ffd640 1 -- 172.21.15.16:0/348392540 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7f74280581b0 con 0x7f74400cd660 2023-12-08T08:39:16.568 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:16.568 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:16.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.567+0000 7f7444f2c640 1 -- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f742006ae40 msgr2=0x7f742006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.567+0000 7f7444f2c640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f742006ae40 0x7f742006d300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7430035000 tx=0x7f7430030000 comp rx=0 tx=0).stop 2023-12-08T08:39:16.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.567+0000 7f7444f2c640 1 -- 172.21.15.16:0/348392540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74400cd660 msgr2=0x7f7440158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.567+0000 7f7444f2c640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74400cd660 0x7f7440158d70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f742800d5f0 tx=0x7f742800dab0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.567+0000 7f7444f2c640 1 -- 172.21.15.16:0/348392540 shutdown_connections 2023-12-08T08:39:16.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.567+0000 7f7444f2c640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f742006ae40 0x7f742006d300 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.567+0000 7f7444f2c640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f74400cdff0 0x7f744015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.567+0000 7f7444f2c640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74400cd660 0x7f7440158d70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.567+0000 7f7444f2c640 1 --2- 172.21.15.16:0/348392540 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f74400b6ac0 0x7f74401f0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.567+0000 7f7444f2c640 1 -- 172.21.15.16:0/348392540 >> 172.21.15.16:0/348392540 conn(0x7f74400d0de0 msgr2=0x7f74400b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:16.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.567+0000 7f7444f2c640 1 -- 172.21.15.16:0/348392540 shutdown_connections 2023-12-08T08:39:16.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.567+0000 7f7444f2c640 1 -- 172.21.15.16:0/348392540 wait complete. 2023-12-08T08:39:16.580 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:39:16.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.689+0000 7f58058b4640 1 Processor -- start 2023-12-08T08:39:16.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.689+0000 7f58058b4640 1 -- start start 2023-12-08T08:39:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.690+0000 7f58058b4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58000b8950 0x7f58000ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.690+0000 7f58058b4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58000cfc80 0x7f58000b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.690+0000 7f58058b4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58000b5f00 con 0x7f58000cebc0 2023-12-08T08:39:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.690+0000 7f58058b4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58000b6170 con 0x7f58000b8950 2023-12-08T08:39:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.690+0000 7f58058b4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58000b6450 con 0x7f58000cfc80 2023-12-08T08:39:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.690+0000 7f57feffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58000b8950 0x7f58000ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.690+0000 7f57feffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58000b8950 0x7f58000ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49166/0 (socket says 172.21.15.16:49166) 2023-12-08T08:39:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.690+0000 7f57feffd640 1 -- 172.21.15.16:0/2726304281 learned_addr learned my addr 172.21.15.16:0/2726304281 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.690+0000 7f57ff7fe640 1 --2- 172.21.15.16:0/2726304281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58000cfc80 0x7f58000b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.690+0000 7f57feffd640 1 -- 172.21.15.16:0/2726304281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58000cfc80 msgr2=0x7f58000b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.690+0000 7f57feffd640 1 --2- 172.21.15.16:0/2726304281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58000cfc80 0x7f58000b6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.690+0000 7f57feffd640 1 -- 172.21.15.16:0/2726304281 >> v1:172.21.15.112:6789/0 conn(0x7f58000cebc0 legacy=0x7f58000cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:16.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.690+0000 7f57feffd640 1 -- 172.21.15.16:0/2726304281 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58000b6730 con 0x7f58000b8950 2023-12-08T08:39:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.691+0000 7f57feffd640 1 --2- 172.21.15.16:0/2726304281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58000b8950 0x7f58000ce680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009870 tx=0x7f57f002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a24582a07a04cca server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.691+0000 7f57fe7fc640 1 -- 172.21.15.16:0/2726304281 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00317c0 con 0x7f58000b8950 2023-12-08T08:39:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.691+0000 7f57fe7fc640 1 -- 172.21.15.16:0/2726304281 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f0031920 con 0x7f58000b8950 2023-12-08T08:39:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.691+0000 7f57fe7fc640 1 -- 172.21.15.16:0/2726304281 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0031cb0 con 0x7f58000b8950 2023-12-08T08:39:16.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.691+0000 7f58058b4640 1 -- 172.21.15.16:0/2726304281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58000b8950 msgr2=0x7f58000ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.691+0000 7f58058b4640 1 --2- 172.21.15.16:0/2726304281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58000b8950 0x7f58000ce680 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009870 tx=0x7f57f002b860 comp rx=0 tx=0).stop 2023-12-08T08:39:16.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.691+0000 7f58058b4640 1 -- 172.21.15.16:0/2726304281 shutdown_connections 2023-12-08T08:39:16.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.691+0000 7f58058b4640 1 --2- 172.21.15.16:0/2726304281 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58000cfc80 0x7f58000b6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.691+0000 7f58058b4640 1 --2- 172.21.15.16:0/2726304281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58000b8950 0x7f58000ce680 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.691+0000 7f58058b4640 1 -- 172.21.15.16:0/2726304281 >> 172.21.15.16:0/2726304281 conn(0x7f58000d3010 msgr2=0x7f58000d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:16.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.691+0000 7f58058b4640 1 -- 172.21.15.16:0/2726304281 shutdown_connections 2023-12-08T08:39:16.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.692+0000 7f58058b4640 1 -- 172.21.15.16:0/2726304281 wait complete. 2023-12-08T08:39:16.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.693+0000 7f58058b4640 1 Processor -- start 2023-12-08T08:39:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.693+0000 7f58058b4640 1 -- start start 2023-12-08T08:39:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.693+0000 7f58058b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58000b8950 0x7f580014f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.693+0000 7f58058b4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58000cebc0 0x7f58001521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.693+0000 7f58058b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58000cfc80 0x7f5800150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.693+0000 7f58058b4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580009f4c0 con 0x7f58000b8950 2023-12-08T08:39:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.693+0000 7f58058b4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5800152700 con 0x7f58000cfc80 2023-12-08T08:39:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.693+0000 7f58058b4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580014fcc0 con 0x7f58000cebc0 2023-12-08T08:39:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58000b8950 0x7f580014f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57feffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58000b8950 0x7f580014f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49176/0 (socket says 172.21.15.16:49176) 2023-12-08T08:39:16.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57feffd640 1 -- 172.21.15.16:0/2915136371 learned_addr learned my addr 172.21.15.16:0/2915136371 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57ff7fe640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58000cfc80 0x7f5800150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57f67fc640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58000cebc0 0x7f58001521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57feffd640 1 -- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58000cebc0 msgr2=0x7f58001521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57feffd640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58000cebc0 0x7f58001521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57feffd640 1 -- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58000cfc80 msgr2=0x7f5800150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57feffd640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58000cfc80 0x7f5800150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57feffd640 1 -- 172.21.15.16:0/2915136371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f00094d0 con 0x7f58000b8950 2023-12-08T08:39:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57ff7fe640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58000cfc80 0x7f5800150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:16.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57f67fc640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58000cebc0 0x7f58001521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57feffd640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58000b8950 0x7f580014f020 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009870 tx=0x7f57f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57fcff9640 1 -- 172.21.15.16:0/2915136371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0032860 con 0x7f58000b8950 2023-12-08T08:39:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57fcff9640 1 -- 172.21.15.16:0/2915136371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f00329c0 con 0x7f58000b8950 2023-12-08T08:39:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f57fcff9640 1 -- 172.21.15.16:0/2915136371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0032d70 con 0x7f58000b8950 2023-12-08T08:39:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f58058b4640 1 -- 172.21.15.16:0/2915136371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580014ffa0 con 0x7f58000b8950 2023-12-08T08:39:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.694+0000 7f58058b4640 1 -- 172.21.15.16:0/2915136371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5800150560 con 0x7f58000b8950 2023-12-08T08:39:16.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.695+0000 7f58058b4640 1 -- 172.21.15.16:0/2915136371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f580008b150 con 0x7f58000b8950 2023-12-08T08:39:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.696+0000 7f57fcff9640 1 -- 172.21.15.16:0/2915136371 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f57f004d3f0 con 0x7f58000b8950 2023-12-08T08:39:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.696+0000 7f57fcff9640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f57e006ae40 0x7f57e006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.696+0000 7f57fcff9640 1 -- 172.21.15.16:0/2915136371 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f57f00ab7f0 con 0x7f58000b8950 2023-12-08T08:39:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.696+0000 7f57f67fc640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f57e006ae40 0x7f57e006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:16.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.697+0000 7f57f67fc640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f57e006ae40 0x7f57e006d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5800155e30 tx=0x7f57e8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:16.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.701+0000 7f57fcff9640 1 -- 172.21.15.16:0/2915136371 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57f0076550 con 0x7f58000b8950 2023-12-08T08:39:16.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.876+0000 7f58058b4640 1 -- 172.21.15.16:0/2915136371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5800067430 con 0x7f58000b8950 2023-12-08T08:39:16.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.877+0000 7f57fcff9640 1 -- 172.21.15.16:0/2915136371 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57f0079b70 con 0x7f58000b8950 2023-12-08T08:39:16.879 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.879+0000 7f58058b4640 1 -- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f57e006ae40 msgr2=0x7f57e006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.879+0000 7f58058b4640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f57e006ae40 0x7f57e006d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5800155e30 tx=0x7f57e8009040 comp rx=0 tx=0).stop 2023-12-08T08:39:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.879+0000 7f58058b4640 1 -- 172.21.15.16:0/2915136371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58000b8950 msgr2=0x7f580014f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.879+0000 7f58058b4640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58000b8950 0x7f580014f020 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009870 tx=0x7f57f00046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.879+0000 7f58058b4640 1 -- 172.21.15.16:0/2915136371 shutdown_connections 2023-12-08T08:39:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.879+0000 7f58058b4640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f57e006ae40 0x7f57e006d300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.879+0000 7f58058b4640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f58000cfc80 0x7f5800150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.879+0000 7f58058b4640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f58000cebc0 0x7f58001521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.879+0000 7f58058b4640 1 --2- 172.21.15.16:0/2915136371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58000b8950 0x7f580014f020 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.879+0000 7f58058b4640 1 -- 172.21.15.16:0/2915136371 >> 172.21.15.16:0/2915136371 conn(0x7f58000d3010 msgr2=0x7f580006b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.879+0000 7f58058b4640 1 -- 172.21.15.16:0/2915136371 shutdown_connections 2023-12-08T08:39:16.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.879+0000 7f58058b4640 1 -- 172.21.15.16:0/2915136371 wait complete. 2023-12-08T08:39:16.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:39:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.999+0000 7f7ca5890640 1 Processor -- start 2023-12-08T08:39:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:16.999+0000 7f7ca5890640 1 -- start start 2023-12-08T08:39:17.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.000+0000 7f7ca5890640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f7ca00b8950 0x7f7ca00ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.000+0000 7f7ca5890640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ca00cfc80 0x7f7ca00b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.000+0000 7f7ca5890640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca00b5f00 con 0x7f7ca00cebc0 2023-12-08T08:39:17.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.000+0000 7f7ca5890640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca00b6170 con 0x7f7ca00cfc80 2023-12-08T08:39:17.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.000+0000 7f7ca5890640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca00b6450 con 0x7f7ca00b8950 2023-12-08T08:39:17.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.000+0000 7f7c9f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ca00cfc80 0x7f7ca00b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.000+0000 7f7c9f7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ca00cfc80 0x7f7ca00b6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49184/0 (socket says 172.21.15.16:49184) 2023-12-08T08:39:17.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.000+0000 7f7c9f7fe640 1 -- 172.21.15.16:0/1489793420 learned_addr learned my addr 172.21.15.16:0/1489793420 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:17.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.000+0000 7f7c9effd640 1 --2- 172.21.15.16:0/1489793420 >> v2:172.21.15.112:3300/0 conn(0x7f7ca00b8950 0x7f7ca00ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.000+0000 7f7c9f7fe640 1 -- 172.21.15.16:0/1489793420 >> v2:172.21.15.112:3300/0 conn(0x7f7ca00b8950 msgr2=0x7f7ca00ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:17.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.000+0000 7f7c9f7fe640 1 --2- 172.21.15.16:0/1489793420 >> v2:172.21.15.112:3300/0 conn(0x7f7ca00b8950 0x7f7ca00ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.000+0000 7f7c9f7fe640 1 -- 172.21.15.16:0/1489793420 >> v1:172.21.15.112:6789/0 conn(0x7f7ca00cebc0 legacy=0x7f7ca00cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:17.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.000+0000 7f7c9f7fe640 1 -- 172.21.15.16:0/1489793420 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca00b6730 con 0x7f7ca00cfc80 2023-12-08T08:39:17.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.001+0000 7f7c9f7fe640 1 --2- 172.21.15.16:0/1489793420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca00cfc80 0x7f7ca00b6c50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7ca008b950 tx=0x7f7c9802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df456f8912290fc2 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:17.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.001+0000 7f7c9e7fc640 1 -- 172.21.15.16:0/1489793420 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9802c9f0 con 0x7f7ca00cfc80 2023-12-08T08:39:17.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.001+0000 7f7c9e7fc640 1 -- 172.21.15.16:0/1489793420 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9802cb50 con 0x7f7ca00cfc80 2023-12-08T08:39:17.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.001+0000 7f7c9e7fc640 1 -- 172.21.15.16:0/1489793420 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c980343f0 con 0x7f7ca00cfc80 2023-12-08T08:39:17.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.002+0000 7f7ca5890640 1 -- 172.21.15.16:0/1489793420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca00cfc80 msgr2=0x7f7ca00b6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.002+0000 7f7ca5890640 1 --2- 172.21.15.16:0/1489793420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca00cfc80 0x7f7ca00b6c50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7ca008b950 tx=0x7f7c9802be50 comp rx=0 tx=0).stop 2023-12-08T08:39:17.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.002+0000 7f7ca5890640 1 -- 172.21.15.16:0/1489793420 shutdown_connections 2023-12-08T08:39:17.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.002+0000 7f7ca5890640 1 --2- 172.21.15.16:0/1489793420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca00cfc80 0x7f7ca00b6c50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.002+0000 7f7ca5890640 1 --2- 172.21.15.16:0/1489793420 >> v2:172.21.15.112:3300/0 conn(0x7f7ca00b8950 0x7f7ca00ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.002+0000 7f7ca5890640 1 -- 172.21.15.16:0/1489793420 >> 172.21.15.16:0/1489793420 conn(0x7f7ca00d3010 msgr2=0x7f7ca00d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:17.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.002+0000 7f7ca5890640 1 -- 172.21.15.16:0/1489793420 shutdown_connections 2023-12-08T08:39:17.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.002+0000 7f7ca5890640 1 -- 172.21.15.16:0/1489793420 wait complete. 2023-12-08T08:39:17.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.003+0000 7f7ca5890640 1 Processor -- start 2023-12-08T08:39:17.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.003+0000 7f7ca5890640 1 -- start start 2023-12-08T08:39:17.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7ca5890640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca00b8950 0x7f7ca00b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7ca5890640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ca00cebc0 0x7f7ca015b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7ca5890640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ca00cfc80 0x7f7ca006b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7ca5890640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca015c170 con 0x7f7ca00b8950 2023-12-08T08:39:17.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7ca5890640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca015c450 con 0x7f7ca00cebc0 2023-12-08T08:39:17.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7ca5890640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca015c730 con 0x7f7ca00cfc80 2023-12-08T08:39:17.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7c9effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca00b8950 0x7f7ca00b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7c9effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca00b8950 0x7f7ca00b3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49200/0 (socket says 172.21.15.16:49200) 2023-12-08T08:39:17.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7c9effd640 1 -- 172.21.15.16:0/3841978448 learned_addr learned my addr 172.21.15.16:0/3841978448 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:17.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7c9effd640 1 -- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ca00cfc80 msgr2=0x7f7ca006b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7c97fff640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ca00cebc0 0x7f7ca015b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7c9f7fe640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ca00cfc80 0x7f7ca006b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7c9effd640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ca00cfc80 0x7f7ca006b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7c9effd640 1 -- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ca00cebc0 msgr2=0x7f7ca015b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7c9effd640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ca00cebc0 0x7f7ca015b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7c9effd640 1 -- 172.21.15.16:0/3841978448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c98009b30 con 0x7f7ca00b8950 2023-12-08T08:39:17.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7c97fff640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ca00cebc0 0x7f7ca015b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:17.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.004+0000 7f7c9effd640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca00b8950 0x7f7ca00b3430 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000d5f0 tx=0x7f7c9000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:17.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.005+0000 7f7c9cff9640 1 -- 172.21.15.16:0/3841978448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c900125f0 con 0x7f7ca00b8950 2023-12-08T08:39:17.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.005+0000 7f7c9cff9640 1 -- 172.21.15.16:0/3841978448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c90012750 con 0x7f7ca00b8950 2023-12-08T08:39:17.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.005+0000 7f7c9cff9640 1 -- 172.21.15.16:0/3841978448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90012b00 con 0x7f7ca00b8950 2023-12-08T08:39:17.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.005+0000 7f7ca5890640 1 -- 172.21.15.16:0/3841978448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca015ca10 con 0x7f7ca00b8950 2023-12-08T08:39:17.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.005+0000 7f7ca5890640 1 -- 172.21.15.16:0/3841978448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca015ccf0 con 0x7f7ca00b8950 2023-12-08T08:39:17.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.006+0000 7f7ca5890640 1 -- 172.21.15.16:0/3841978448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca008b150 con 0x7f7ca00b8950 2023-12-08T08:39:17.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.006+0000 7f7c9cff9640 1 -- 172.21.15.16:0/3841978448 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c90012c60 con 0x7f7ca00b8950 2023-12-08T08:39:17.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.006+0000 7f7c9cff9640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7c8006ae40 0x7f7c8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.007+0000 7f7c9cff9640 1 -- 172.21.15.16:0/3841978448 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f7c90089e50 con 0x7f7ca00b8950 2023-12-08T08:39:17.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.007+0000 7f7c97fff640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7c8006ae40 0x7f7c8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.007+0000 7f7c97fff640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7c8006ae40 0x7f7c8006d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ca006be00 tx=0x7f7c88009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:17.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.011+0000 7f7c9cff9640 1 -- 172.21.15.16:0/3841978448 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c90054bb0 con 0x7f7ca00b8950 2023-12-08T08:39:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.186+0000 7f7ca5890640 1 -- 172.21.15.16:0/3841978448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7ca0067430 con 0x7f7ca00b8950 2023-12-08T08:39:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.186+0000 7f7c9cff9640 1 -- 172.21.15.16:0/3841978448 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7c900581d0 con 0x7f7ca00b8950 2023-12-08T08:39:17.188 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:17.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.188+0000 7f7ca5890640 1 -- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7c8006ae40 msgr2=0x7f7c8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.188+0000 7f7ca5890640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7c8006ae40 0x7f7c8006d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7ca006be00 tx=0x7f7c88009040 comp rx=0 tx=0).stop 2023-12-08T08:39:17.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.188+0000 7f7ca5890640 1 -- 172.21.15.16:0/3841978448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca00b8950 msgr2=0x7f7ca00b3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.188+0000 7f7ca5890640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca00b8950 0x7f7ca00b3430 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000d5f0 tx=0x7f7c9000dab0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.188+0000 7f7ca5890640 1 -- 172.21.15.16:0/3841978448 shutdown_connections 2023-12-08T08:39:17.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.188+0000 7f7ca5890640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7c8006ae40 0x7f7c8006d300 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.188+0000 7f7ca5890640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7ca00cfc80 0x7f7ca006b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.188+0000 7f7ca5890640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7ca00cebc0 0x7f7ca015b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.188+0000 7f7ca5890640 1 --2- 172.21.15.16:0/3841978448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca00b8950 0x7f7ca00b3430 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.188+0000 7f7ca5890640 1 -- 172.21.15.16:0/3841978448 >> 172.21.15.16:0/3841978448 conn(0x7f7ca00d3010 msgr2=0x7f7ca01605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:17.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.188+0000 7f7ca5890640 1 -- 172.21.15.16:0/3841978448 shutdown_connections 2023-12-08T08:39:17.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.188+0000 7f7ca5890640 1 -- 172.21.15.16:0/3841978448 wait complete. 2023-12-08T08:39:17.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:39:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.309+0000 7f7a91b61640 1 Processor -- start 2023-12-08T08:39:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.309+0000 7f7a91b61640 1 -- start start 2023-12-08T08:39:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.309+0000 7f7a91b61640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a8c0cf570 0x7f7a8c0cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.309+0000 7f7a91b61640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a8c0b6c50 0x7f7a8c0b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.309+0000 7f7a91b61640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8c0cfef0 con 0x7f7a8c0b8950 2023-12-08T08:39:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.309+0000 7f7a91b61640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8c0b5f00 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.309+0000 7f7a91b61640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0b6170 con 0x7f7a8c0cf570 2023-12-08T08:39:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.310+0000 7f7a8bfff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a8c0b6c50 0x7f7a8c0b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.310+0000 7f7a8b7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7a8c0b8950 0x7f7a8c0ce680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34238/0 (socket says 172.21.15.16:34238) 2023-12-08T08:39:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.310+0000 7f7a8b7fe640 1 -- 172.21.15.16:0/1469302772 learned_addr learned my addr 172.21.15.16:0/1469302772 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.310+0000 7f7a8affd640 1 --2- 172.21.15.16:0/1469302772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a8c0cf570 0x7f7a8c0cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.310+0000 7f7a8bfff640 1 -- 172.21.15.16:0/1469302772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a8c0cf570 msgr2=0x7f7a8c0cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.310+0000 7f7a8bfff640 1 --2- 172.21.15.16:0/1469302772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a8c0cf570 0x7f7a8c0cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.310+0000 7f7a8bfff640 1 -- 172.21.15.16:0/1469302772 >> v1:172.21.15.16:6789/0 conn(0x7f7a8c0b8950 legacy=0x7f7a8c0ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.310+0000 7f7a8bfff640 1 -- 172.21.15.16:0/1469302772 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8c0b6450 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.310+0000 7f7a8affd640 1 --2- 172.21.15.16:0/1469302772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a8c0cf570 0x7f7a8c0cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.311+0000 7f7a8bfff640 1 --2- 172.21.15.16:0/1469302772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c0b6c50 0x7f7a8c0b70e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009d90 tx=0x7f7a8402bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75af23e6a4011516 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.311+0000 7f7a8a7fc640 1 -- 172.21.15.16:0/1469302772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8402c980 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.311+0000 7f7a8a7fc640 1 -- 172.21.15.16:0/1469302772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8402cae0 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.311+0000 7f7a8a7fc640 1 -- 172.21.15.16:0/1469302772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8402ce70 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.311+0000 7f7a91b61640 1 -- 172.21.15.16:0/1469302772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c0b6c50 msgr2=0x7f7a8c0b70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.311+0000 7f7a91b61640 1 --2- 172.21.15.16:0/1469302772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c0b6c50 0x7f7a8c0b70e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7a84009d90 tx=0x7f7a8402bc50 comp rx=0 tx=0).stop 2023-12-08T08:39:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.311+0000 7f7a91b61640 1 -- 172.21.15.16:0/1469302772 shutdown_connections 2023-12-08T08:39:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.311+0000 7f7a91b61640 1 --2- 172.21.15.16:0/1469302772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c0b6c50 0x7f7a8c0b70e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.311+0000 7f7a91b61640 1 --2- 172.21.15.16:0/1469302772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a8c0cf570 0x7f7a8c0cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.311+0000 7f7a91b61640 1 -- 172.21.15.16:0/1469302772 >> 172.21.15.16:0/1469302772 conn(0x7f7a8c0d3010 msgr2=0x7f7a8c0d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.311+0000 7f7a91b61640 1 -- 172.21.15.16:0/1469302772 shutdown_connections 2023-12-08T08:39:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.311+0000 7f7a91b61640 1 -- 172.21.15.16:0/1469302772 wait complete. 2023-12-08T08:39:17.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.313+0000 7f7a91b61640 1 Processor -- start 2023-12-08T08:39:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.313+0000 7f7a91b61640 1 -- start start 2023-12-08T08:39:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.313+0000 7f7a91b61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c0b6c50 0x7f7a8c15da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.313+0000 7f7a91b61640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a8c0b8950 0x7f7a8c1f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.313+0000 7f7a91b61640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a8c0cf570 0x7f7a8c0c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.313+0000 7f7a91b61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c1f3e90 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.313+0000 7f7a91b61640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0c8280 con 0x7f7a8c0cf570 2023-12-08T08:39:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.313+0000 7f7a91b61640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8c0c8e40 con 0x7f7a8c0b8950 2023-12-08T08:39:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c0b6c50 0x7f7a8c15da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c0b6c50 0x7f7a8c15da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49224/0 (socket says 172.21.15.16:49224) 2023-12-08T08:39:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a8b7fe640 1 -- 172.21.15.16:0/1323378765 learned_addr learned my addr 172.21.15.16:0/1323378765 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a8b7fe640 1 -- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a8c0b8950 msgr2=0x7f7a8c1f3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a8affd640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a8c0b8950 0x7f7a8c1f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a8bfff640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a8c0cf570 0x7f7a8c0c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a8b7fe640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a8c0b8950 0x7f7a8c1f3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a8b7fe640 1 -- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a8c0cf570 msgr2=0x7f7a8c0c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a8b7fe640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a8c0cf570 0x7f7a8c0c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a8b7fe640 1 -- 172.21.15.16:0/1323378765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a84009a70 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a8affd640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a8c0b8950 0x7f7a8c1f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a8bfff640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a8c0cf570 0x7f7a8c0c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a8b7fe640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c0b6c50 0x7f7a8c15da30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00c800 tx=0x7f7a7c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a73fff640 1 -- 172.21.15.16:0/1323378765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0072d0 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a73fff640 1 -- 172.21.15.16:0/1323378765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c007430 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a91b61640 1 -- 172.21.15.16:0/1323378765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8c0c6ca0 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a73fff640 1 -- 172.21.15.16:0/1323378765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0047e0 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.314+0000 7f7a91b61640 1 -- 172.21.15.16:0/1323378765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8c0c7260 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.315+0000 7f7a91b61640 1 -- 172.21.15.16:0/1323378765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8c08b150 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.316+0000 7f7a73fff640 1 -- 172.21.15.16:0/1323378765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7a7c004960 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.316+0000 7f7a73fff640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7a7806ae40 0x7f7a7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.316+0000 7f7a73fff640 1 -- 172.21.15.16:0/1323378765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f7a7c089080 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.316+0000 7f7a8affd640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7a7806ae40 0x7f7a7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.317+0000 7f7a8affd640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7a7806ae40 0x7f7a7806d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7a80006fd0 tx=0x7f7a80008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.320+0000 7f7a73fff640 1 -- 172.21.15.16:0/1323378765 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a7c053de0 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.497+0000 7f7a91b61640 1 -- 172.21.15.16:0/1323378765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7a8c0d2090 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.497+0000 7f7a73fff640 1 -- 172.21.15.16:0/1323378765 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7a7c0180a0 con 0x7f7a8c0b6c50 2023-12-08T08:39:17.500 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:39:17.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.499+0000 7f7a91b61640 1 -- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7a7806ae40 msgr2=0x7f7a7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.499+0000 7f7a91b61640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7a7806ae40 0x7f7a7806d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7a80006fd0 tx=0x7f7a80008040 comp rx=0 tx=0).stop 2023-12-08T08:39:17.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.499+0000 7f7a91b61640 1 -- 172.21.15.16:0/1323378765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c0b6c50 msgr2=0x7f7a8c15da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.499+0000 7f7a91b61640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c0b6c50 0x7f7a8c15da30 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00c800 tx=0x7f7a7c00ccc0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.499+0000 7f7a91b61640 1 -- 172.21.15.16:0/1323378765 shutdown_connections 2023-12-08T08:39:17.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.499+0000 7f7a91b61640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7a7806ae40 0x7f7a7806d300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.499+0000 7f7a91b61640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7a8c0cf570 0x7f7a8c0c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.499+0000 7f7a91b61640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7a8c0b8950 0x7f7a8c1f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.499+0000 7f7a91b61640 1 --2- 172.21.15.16:0/1323378765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c0b6c50 0x7f7a8c15da30 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.499+0000 7f7a91b61640 1 -- 172.21.15.16:0/1323378765 >> 172.21.15.16:0/1323378765 conn(0x7f7a8c0d3010 msgr2=0x7f7a8c0b7030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:17.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.499+0000 7f7a91b61640 1 -- 172.21.15.16:0/1323378765 shutdown_connections 2023-12-08T08:39:17.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.499+0000 7f7a91b61640 1 -- 172.21.15.16:0/1323378765 wait complete. 2023-12-08T08:39:17.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:39:17.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.620+0000 7f3d94bec640 1 Processor -- start 2023-12-08T08:39:17.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.620+0000 7f3d94bec640 1 -- start start 2023-12-08T08:39:17.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.620+0000 7f3d94bec640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f3d900b0af0 0x7f3d900b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.620+0000 7f3d94bec640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d900b2280 con 0x7f3d9015d4f0 2023-12-08T08:39:17.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.620+0000 7f3d94bec640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d900b1430 con 0x7f3d900cef90 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.620+0000 7f3d94bec640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9006bb40 con 0x7f3d900b0af0 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8e59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3d9015d4f0 0x7f3d900b03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34252/0 (socket says 172.21.15.16:34252) 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8e59c640 1 -- 172.21.15.16:0/3091033990 learned_addr learned my addr 172.21.15.16:0/3091033990 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 -- 172.21.15.16:0/3091033990 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561697913 0 0) 0x7f3d900b2280 con 0x7f3d9015d4f0 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d87fff640 1 --2- 172.21.15.16:0/3091033990 >> v2:172.21.15.112:3300/0 conn(0x7f3d900b0af0 0x7f3d900b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 -- 172.21.15.16:0/3091033990 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d7c003520 con 0x7f3d9015d4f0 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 -- 172.21.15.16:0/3091033990 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1869964216 0 0) 0x7f3d7c003520 con 0x7f3d9015d4f0 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 -- 172.21.15.16:0/3091033990 >> v2:172.21.15.112:3300/0 conn(0x7f3d900b0af0 msgr2=0x7f3d900b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 --2- 172.21.15.16:0/3091033990 >> v2:172.21.15.112:3300/0 conn(0x7f3d900b0af0 0x7f3d900b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 -- 172.21.15.16:0/3091033990 >> v1:172.21.15.112:6789/0 conn(0x7f3d900cef90 legacy=0x7f3d900cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 -- 172.21.15.16:0/3091033990 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900b2710 con 0x7f3d9015d4f0 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 -- 172.21.15.16:0/3091033990 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f3d80001670 con 0x7f3d9015d4f0 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 -- 172.21.15.16:0/3091033990 >> v1:172.21.15.16:6789/0 conn(0x7f3d9015d4f0 legacy=0x7f3d900b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 --2- 172.21.15.16:0/3091033990 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d7c003970 0x7f3d7c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 --2- 172.21.15.16:0/3091033990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d7c0047c0 0x7f3d7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 --2- 172.21.15.16:0/3091033990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d7c005180 0x7f3d7c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 -- 172.21.15.16:0/3091033990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9006bb40 con 0x7f3d7c0047c0 2023-12-08T08:39:17.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 -- 172.21.15.16:0/3091033990 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c00c020 con 0x7f3d7c005180 2023-12-08T08:39:17.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8dd9b640 1 -- 172.21.15.16:0/3091033990 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c00c1e0 con 0x7f3d7c003970 2023-12-08T08:39:17.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d87fff640 1 --2- 172.21.15.16:0/3091033990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d7c0047c0 0x7f3d7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.621+0000 7f3d8ed9d640 1 --2- 172.21.15.16:0/3091033990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d7c005180 0x7f3d7c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.622+0000 7f3d8e59c640 1 --2- 172.21.15.16:0/3091033990 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d7c003970 0x7f3d7c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.622+0000 7f3d87fff640 1 -- 172.21.15.16:0/3091033990 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d7c003970 msgr2=0x7f3d7c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:17.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.622+0000 7f3d87fff640 1 --2- 172.21.15.16:0/3091033990 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d7c003970 0x7f3d7c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.622+0000 7f3d87fff640 1 -- 172.21.15.16:0/3091033990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d7c005180 msgr2=0x7f3d7c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:17.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.622+0000 7f3d87fff640 1 --2- 172.21.15.16:0/3091033990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d7c005180 0x7f3d7c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.622+0000 7f3d87fff640 1 -- 172.21.15.16:0/3091033990 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c003520 con 0x7f3d7c0047c0 2023-12-08T08:39:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.622+0000 7f3d8ed9d640 1 --2- 172.21.15.16:0/3091033990 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d7c005180 0x7f3d7c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.622+0000 7f3d8ed9d640 1 -- 172.21.15.16:0/3091033990 reap_dead start 2023-12-08T08:39:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.622+0000 7f3d87fff640 1 --2- 172.21.15.16:0/3091033990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d7c0047c0 0x7f3d7c004c40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009ac0 tx=0x7f3d7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce170ec45f3c3630 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.622+0000 7f3d8dd9b640 1 -- 172.21.15.16:0/3091033990 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7802c6b0 con 0x7f3d7c0047c0 2023-12-08T08:39:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.623+0000 7f3d8dd9b640 1 -- 172.21.15.16:0/3091033990 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7802c810 con 0x7f3d7c0047c0 2023-12-08T08:39:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.623+0000 7f3d8dd9b640 1 -- 172.21.15.16:0/3091033990 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7802cba0 con 0x7f3d7c0047c0 2023-12-08T08:39:17.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.623+0000 7f3d94bec640 1 -- 172.21.15.16:0/3091033990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d7c0047c0 msgr2=0x7f3d7c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.623+0000 7f3d94bec640 1 --2- 172.21.15.16:0/3091033990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d7c0047c0 0x7f3d7c004c40 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009ac0 tx=0x7f3d7c0057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.623+0000 7f3d94bec640 1 -- 172.21.15.16:0/3091033990 shutdown_connections 2023-12-08T08:39:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.623+0000 7f3d94bec640 1 --2- 172.21.15.16:0/3091033990 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d7c0047c0 0x7f3d7c004c40 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.623+0000 7f3d94bec640 1 -- 172.21.15.16:0/3091033990 >> 172.21.15.16:0/3091033990 conn(0x7f3d900d0e00 msgr2=0x7f3d900d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.623+0000 7f3d94bec640 1 -- 172.21.15.16:0/3091033990 shutdown_connections 2023-12-08T08:39:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.623+0000 7f3d94bec640 1 -- 172.21.15.16:0/3091033990 wait complete. 2023-12-08T08:39:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.624+0000 7f3d94bec640 1 Processor -- start 2023-12-08T08:39:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.624+0000 7f3d94bec640 1 -- start start 2023-12-08T08:39:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.625+0000 7f3d94bec640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d7c0047c0 0x7f3d900c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.625+0000 7f3d94bec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d9014bc90 0x7f3d9009e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.625+0000 7f3d94bec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d900c5310 0x7f3d900cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.625+0000 7f3d94bec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9009eb30 con 0x7f3d900c5310 2023-12-08T08:39:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.625+0000 7f3d94bec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900c5e70 con 0x7f3d9014bc90 2023-12-08T08:39:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.625+0000 7f3d94bec640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900c5b90 con 0x7f3d7c0047c0 2023-12-08T08:39:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.625+0000 7f3d8ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d900c5310 0x7f3d900cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.625+0000 7f3d8ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d900c5310 0x7f3d900cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49252/0 (socket says 172.21.15.16:49252) 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.625+0000 7f3d8ed9d640 1 -- 172.21.15.16:0/1327339405 learned_addr learned my addr 172.21.15.16:0/1327339405 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.625+0000 7f3d87fff640 1 --2- 172.21.15.16:0/1327339405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d9014bc90 0x7f3d9009e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.625+0000 7f3d8e59c640 1 --2- 172.21.15.16:0/1327339405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d7c0047c0 0x7f3d900c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.626+0000 7f3d8ed9d640 1 -- 172.21.15.16:0/1327339405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d7c0047c0 msgr2=0x7f3d900c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.626+0000 7f3d8ed9d640 1 --2- 172.21.15.16:0/1327339405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d7c0047c0 0x7f3d900c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.626+0000 7f3d8ed9d640 1 -- 172.21.15.16:0/1327339405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d9014bc90 msgr2=0x7f3d9009e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.626+0000 7f3d8ed9d640 1 --2- 172.21.15.16:0/1327339405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d9014bc90 0x7f3d9009e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.626+0000 7f3d8ed9d640 1 -- 172.21.15.16:0/1327339405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900c91c0 con 0x7f3d900c5310 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.626+0000 7f3d8ed9d640 1 --2- 172.21.15.16:0/1327339405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d900c5310 0x7f3d900cc990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3d9006d5a0 tx=0x7f3d8800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.626+0000 7f3d86ffd640 1 -- 172.21.15.16:0/1327339405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88003b80 con 0x7f3d900c5310 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.626+0000 7f3d86ffd640 1 -- 172.21.15.16:0/1327339405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d88003ce0 con 0x7f3d900c5310 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.626+0000 7f3d94bec640 1 -- 172.21.15.16:0/1327339405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d900c94a0 con 0x7f3d900c5310 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.626+0000 7f3d86ffd640 1 -- 172.21.15.16:0/1327339405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88005310 con 0x7f3d900c5310 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.626+0000 7f3d94bec640 1 -- 172.21.15.16:0/1327339405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d900c57c0 con 0x7f3d900c5310 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.627+0000 7f3d94bec640 1 -- 172.21.15.16:0/1327339405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d900ce820 con 0x7f3d900c5310 2023-12-08T08:39:17.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.628+0000 7f3d86ffd640 1 -- 172.21.15.16:0/1327339405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3d88005470 con 0x7f3d900c5310 2023-12-08T08:39:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.628+0000 7f3d86ffd640 1 --2- 172.21.15.16:0/1327339405 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d7006ae40 0x7f3d7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.628+0000 7f3d86ffd640 1 -- 172.21.15.16:0/1327339405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f3d88089170 con 0x7f3d900c5310 2023-12-08T08:39:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.628+0000 7f3d8e59c640 1 --2- 172.21.15.16:0/1327339405 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d7006ae40 0x7f3d7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.629+0000 7f3d8e59c640 1 --2- 172.21.15.16:0/1327339405 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d7006ae40 0x7f3d7006d300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3d80001c90 tx=0x7f3d80007040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:17.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.632+0000 7f3d86ffd640 1 -- 172.21.15.16:0/1327339405 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d88053f80 con 0x7f3d900c5310 2023-12-08T08:39:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.806+0000 7f3d94bec640 1 -- 172.21.15.16:0/1327339405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3d900c99c0 con 0x7f3d900c5310 2023-12-08T08:39:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.807+0000 7f3d86ffd640 1 -- 172.21.15.16:0/1327339405 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3d900c99c0 con 0x7f3d900c5310 2023-12-08T08:39:17.809 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.809+0000 7f3d94bec640 1 -- 172.21.15.16:0/1327339405 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d7006ae40 msgr2=0x7f3d7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.809+0000 7f3d94bec640 1 --2- 172.21.15.16:0/1327339405 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d7006ae40 0x7f3d7006d300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3d80001c90 tx=0x7f3d80007040 comp rx=0 tx=0).stop 2023-12-08T08:39:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.809+0000 7f3d94bec640 1 -- 172.21.15.16:0/1327339405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d900c5310 msgr2=0x7f3d900cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.809+0000 7f3d94bec640 1 --2- 172.21.15.16:0/1327339405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d900c5310 0x7f3d900cc990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3d9006d5a0 tx=0x7f3d8800e960 comp rx=0 tx=0).stop 2023-12-08T08:39:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.809+0000 7f3d94bec640 1 -- 172.21.15.16:0/1327339405 shutdown_connections 2023-12-08T08:39:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.809+0000 7f3d94bec640 1 --2- 172.21.15.16:0/1327339405 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f3d7006ae40 0x7f3d7006d300 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.809+0000 7f3d94bec640 1 --2- 172.21.15.16:0/1327339405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d900c5310 0x7f3d900cc990 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.809+0000 7f3d94bec640 1 --2- 172.21.15.16:0/1327339405 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f3d9014bc90 0x7f3d9009e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.809+0000 7f3d94bec640 1 --2- 172.21.15.16:0/1327339405 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f3d7c0047c0 0x7f3d900c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.809+0000 7f3d94bec640 1 -- 172.21.15.16:0/1327339405 >> 172.21.15.16:0/1327339405 conn(0x7f3d900d0e00 msgr2=0x7f3d900adb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.809+0000 7f3d94bec640 1 -- 172.21.15.16:0/1327339405 shutdown_connections 2023-12-08T08:39:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.809+0000 7f3d94bec640 1 -- 172.21.15.16:0/1327339405 wait complete. 2023-12-08T08:39:17.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-08T08:39:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.928+0000 7f49d2dc9640 1 Processor -- start 2023-12-08T08:39:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.928+0000 7f49d2dc9640 1 -- start start 2023-12-08T08:39:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.929+0000 7f49d2dc9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f49cc0b6ac0 0x7f49cc0cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.929+0000 7f49d2dc9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49cc0ce1c0 0x7f49cc0cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.929+0000 7f49d2dc9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49cc0d0070 con 0x7f49cc0cd660 2023-12-08T08:39:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.929+0000 7f49d2dc9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49cc160cb0 con 0x7f49cc0b6ac0 2023-12-08T08:39:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.929+0000 7f49d2dc9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc0d02b0 con 0x7f49cc0ce1c0 2023-12-08T08:39:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.929+0000 7f49cbfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f49cc0cd660 0x7f49cc0cdab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34254/0 (socket says 172.21.15.16:34254) 2023-12-08T08:39:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.929+0000 7f49cbfff640 1 -- 172.21.15.16:0/3319217261 learned_addr learned my addr 172.21.15.16:0/3319217261 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.929+0000 7f49d0b65640 1 --2- 172.21.15.16:0/3319217261 >> v2:172.21.15.112:3300/0 conn(0x7f49cc0b6ac0 0x7f49cc0cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.929+0000 7f49d1366640 1 --2- 172.21.15.16:0/3319217261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49cc0ce1c0 0x7f49cc0cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.929+0000 7f49cb7fe640 1 -- 172.21.15.16:0/3319217261 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2039220327 0 0) 0x7f49bc001490 con 0x7f49cc0cd660 2023-12-08T08:39:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.929+0000 7f49cb7fe640 1 -- 172.21.15.16:0/3319217261 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b8003520 con 0x7f49cc0cd660 2023-12-08T08:39:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cb7fe640 1 -- 172.21.15.16:0/3319217261 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 786613172 0 0) 0x7f49b8003520 con 0x7f49cc0cd660 2023-12-08T08:39:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cb7fe640 1 -- 172.21.15.16:0/3319217261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49cc0ce1c0 msgr2=0x7f49cc0cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cb7fe640 1 --2- 172.21.15.16:0/3319217261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49cc0ce1c0 0x7f49cc0cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cb7fe640 1 -- 172.21.15.16:0/3319217261 >> v2:172.21.15.112:3300/0 conn(0x7f49cc0b6ac0 msgr2=0x7f49cc0cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cb7fe640 1 --2- 172.21.15.16:0/3319217261 >> v2:172.21.15.112:3300/0 conn(0x7f49cc0b6ac0 0x7f49cc0cfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cb7fe640 1 -- 172.21.15.16:0/3319217261 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0cb430 con 0x7f49cc0cd660 2023-12-08T08:39:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cb7fe640 1 -- 172.21.15.16:0/3319217261 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f49bc001ba0 con 0x7f49cc0cd660 2023-12-08T08:39:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49d0b65640 1 --2- 172.21.15.16:0/3319217261 >> v2:172.21.15.112:3300/0 conn(0x7f49cc0b6ac0 0x7f49cc0cfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:39:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cb7fe640 1 -- 172.21.15.16:0/3319217261 >> v1:172.21.15.16:6789/0 conn(0x7f49cc0cd660 legacy=0x7f49cc0cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cb7fe640 1 --2- 172.21.15.16:0/3319217261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49b8003970 0x7f49b8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cb7fe640 1 --2- 172.21.15.16:0/3319217261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49cc0ce1c0 0x7f49b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cb7fe640 1 --2- 172.21.15.16:0/3319217261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b8004d00 0x7f49b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cb7fe640 1 -- 172.21.15.16:0/3319217261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc160cb0 con 0x7f49b8004d00 2023-12-08T08:39:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cb7fe640 1 -- 172.21.15.16:0/3319217261 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc0d02b0 con 0x7f49b8003970 2023-12-08T08:39:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cb7fe640 1 -- 172.21.15.16:0/3319217261 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b800ba30 con 0x7f49cc0ce1c0 2023-12-08T08:39:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49d1366640 1 --2- 172.21.15.16:0/3319217261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b8004d00 0x7f49b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49d0b65640 1 --2- 172.21.15.16:0/3319217261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49b8003970 0x7f49b8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49cbfff640 1 --2- 172.21.15.16:0/3319217261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49cc0ce1c0 0x7f49b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49d1366640 1 -- 172.21.15.16:0/3319217261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49cc0ce1c0 msgr2=0x7f49b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49d1366640 1 --2- 172.21.15.16:0/3319217261 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49cc0ce1c0 0x7f49b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49d1366640 1 -- 172.21.15.16:0/3319217261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49b8003970 msgr2=0x7f49b8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49d1366640 1 --2- 172.21.15.16:0/3319217261 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49b8003970 0x7f49b8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.930+0000 7f49d1366640 1 -- 172.21.15.16:0/3319217261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b8003520 con 0x7f49b8004d00 2023-12-08T08:39:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.931+0000 7f49d1366640 1 --2- 172.21.15.16:0/3319217261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b8004d00 0x7f49b8005180 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009aa0 tx=0x7f49b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae60d8c976f086ce server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.931+0000 7f49cb7fe640 1 -- 172.21.15.16:0/3319217261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b402c6b0 con 0x7f49b8004d00 2023-12-08T08:39:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.931+0000 7f49cb7fe640 1 -- 172.21.15.16:0/3319217261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b402c810 con 0x7f49b8004d00 2023-12-08T08:39:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.931+0000 7f49cb7fe640 1 -- 172.21.15.16:0/3319217261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b402cbc0 con 0x7f49b8004d00 2023-12-08T08:39:17.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.932+0000 7f49d2dc9640 1 -- 172.21.15.16:0/3319217261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b8004d00 msgr2=0x7f49b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.932+0000 7f49d2dc9640 1 --2- 172.21.15.16:0/3319217261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49b8004d00 0x7f49b8005180 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009aa0 tx=0x7f49b80057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.932+0000 7f49d1366640 1 -- 172.21.15.16:0/3319217261 reap_dead start 2023-12-08T08:39:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.932+0000 7f49d2dc9640 1 -- 172.21.15.16:0/3319217261 shutdown_connections 2023-12-08T08:39:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.932+0000 7f49d2dc9640 1 -- 172.21.15.16:0/3319217261 >> 172.21.15.16:0/3319217261 conn(0x7f49cc0d0de0 msgr2=0x7f49cc0d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.932+0000 7f49d2dc9640 1 -- 172.21.15.16:0/3319217261 shutdown_connections 2023-12-08T08:39:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.932+0000 7f49d2dc9640 1 -- 172.21.15.16:0/3319217261 wait complete. 2023-12-08T08:39:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.933+0000 7f49d2dc9640 1 Processor -- start 2023-12-08T08:39:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.933+0000 7f49d2dc9640 1 -- start start 2023-12-08T08:39:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49d2dc9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49cc153310 0x7f49cc0b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49d2dc9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc0aff00 0x7f49cc0b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49d2dc9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49cc09d2f0 0x7f49cc09d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49d2dc9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc0ae760 con 0x7f49cc0aff00 2023-12-08T08:39:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49d2dc9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc09c250 con 0x7f49cc09d2f0 2023-12-08T08:39:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49d2dc9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc09c530 con 0x7f49cc153310 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc0aff00 0x7f49cc0b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49cbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc0aff00 0x7f49cc0b0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49276/0 (socket says 172.21.15.16:49276) 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49cbfff640 1 -- 172.21.15.16:0/809960998 learned_addr learned my addr 172.21.15.16:0/809960998 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49d0b65640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49cc153310 0x7f49cc0b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49d1366640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49cc09d2f0 0x7f49cc09d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49cbfff640 1 -- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49cc153310 msgr2=0x7f49cc0b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49cbfff640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49cc153310 0x7f49cc0b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49cbfff640 1 -- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49cc09d2f0 msgr2=0x7f49cc09d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49cbfff640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49cc09d2f0 0x7f49cc09d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49cbfff640 1 -- 172.21.15.16:0/809960998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc09c810 con 0x7f49cc0aff00 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49d1366640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49cc09d2f0 0x7f49cc09d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49cbfff640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc0aff00 0x7f49cc0b0380 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00aec0 tx=0x7f49bc007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49c97fa640 1 -- 172.21.15.16:0/809960998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc013600 con 0x7f49cc0aff00 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49c97fa640 1 -- 172.21.15.16:0/809960998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc013780 con 0x7f49cc0aff00 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49d2dc9640 1 -- 172.21.15.16:0/809960998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49cc09caf0 con 0x7f49cc0aff00 2023-12-08T08:39:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49c97fa640 1 -- 172.21.15.16:0/809960998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc013b30 con 0x7f49cc0aff00 2023-12-08T08:39:17.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.934+0000 7f49d2dc9640 1 -- 172.21.15.16:0/809960998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc0b08c0 con 0x7f49cc0aff00 2023-12-08T08:39:17.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.935+0000 7f49d2dc9640 1 -- 172.21.15.16:0/809960998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49cc09d010 con 0x7f49cc0aff00 2023-12-08T08:39:17.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.936+0000 7f49c97fa640 1 -- 172.21.15.16:0/809960998 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49bc013c90 con 0x7f49cc0aff00 2023-12-08T08:39:17.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.936+0000 7f49c97fa640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f49ac06ae70 0x7f49ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:17.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.936+0000 7f49c97fa640 1 -- 172.21.15.16:0/809960998 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f49bc08a860 con 0x7f49cc0aff00 2023-12-08T08:39:17.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.937+0000 7f49d0b65640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f49ac06ae70 0x7f49ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:17.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.937+0000 7f49d0b65640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f49ac06ae70 0x7f49ac06d330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0b70b0 tx=0x7f49c4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:17.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:17.940+0000 7f49c97fa640 1 -- 172.21.15.16:0/809960998 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49cc09d010 con 0x7f49cc0aff00 2023-12-08T08:39:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.130+0000 7f49d2dc9640 1 -- 172.21.15.16:0/809960998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f49cc0d0070 con 0x7f49cc0aff00 2023-12-08T08:39:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.130+0000 7f49c97fa640 1 -- 172.21.15.16:0/809960998 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f49cc0d0070 con 0x7f49cc0aff00 2023-12-08T08:39:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-12-08T08:39:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.132+0000 7f49d2dc9640 1 -- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f49ac06ae70 msgr2=0x7f49ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.132+0000 7f49d2dc9640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f49ac06ae70 0x7f49ac06d330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0b70b0 tx=0x7f49c4008040 comp rx=0 tx=0).stop 2023-12-08T08:39:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.132+0000 7f49d2dc9640 1 -- 172.21.15.16:0/809960998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc0aff00 msgr2=0x7f49cc0b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.132+0000 7f49d2dc9640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc0aff00 0x7f49cc0b0380 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00aec0 tx=0x7f49bc007430 comp rx=0 tx=0).stop 2023-12-08T08:39:18.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.132+0000 7f49d2dc9640 1 -- 172.21.15.16:0/809960998 shutdown_connections 2023-12-08T08:39:18.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.132+0000 7f49d2dc9640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f49ac06ae70 0x7f49ac06d330 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:18.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.132+0000 7f49d2dc9640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f49cc09d2f0 0x7f49cc09d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:18.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.132+0000 7f49d2dc9640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49cc0aff00 0x7f49cc0b0380 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:18.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.132+0000 7f49d2dc9640 1 --2- 172.21.15.16:0/809960998 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f49cc153310 0x7f49cc0b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:18.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.132+0000 7f49d2dc9640 1 -- 172.21.15.16:0/809960998 >> 172.21.15.16:0/809960998 conn(0x7f49cc0d0de0 msgr2=0x7f49cc06f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:18.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.132+0000 7f49d2dc9640 1 -- 172.21.15.16:0/809960998 shutdown_connections 2023-12-08T08:39:18.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.132+0000 7f49d2dc9640 1 -- 172.21.15.16:0/809960998 wait complete. 2023-12-08T08:39:18.144 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi112.front.sepia.ceph.com... 2023-12-08T08:39:18.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:18.145 DEBUG:teuthology.orchestra.run.smithi112:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-08T08:39:18.177 INFO:teuthology.orchestra.run:waiting for 300 2023-12-08T08:39:18.177 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-12-08T08:39:18.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:18.178 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-08T08:39:18.470 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:39:18.470 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T08:39:18.487 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:39:18.487 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T08:39:18.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.693+0000 7f4fb4210640 1 Processor -- start 2023-12-08T08:39:18.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.693+0000 7f4fb4210640 1 -- start start 2023-12-08T08:39:18.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.694+0000 7f4fb4210640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4fac160ca0 0x7f4fac161080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.694+0000 7f4fb4210640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4fac0ad710 0x7f4fac15ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.694+0000 7f4fb4210640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fac06aee0 con 0x7f4fac0ad290 2023-12-08T08:39:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.694+0000 7f4fb4210640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac06bd40 con 0x7f4fac160ca0 2023-12-08T08:39:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.694+0000 7f4fb4210640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fac1615c0 con 0x7f4fac0ad710 2023-12-08T08:39:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.694+0000 7f4fb1fac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4fac160ca0 0x7f4fac161080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.694+0000 7f4fb1fac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4fac160ca0 0x7f4fac161080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49278/0 (socket says 172.21.15.16:49278) 2023-12-08T08:39:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.694+0000 7f4fb1fac640 1 -- 172.21.15.16:0/1611165774 learned_addr learned my addr 172.21.15.16:0/1611165774 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.694+0000 7f4fb27ad640 1 --2- 172.21.15.16:0/1611165774 >> v2:172.21.15.112:3300/0 conn(0x7f4fac0ad710 0x7f4fac15ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.694+0000 7f4fb1fac640 1 -- 172.21.15.16:0/1611165774 >> v2:172.21.15.112:3300/0 conn(0x7f4fac0ad710 msgr2=0x7f4fac15ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:18.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.694+0000 7f4fb1fac640 1 --2- 172.21.15.16:0/1611165774 >> v2:172.21.15.112:3300/0 conn(0x7f4fac0ad710 0x7f4fac15ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.694+0000 7f4fb1fac640 1 -- 172.21.15.16:0/1611165774 >> v1:172.21.15.112:6789/0 conn(0x7f4fac0ad290 legacy=0x7f4fac15d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.694+0000 7f4fb1fac640 1 -- 172.21.15.16:0/1611165774 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fac06b130 con 0x7f4fac160ca0 2023-12-08T08:39:18.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.694+0000 7f4fb27ad640 1 --2- 172.21.15.16:0/1611165774 >> v2:172.21.15.112:3300/0 conn(0x7f4fac0ad710 0x7f4fac15ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.695+0000 7f4fb1fac640 1 --2- 172.21.15.16:0/1611165774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac160ca0 0x7f4fac161080 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0af140 tx=0x7f4fa802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a93112f707e15 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.695+0000 7f4fb0faa640 1 -- 172.21.15.16:0/1611165774 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8031870 con 0x7f4fac160ca0 2023-12-08T08:39:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.695+0000 7f4fb0faa640 1 -- 172.21.15.16:0/1611165774 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa80319d0 con 0x7f4fac160ca0 2023-12-08T08:39:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.695+0000 7f4fb0faa640 1 -- 172.21.15.16:0/1611165774 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8031d60 con 0x7f4fac160ca0 2023-12-08T08:39:18.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.696+0000 7f4fb4210640 1 -- 172.21.15.16:0/1611165774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac160ca0 msgr2=0x7f4fac161080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.696+0000 7f4fb4210640 1 --2- 172.21.15.16:0/1611165774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac160ca0 0x7f4fac161080 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0af140 tx=0x7f4fa802b910 comp rx=0 tx=0).stop 2023-12-08T08:39:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.696+0000 7f4fb4210640 1 -- 172.21.15.16:0/1611165774 shutdown_connections 2023-12-08T08:39:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.696+0000 7f4fb4210640 1 --2- 172.21.15.16:0/1611165774 >> v2:172.21.15.112:3300/0 conn(0x7f4fac0ad710 0x7f4fac15ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.696+0000 7f4fb4210640 1 --2- 172.21.15.16:0/1611165774 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac160ca0 0x7f4fac161080 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.696+0000 7f4fb4210640 1 -- 172.21.15.16:0/1611165774 >> 172.21.15.16:0/1611165774 conn(0x7f4fac0cf470 msgr2=0x7f4fac0cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.696+0000 7f4fb4210640 1 -- 172.21.15.16:0/1611165774 shutdown_connections 2023-12-08T08:39:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.696+0000 7f4fb4210640 1 -- 172.21.15.16:0/1611165774 wait complete. 2023-12-08T08:39:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.697+0000 7f4fb4210640 1 Processor -- start 2023-12-08T08:39:18.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.697+0000 7f4fb4210640 1 -- start start 2023-12-08T08:39:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb4210640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac0ad290 0x7f4fac150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb4210640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fac0ad710 0x7f4fac15f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb4210640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4fac160ca0 0x7f4fac15fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb4210640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac1505c0 con 0x7f4fac0ad290 2023-12-08T08:39:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb4210640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac156080 con 0x7f4fac0ad710 2023-12-08T08:39:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb4210640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac1f3f00 con 0x7f4fac160ca0 2023-12-08T08:39:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb1fac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac0ad290 0x7f4fac150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb1fac640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac0ad290 0x7f4fac150080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49284/0 (socket says 172.21.15.16:49284) 2023-12-08T08:39:18.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb1fac640 1 -- 172.21.15.16:0/3234248087 learned_addr learned my addr 172.21.15.16:0/3234248087 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb17ab640 1 --2- 172.21.15.16:0/3234248087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fac0ad710 0x7f4fac15f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb27ad640 1 --2- 172.21.15.16:0/3234248087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4fac160ca0 0x7f4fac15fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb1fac640 1 -- 172.21.15.16:0/3234248087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4fac160ca0 msgr2=0x7f4fac15fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb1fac640 1 --2- 172.21.15.16:0/3234248087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4fac160ca0 0x7f4fac15fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb1fac640 1 -- 172.21.15.16:0/3234248087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fac0ad710 msgr2=0x7f4fac15f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb1fac640 1 --2- 172.21.15.16:0/3234248087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fac0ad710 0x7f4fac15f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.698+0000 7f4fb1fac640 1 -- 172.21.15.16:0/3234248087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa80094d0 con 0x7f4fac0ad290 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.699+0000 7f4fb1fac640 1 --2- 172.21.15.16:0/3234248087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac0ad290 0x7f4fac150080 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4fa802b950 tx=0x7f4fa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.699+0000 7f4fa27fc640 1 -- 172.21.15.16:0/3234248087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa8031860 con 0x7f4fac0ad290 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.699+0000 7f4fa27fc640 1 -- 172.21.15.16:0/3234248087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fa80443f0 con 0x7f4fac0ad290 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.699+0000 7f4fb4210640 1 -- 172.21.15.16:0/3234248087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac1559c0 con 0x7f4fac0ad290 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.699+0000 7f4fa27fc640 1 -- 172.21.15.16:0/3234248087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa80447a0 con 0x7f4fac0ad290 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.699+0000 7f4fb4210640 1 -- 172.21.15.16:0/3234248087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac1556e0 con 0x7f4fac0ad290 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.700+0000 7f4fb4210640 1 -- 172.21.15.16:0/3234248087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fac067430 con 0x7f4fac0ad290 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.700+0000 7f4fa27fc640 1 -- 172.21.15.16:0/3234248087 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4fa804d450 con 0x7f4fac0ad290 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.701+0000 7f4fa27fc640 1 --2- 172.21.15.16:0/3234248087 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4f9006ae40 0x7f4f9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:18.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.701+0000 7f4fa27fc640 1 -- 172.21.15.16:0/3234248087 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f4fa802d040 con 0x7f4fac0ad290 2023-12-08T08:39:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.701+0000 7f4fb17ab640 1 --2- 172.21.15.16:0/3234248087 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4f9006ae40 0x7f4f9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:18.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.701+0000 7f4fb17ab640 1 --2- 172.21.15.16:0/3234248087 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4f9006ae40 0x7f4f9006d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0cbac0 tx=0x7f4f9c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:18.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.705+0000 7f4fa27fc640 1 -- 172.21.15.16:0/3234248087 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4fa8076e20 con 0x7f4fac0ad290 2023-12-08T08:39:18.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.884+0000 7f4fb4210640 1 -- 172.21.15.16:0/3234248087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fac073ce0 con 0x7f4fac0ad290 2023-12-08T08:39:18.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.885+0000 7f4fa27fc640 1 -- 172.21.15.16:0/3234248087 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7f4fa8035020 con 0x7f4fac0ad290 2023-12-08T08:39:18.887 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:18.887 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.887+0000 7f4fb4210640 1 -- 172.21.15.16:0/3234248087 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4f9006ae40 msgr2=0x7f4f9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.887+0000 7f4fb4210640 1 --2- 172.21.15.16:0/3234248087 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4f9006ae40 0x7f4f9006d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4fac0cbac0 tx=0x7f4f9c009040 comp rx=0 tx=0).stop 2023-12-08T08:39:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.887+0000 7f4fb4210640 1 -- 172.21.15.16:0/3234248087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac0ad290 msgr2=0x7f4fac150080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.887+0000 7f4fb4210640 1 --2- 172.21.15.16:0/3234248087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac0ad290 0x7f4fac150080 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4fa802b950 tx=0x7f4fa80046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.887+0000 7f4fb4210640 1 -- 172.21.15.16:0/3234248087 shutdown_connections 2023-12-08T08:39:18.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.887+0000 7f4fb4210640 1 --2- 172.21.15.16:0/3234248087 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4f9006ae40 0x7f4f9006d300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.887+0000 7f4fb4210640 1 --2- 172.21.15.16:0/3234248087 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4fac160ca0 0x7f4fac15fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.887+0000 7f4fb4210640 1 --2- 172.21.15.16:0/3234248087 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4fac0ad710 0x7f4fac15f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.887+0000 7f4fb4210640 1 --2- 172.21.15.16:0/3234248087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fac0ad290 0x7f4fac150080 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.887+0000 7f4fb4210640 1 -- 172.21.15.16:0/3234248087 >> 172.21.15.16:0/3234248087 conn(0x7f4fac0cf470 msgr2=0x7f4fac152c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.887+0000 7f4fb4210640 1 -- 172.21.15.16:0/3234248087 shutdown_connections 2023-12-08T08:39:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:18.887+0000 7f4fb4210640 1 -- 172.21.15.16:0/3234248087 wait complete. 2023-12-08T08:39:18.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:39:18.900 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:19.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.008+0000 7f6243410640 1 Processor -- start 2023-12-08T08:39:19.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.008+0000 7f6243410640 1 -- start start 2023-12-08T08:39:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.009+0000 7f6243410640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f623c0b8950 0x7f623c0ce680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.009+0000 7f6243410640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f623c0cfc80 0x7f623c0b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.009+0000 7f6243410640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623c0b5f00 con 0x7f623c0cebc0 2023-12-08T08:39:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.009+0000 7f6243410640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623c0b6170 con 0x7f623c0b8950 2023-12-08T08:39:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.009+0000 7f6243410640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623c0b6450 con 0x7f623c0cfc80 2023-12-08T08:39:19.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.009+0000 7f62409ab640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f623c0cebc0 0x7f623c0cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34260/0 (socket says 172.21.15.16:34260) 2023-12-08T08:39:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.009+0000 7f62411ac640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f623c0b8950 0x7f623c0ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.009+0000 7f62409ab640 1 -- 172.21.15.16:0/2113908997 learned_addr learned my addr 172.21.15.16:0/2113908997 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.009+0000 7f62419ad640 1 --2- 172.21.15.16:0/2113908997 >> v2:172.21.15.112:3300/0 conn(0x7f623c0cfc80 0x7f623c0b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.009+0000 7f62411ac640 1 -- 172.21.15.16:0/2113908997 >> v2:172.21.15.112:3300/0 conn(0x7f623c0cfc80 msgr2=0x7f623c0b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.009+0000 7f62411ac640 1 --2- 172.21.15.16:0/2113908997 >> v2:172.21.15.112:3300/0 conn(0x7f623c0cfc80 0x7f623c0b6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.009+0000 7f62411ac640 1 -- 172.21.15.16:0/2113908997 >> v1:172.21.15.16:6789/0 conn(0x7f623c0cebc0 legacy=0x7f623c0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.009+0000 7f6233fff640 1 -- 172.21.15.16:0/2113908997 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576167101 0 0) 0x7f623c0b5f00 con 0x7f623c0cebc0 2023-12-08T08:39:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.009+0000 7f62411ac640 1 -- 172.21.15.16:0/2113908997 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c0b6730 con 0x7f623c0b8950 2023-12-08T08:39:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.010+0000 7f62411ac640 1 --2- 172.21.15.16:0/2113908997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623c0b8950 0x7f623c0ce680 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f623c08d210 tx=0x7f623802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57239bb469217f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.010+0000 7f6233fff640 1 -- 172.21.15.16:0/2113908997 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623802e070 con 0x7f623c0b8950 2023-12-08T08:39:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.010+0000 7f6233fff640 1 -- 172.21.15.16:0/2113908997 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62380315a0 con 0x7f623c0b8950 2023-12-08T08:39:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.010+0000 7f6233fff640 1 -- 172.21.15.16:0/2113908997 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238031930 con 0x7f623c0b8950 2023-12-08T08:39:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.010+0000 7f6243410640 1 -- 172.21.15.16:0/2113908997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623c0b8950 msgr2=0x7f623c0ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.010+0000 7f6243410640 1 --2- 172.21.15.16:0/2113908997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623c0b8950 0x7f623c0ce680 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f623c08d210 tx=0x7f623802b840 comp rx=0 tx=0).stop 2023-12-08T08:39:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.011+0000 7f6243410640 1 -- 172.21.15.16:0/2113908997 shutdown_connections 2023-12-08T08:39:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.011+0000 7f6243410640 1 --2- 172.21.15.16:0/2113908997 >> v2:172.21.15.112:3300/0 conn(0x7f623c0cfc80 0x7f623c0b6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.011+0000 7f6243410640 1 --2- 172.21.15.16:0/2113908997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623c0b8950 0x7f623c0ce680 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.011+0000 7f6243410640 1 -- 172.21.15.16:0/2113908997 >> 172.21.15.16:0/2113908997 conn(0x7f623c0d3010 msgr2=0x7f623c0d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.011+0000 7f6243410640 1 -- 172.21.15.16:0/2113908997 shutdown_connections 2023-12-08T08:39:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.011+0000 7f6243410640 1 -- 172.21.15.16:0/2113908997 wait complete. 2023-12-08T08:39:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.012+0000 7f6243410640 1 Processor -- start 2023-12-08T08:39:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.012+0000 7f6243410640 1 -- start start 2023-12-08T08:39:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f6243410640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f623c0b8950 0x7f623c0cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f6243410640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623c0cebc0 0x7f623c1f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f6243410640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f623c0cfc80 0x7f623c09d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f6243410640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c09ccb0 con 0x7f623c0cebc0 2023-12-08T08:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f6243410640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c1f35b0 con 0x7f623c0b8950 2023-12-08T08:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f6243410640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c1f3890 con 0x7f623c0cfc80 2023-12-08T08:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f62409ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623c0cebc0 0x7f623c1f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f62409ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623c0cebc0 0x7f623c1f2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49306/0 (socket says 172.21.15.16:49306) 2023-12-08T08:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f62409ab640 1 -- 172.21.15.16:0/409503694 learned_addr learned my addr 172.21.15.16:0/409503694 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f62411ac640 1 --2- 172.21.15.16:0/409503694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f623c0b8950 0x7f623c0cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f62419ad640 1 --2- 172.21.15.16:0/409503694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f623c0cfc80 0x7f623c09d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f62409ab640 1 -- 172.21.15.16:0/409503694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f623c0cfc80 msgr2=0x7f623c09d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f62409ab640 1 --2- 172.21.15.16:0/409503694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f623c0cfc80 0x7f623c09d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f62409ab640 1 -- 172.21.15.16:0/409503694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f623c0b8950 msgr2=0x7f623c0cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f62409ab640 1 --2- 172.21.15.16:0/409503694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f623c0b8950 0x7f623c0cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f62409ab640 1 -- 172.21.15.16:0/409503694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6238009400 con 0x7f623c0cebc0 2023-12-08T08:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f62409ab640 1 --2- 172.21.15.16:0/409503694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623c0cebc0 0x7f623c1f2ca0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f622c00c4b0 tx=0x7f622c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f6231ffb640 1 -- 172.21.15.16:0/409503694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c0074e0 con 0x7f623c0cebc0 2023-12-08T08:39:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f6231ffb640 1 -- 172.21.15.16:0/409503694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c007660 con 0x7f623c0cebc0 2023-12-08T08:39:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f6243410640 1 -- 172.21.15.16:0/409503694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c1f3b70 con 0x7f623c0cebc0 2023-12-08T08:39:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.013+0000 7f6231ffb640 1 -- 172.21.15.16:0/409503694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c004a20 con 0x7f623c0cebc0 2023-12-08T08:39:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.014+0000 7f6243410640 1 -- 172.21.15.16:0/409503694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c1f31e0 con 0x7f623c0cebc0 2023-12-08T08:39:19.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.015+0000 7f6243410640 1 -- 172.21.15.16:0/409503694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c08b150 con 0x7f623c0cebc0 2023-12-08T08:39:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.015+0000 7f6231ffb640 1 -- 172.21.15.16:0/409503694 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f622c004b80 con 0x7f623c0cebc0 2023-12-08T08:39:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.015+0000 7f6231ffb640 1 --2- 172.21.15.16:0/409503694 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f622006ae40 0x7f622006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.015+0000 7f6231ffb640 1 -- 172.21.15.16:0/409503694 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f622c088ed0 con 0x7f623c0cebc0 2023-12-08T08:39:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.016+0000 7f62411ac640 1 --2- 172.21.15.16:0/409503694 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f622006ae40 0x7f622006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.016+0000 7f62411ac640 1 --2- 172.21.15.16:0/409503694 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f622006ae40 0x7f622006d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6238036000 tx=0x7f6238008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.020+0000 7f6231ffb640 1 -- 172.21.15.16:0/409503694 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f622c053cf0 con 0x7f623c0cebc0 2023-12-08T08:39:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.194+0000 7f6243410640 1 -- 172.21.15.16:0/409503694 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f623c067430 con 0x7f623c0cebc0 2023-12-08T08:39:19.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.194+0000 7f6231ffb640 1 -- 172.21.15.16:0/409503694 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f622c018020 con 0x7f623c0cebc0 2023-12-08T08:39:19.196 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:19.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.196+0000 7f6243410640 1 -- 172.21.15.16:0/409503694 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f622006ae40 msgr2=0x7f622006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.196+0000 7f6243410640 1 --2- 172.21.15.16:0/409503694 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f622006ae40 0x7f622006d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6238036000 tx=0x7f6238008030 comp rx=0 tx=0).stop 2023-12-08T08:39:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.196+0000 7f6243410640 1 -- 172.21.15.16:0/409503694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623c0cebc0 msgr2=0x7f623c1f2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.196+0000 7f6243410640 1 --2- 172.21.15.16:0/409503694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623c0cebc0 0x7f623c1f2ca0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f622c00c4b0 tx=0x7f622c00c970 comp rx=0 tx=0).stop 2023-12-08T08:39:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.196+0000 7f6243410640 1 -- 172.21.15.16:0/409503694 shutdown_connections 2023-12-08T08:39:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.196+0000 7f6243410640 1 --2- 172.21.15.16:0/409503694 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f622006ae40 0x7f622006d300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.196+0000 7f6243410640 1 --2- 172.21.15.16:0/409503694 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f623c0cfc80 0x7f623c09d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.196+0000 7f6243410640 1 --2- 172.21.15.16:0/409503694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f623c0cebc0 0x7f623c1f2ca0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.196+0000 7f6243410640 1 --2- 172.21.15.16:0/409503694 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f623c0b8950 0x7f623c0cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.196+0000 7f6243410640 1 -- 172.21.15.16:0/409503694 >> 172.21.15.16:0/409503694 conn(0x7f623c0d3010 msgr2=0x7f623c0cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.196+0000 7f6243410640 1 -- 172.21.15.16:0/409503694 shutdown_connections 2023-12-08T08:39:19.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.196+0000 7f6243410640 1 -- 172.21.15.16:0/409503694 wait complete. 2023-12-08T08:39:19.209 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:39:19.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.316+0000 7fcd55161640 1 Processor -- start 2023-12-08T08:39:19.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.316+0000 7fcd55161640 1 -- start start 2023-12-08T08:39:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.317+0000 7fcd55161640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fcd5006d7d0 0x7fcd50164d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.317+0000 7fcd55161640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd500b2b60 con 0x7fcd500cb370 2023-12-08T08:39:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.317+0000 7fcd55161640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd5015e7f0 con 0x7fcd50157a20 2023-12-08T08:39:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.317+0000 7fcd55161640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd50157e00 con 0x7fcd5006d7d0 2023-12-08T08:39:19.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.317+0000 7fcd4e59c640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcd500cb370 0x7fcd5015e0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34274/0 (socket says 172.21.15.16:34274) 2023-12-08T08:39:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.317+0000 7fcd4e59c640 1 -- 172.21.15.16:0/2101235063 learned_addr learned my addr 172.21.15.16:0/2101235063 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.317+0000 7fcd4f59e640 1 --2- 172.21.15.16:0/2101235063 >> v2:172.21.15.112:3300/0 conn(0x7fcd5006d7d0 0x7fcd50164d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 -- 172.21.15.16:0/2101235063 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2573166914 0 0) 0x7fcd500b2b60 con 0x7fcd500cb370 2023-12-08T08:39:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 -- 172.21.15.16:0/2101235063 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd3c003520 con 0x7fcd500cb370 2023-12-08T08:39:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 -- 172.21.15.16:0/2101235063 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 828478887 0 0) 0x7fcd3c003520 con 0x7fcd500cb370 2023-12-08T08:39:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 -- 172.21.15.16:0/2101235063 >> v2:172.21.15.112:3300/0 conn(0x7fcd5006d7d0 msgr2=0x7fcd50164d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 --2- 172.21.15.16:0/2101235063 >> v2:172.21.15.112:3300/0 conn(0x7fcd5006d7d0 0x7fcd50164d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 -- 172.21.15.16:0/2101235063 >> v1:172.21.15.112:6789/0 conn(0x7fcd50157a20 legacy=0x7fcd5015d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 -- 172.21.15.16:0/2101235063 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5006dd20 con 0x7fcd500cb370 2023-12-08T08:39:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 -- 172.21.15.16:0/2101235063 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7fcd38001670 con 0x7fcd500cb370 2023-12-08T08:39:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 -- 172.21.15.16:0/2101235063 >> v1:172.21.15.16:6789/0 conn(0x7fcd500cb370 legacy=0x7fcd5015e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 --2- 172.21.15.16:0/2101235063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd3c003970 0x7fcd3c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 --2- 172.21.15.16:0/2101235063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd3c0047c0 0x7fcd3c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 --2- 172.21.15.16:0/2101235063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd3c005180 0x7fcd3c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 -- 172.21.15.16:0/2101235063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd50157e00 con 0x7fcd3c003970 2023-12-08T08:39:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 -- 172.21.15.16:0/2101235063 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c00e230 con 0x7fcd3c0047c0 2023-12-08T08:39:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4dd9b640 1 -- 172.21.15.16:0/2101235063 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c00e3f0 con 0x7fcd3c005180 2023-12-08T08:39:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4ed9d640 1 --2- 172.21.15.16:0/2101235063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd3c003970 0x7fcd3c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4e59c640 1 --2- 172.21.15.16:0/2101235063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd3c0047c0 0x7fcd3c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4f59e640 1 --2- 172.21.15.16:0/2101235063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd3c005180 0x7fcd3c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4ed9d640 1 -- 172.21.15.16:0/2101235063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd3c005180 msgr2=0x7fcd3c00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4ed9d640 1 --2- 172.21.15.16:0/2101235063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd3c005180 0x7fcd3c00db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4ed9d640 1 -- 172.21.15.16:0/2101235063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd3c0047c0 msgr2=0x7fcd3c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4ed9d640 1 --2- 172.21.15.16:0/2101235063 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd3c0047c0 0x7fcd3c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4ed9d640 1 -- 172.21.15.16:0/2101235063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c003520 con 0x7fcd3c003970 2023-12-08T08:39:19.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4f59e640 1 --2- 172.21.15.16:0/2101235063 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd3c005180 0x7fcd3c00db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.318+0000 7fcd4f59e640 1 -- 172.21.15.16:0/2101235063 reap_dead start 2023-12-08T08:39:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.319+0000 7fcd4ed9d640 1 --2- 172.21.15.16:0/2101235063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd3c003970 0x7fcd3c003d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcd48001750 tx=0x7fcd3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89fb32d96dde4d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.319+0000 7fcd4dd9b640 1 -- 172.21.15.16:0/2101235063 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd480052d0 con 0x7fcd3c003970 2023-12-08T08:39:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.319+0000 7fcd4dd9b640 1 -- 172.21.15.16:0/2101235063 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4802e050 con 0x7fcd3c003970 2023-12-08T08:39:19.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.319+0000 7fcd4dd9b640 1 -- 172.21.15.16:0/2101235063 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd48003830 con 0x7fcd3c003970 2023-12-08T08:39:19.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.320+0000 7fcd55161640 1 -- 172.21.15.16:0/2101235063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd3c003970 msgr2=0x7fcd3c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.320+0000 7fcd55161640 1 --2- 172.21.15.16:0/2101235063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd3c003970 0x7fcd3c003d90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcd48001750 tx=0x7fcd3c0057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.320+0000 7fcd55161640 1 -- 172.21.15.16:0/2101235063 shutdown_connections 2023-12-08T08:39:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.320+0000 7fcd55161640 1 --2- 172.21.15.16:0/2101235063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd3c003970 0x7fcd3c003d90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.320+0000 7fcd55161640 1 -- 172.21.15.16:0/2101235063 >> 172.21.15.16:0/2101235063 conn(0x7fcd500b0850 msgr2=0x7fcd500b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.320+0000 7fcd55161640 1 -- 172.21.15.16:0/2101235063 shutdown_connections 2023-12-08T08:39:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.320+0000 7fcd55161640 1 -- 172.21.15.16:0/2101235063 wait complete. 2023-12-08T08:39:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.321+0000 7fcd55161640 1 Processor -- start 2023-12-08T08:39:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.321+0000 7fcd55161640 1 -- start start 2023-12-08T08:39:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.322+0000 7fcd55161640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd3c003970 0x7fcd5014e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.322+0000 7fcd55161640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd500b2110 0x7fcd500b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.322+0000 7fcd55161640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd500ae7c0 0x7fcd500aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.322+0000 7fcd55161640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd50154860 con 0x7fcd500ae7c0 2023-12-08T08:39:19.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.322+0000 7fcd55161640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500af180 con 0x7fcd500b2110 2023-12-08T08:39:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.322+0000 7fcd55161640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd500b2fe0 con 0x7fcd3c003970 2023-12-08T08:39:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.322+0000 7fcd4f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd500ae7c0 0x7fcd500aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.322+0000 7fcd4f59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd500ae7c0 0x7fcd500aec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49336/0 (socket says 172.21.15.16:49336) 2023-12-08T08:39:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.322+0000 7fcd4f59e640 1 -- 172.21.15.16:0/3261598124 learned_addr learned my addr 172.21.15.16:0/3261598124 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.322+0000 7fcd4ed9d640 1 --2- 172.21.15.16:0/3261598124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd3c003970 0x7fcd5014e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.322+0000 7fcd4e59c640 1 --2- 172.21.15.16:0/3261598124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd500b2110 0x7fcd500b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.323+0000 7fcd4f59e640 1 -- 172.21.15.16:0/3261598124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd3c003970 msgr2=0x7fcd5014e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.323+0000 7fcd4f59e640 1 --2- 172.21.15.16:0/3261598124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd3c003970 0x7fcd5014e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.323+0000 7fcd4f59e640 1 -- 172.21.15.16:0/3261598124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd500b2110 msgr2=0x7fcd500b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.323+0000 7fcd4f59e640 1 --2- 172.21.15.16:0/3261598124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd500b2110 0x7fcd500b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.323+0000 7fcd4f59e640 1 -- 172.21.15.16:0/3261598124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd500b3210 con 0x7fcd500ae7c0 2023-12-08T08:39:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.323+0000 7fcd4f59e640 1 --2- 172.21.15.16:0/3261598124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd500ae7c0 0x7fcd500aec40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcd500cba00 tx=0x7fcd40010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:19.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.323+0000 7fcd477fe640 1 -- 172.21.15.16:0/3261598124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4000e050 con 0x7fcd500ae7c0 2023-12-08T08:39:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.323+0000 7fcd477fe640 1 -- 172.21.15.16:0/3261598124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4000f040 con 0x7fcd500ae7c0 2023-12-08T08:39:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.323+0000 7fcd55161640 1 -- 172.21.15.16:0/3261598124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd500b34f0 con 0x7fcd500ae7c0 2023-12-08T08:39:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.323+0000 7fcd477fe640 1 -- 172.21.15.16:0/3261598124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40003990 con 0x7fcd500ae7c0 2023-12-08T08:39:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.323+0000 7fcd55161640 1 -- 172.21.15.16:0/3261598124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5015d300 con 0x7fcd500ae7c0 2023-12-08T08:39:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.324+0000 7fcd55161640 1 -- 172.21.15.16:0/3261598124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd500b3a10 con 0x7fcd500ae7c0 2023-12-08T08:39:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.324+0000 7fcd477fe640 1 -- 172.21.15.16:0/3261598124 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd40015440 con 0x7fcd500ae7c0 2023-12-08T08:39:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.325+0000 7fcd477fe640 1 --2- 172.21.15.16:0/3261598124 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcd3006ae40 0x7fcd3006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.325+0000 7fcd477fe640 1 -- 172.21.15.16:0/3261598124 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fcd4008a1e0 con 0x7fcd500ae7c0 2023-12-08T08:39:19.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.325+0000 7fcd4ed9d640 1 --2- 172.21.15.16:0/3261598124 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcd3006ae40 0x7fcd3006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.325+0000 7fcd4ed9d640 1 --2- 172.21.15.16:0/3261598124 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcd3006ae40 0x7fcd3006d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcd48033000 tx=0x7fcd48009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:19.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.329+0000 7fcd477fe640 1 -- 172.21.15.16:0/3261598124 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd40054ff0 con 0x7fcd500ae7c0 2023-12-08T08:39:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.504+0000 7fcd55161640 1 -- 172.21.15.16:0/3261598124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcd50067430 con 0x7fcd500ae7c0 2023-12-08T08:39:19.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.505+0000 7fcd477fe640 1 -- 172.21.15.16:0/3261598124 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd40058610 con 0x7fcd500ae7c0 2023-12-08T08:39:19.507 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.506+0000 7fcd55161640 1 -- 172.21.15.16:0/3261598124 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcd3006ae40 msgr2=0x7fcd3006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.506+0000 7fcd55161640 1 --2- 172.21.15.16:0/3261598124 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcd3006ae40 0x7fcd3006d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcd48033000 tx=0x7fcd48009030 comp rx=0 tx=0).stop 2023-12-08T08:39:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.506+0000 7fcd55161640 1 -- 172.21.15.16:0/3261598124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd500ae7c0 msgr2=0x7fcd500aec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.506+0000 7fcd55161640 1 --2- 172.21.15.16:0/3261598124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd500ae7c0 0x7fcd500aec40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcd500cba00 tx=0x7fcd40010590 comp rx=0 tx=0).stop 2023-12-08T08:39:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.506+0000 7fcd55161640 1 -- 172.21.15.16:0/3261598124 shutdown_connections 2023-12-08T08:39:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.506+0000 7fcd55161640 1 --2- 172.21.15.16:0/3261598124 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fcd3006ae40 0x7fcd3006d300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.506+0000 7fcd55161640 1 --2- 172.21.15.16:0/3261598124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd500ae7c0 0x7fcd500aec40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.506+0000 7fcd55161640 1 --2- 172.21.15.16:0/3261598124 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fcd500b2110 0x7fcd500b2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.506+0000 7fcd55161640 1 --2- 172.21.15.16:0/3261598124 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fcd3c003970 0x7fcd5014e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.506+0000 7fcd55161640 1 -- 172.21.15.16:0/3261598124 >> 172.21.15.16:0/3261598124 conn(0x7fcd500b0850 msgr2=0x7fcd500700b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.507+0000 7fcd55161640 1 -- 172.21.15.16:0/3261598124 shutdown_connections 2023-12-08T08:39:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.507+0000 7fcd55161640 1 -- 172.21.15.16:0/3261598124 wait complete. 2023-12-08T08:39:19.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:39:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.627+0000 7f77dbe1c640 1 Processor -- start 2023-12-08T08:39:19.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.627+0000 7f77dbe1c640 1 -- start start 2023-12-08T08:39:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.628+0000 7f77dbe1c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77d40cfa30 0x7f77d40b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.628+0000 7f77dbe1c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d40c6be0 con 0x7f77d40cfe10 2023-12-08T08:39:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.628+0000 7f77dbe1c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d40d3ff0 con 0x7f77d40b4150 2023-12-08T08:39:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.628+0000 7f77dbe1c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d40b4860 con 0x7f77d40cfa30 2023-12-08T08:39:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.628+0000 7f77da3b9640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f77d40cfe10 0x7f77d40cdf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34280/0 (socket says 172.21.15.16:34280) 2023-12-08T08:39:19.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.628+0000 7f77da3b9640 1 -- 172.21.15.16:0/720900077 learned_addr learned my addr 172.21.15.16:0/720900077 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d9bb8640 1 --2- 172.21.15.16:0/720900077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77d40cfa30 0x7f77d40b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 -- 172.21.15.16:0/720900077 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648218994 0 0) 0x7f77d40c6be0 con 0x7f77d40cfe10 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 -- 172.21.15.16:0/720900077 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77bc003520 con 0x7f77d40cfe10 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 -- 172.21.15.16:0/720900077 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3729711088 0 0) 0x7f77bc003520 con 0x7f77d40cfe10 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 -- 172.21.15.16:0/720900077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77d40cfa30 msgr2=0x7f77d40b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 --2- 172.21.15.16:0/720900077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77d40cfa30 0x7f77d40b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 -- 172.21.15.16:0/720900077 >> v1:172.21.15.112:6789/0 conn(0x7f77d40b4150 legacy=0x7f77d40b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 -- 172.21.15.16:0/720900077 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d40d3d50 con 0x7f77d40cfe10 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 -- 172.21.15.16:0/720900077 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f77c40019f0 con 0x7f77d40cfe10 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 -- 172.21.15.16:0/720900077 >> v1:172.21.15.16:6789/0 conn(0x7f77d40cfe10 legacy=0x7f77d40cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 --2- 172.21.15.16:0/720900077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77bc0039a0 0x7f77bc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 --2- 172.21.15.16:0/720900077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77d40cfa30 0x7f77bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 --2- 172.21.15.16:0/720900077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc004d00 0x7f77bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 -- 172.21.15.16:0/720900077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d40b4860 con 0x7f77bc004d00 2023-12-08T08:39:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 -- 172.21.15.16:0/720900077 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc00bab0 con 0x7f77bc0039a0 2023-12-08T08:39:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d8bb6640 1 -- 172.21.15.16:0/720900077 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77bc00bc70 con 0x7f77d40cfa30 2023-12-08T08:39:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77da3b9640 1 --2- 172.21.15.16:0/720900077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc004d00 0x7f77bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d9bb8640 1 --2- 172.21.15.16:0/720900077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77bc0039a0 0x7f77bc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d93b7640 1 --2- 172.21.15.16:0/720900077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77d40cfa30 0x7f77bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77da3b9640 1 -- 172.21.15.16:0/720900077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77d40cfa30 msgr2=0x7f77bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77da3b9640 1 --2- 172.21.15.16:0/720900077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77d40cfa30 0x7f77bc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77da3b9640 1 -- 172.21.15.16:0/720900077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77bc0039a0 msgr2=0x7f77bc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77da3b9640 1 --2- 172.21.15.16:0/720900077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77bc0039a0 0x7f77bc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77da3b9640 1 -- 172.21.15.16:0/720900077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77bc003520 con 0x7f77bc004d00 2023-12-08T08:39:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.629+0000 7f77d93b7640 1 --2- 172.21.15.16:0/720900077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77d40cfa30 0x7f77bc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.630+0000 7f77da3b9640 1 --2- 172.21.15.16:0/720900077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc004d00 0x7f77bc005180 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f77c400a820 tx=0x7f77bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5018546b9e1adf server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.630+0000 7f77d8bb6640 1 -- 172.21.15.16:0/720900077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c40072d0 con 0x7f77bc004d00 2023-12-08T08:39:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.630+0000 7f77d8bb6640 1 -- 172.21.15.16:0/720900077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c402f070 con 0x7f77bc004d00 2023-12-08T08:39:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.630+0000 7f77d8bb6640 1 -- 172.21.15.16:0/720900077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4007650 con 0x7f77bc004d00 2023-12-08T08:39:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.631+0000 7f77dbe1c640 1 -- 172.21.15.16:0/720900077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc004d00 msgr2=0x7f77bc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.631+0000 7f77dbe1c640 1 --2- 172.21.15.16:0/720900077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77bc004d00 0x7f77bc005180 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f77c400a820 tx=0x7f77bc0057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.631+0000 7f77da3b9640 1 -- 172.21.15.16:0/720900077 reap_dead start 2023-12-08T08:39:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.631+0000 7f77dbe1c640 1 -- 172.21.15.16:0/720900077 shutdown_connections 2023-12-08T08:39:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.631+0000 7f77dbe1c640 1 -- 172.21.15.16:0/720900077 >> 172.21.15.16:0/720900077 conn(0x7f77d40d3090 msgr2=0x7f77d40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.631+0000 7f77dbe1c640 1 -- 172.21.15.16:0/720900077 shutdown_connections 2023-12-08T08:39:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.631+0000 7f77dbe1c640 1 -- 172.21.15.16:0/720900077 wait complete. 2023-12-08T08:39:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.632+0000 7f77dbe1c640 1 Processor -- start 2023-12-08T08:39:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.632+0000 7f77dbe1c640 1 -- start start 2023-12-08T08:39:19.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77dbe1c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d41524b0 0x7f77d4155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77dbe1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d4152890 0x7f77d4154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77dbe1c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77d4154850 0x7f77d40caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77dbe1c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d40caa50 con 0x7f77d4152890 2023-12-08T08:39:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77dbe1c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d415efb0 con 0x7f77d41524b0 2023-12-08T08:39:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77dbe1c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d40ca400 con 0x7f77d4154850 2023-12-08T08:39:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77d93b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d4152890 0x7f77d4154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77d93b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d4152890 0x7f77d4154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49354/0 (socket says 172.21.15.16:49354) 2023-12-08T08:39:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77d93b7640 1 -- 172.21.15.16:0/373624904 learned_addr learned my addr 172.21.15.16:0/373624904 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77da3b9640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77d4154850 0x7f77d40caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77d9bb8640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d41524b0 0x7f77d4155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77d93b7640 1 -- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77d4154850 msgr2=0x7f77d40caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77d93b7640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77d4154850 0x7f77d40caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77d93b7640 1 -- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d41524b0 msgr2=0x7f77d4155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77d93b7640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d41524b0 0x7f77d4155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77d93b7640 1 -- 172.21.15.16:0/373624904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d40ca6e0 con 0x7f77d4152890 2023-12-08T08:39:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.633+0000 7f77d9bb8640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d41524b0 0x7f77d4155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.634+0000 7f77d93b7640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d4152890 0x7f77d4154310 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f77c800e600 tx=0x7f77c800eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.634+0000 7f77d27fc640 1 -- 172.21.15.16:0/373624904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80052e0 con 0x7f77d4152890 2023-12-08T08:39:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.634+0000 7f77d27fc640 1 -- 172.21.15.16:0/373624904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c8005440 con 0x7f77d4152890 2023-12-08T08:39:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.634+0000 7f77dbe1c640 1 -- 172.21.15.16:0/373624904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d4154ce0 con 0x7f77d4152890 2023-12-08T08:39:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.634+0000 7f77d27fc640 1 -- 172.21.15.16:0/373624904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80037f0 con 0x7f77d4152890 2023-12-08T08:39:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.634+0000 7f77dbe1c640 1 -- 172.21.15.16:0/373624904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d40cb600 con 0x7f77d4152890 2023-12-08T08:39:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.635+0000 7f77dbe1c640 1 -- 172.21.15.16:0/373624904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d40cabc0 con 0x7f77d4152890 2023-12-08T08:39:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.635+0000 7f77d27fc640 1 -- 172.21.15.16:0/373624904 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f77c8003950 con 0x7f77d4152890 2023-12-08T08:39:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.636+0000 7f77d27fc640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f77c006ae70 0x7f77c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.636+0000 7f77d27fc640 1 -- 172.21.15.16:0/373624904 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f77c80897f0 con 0x7f77d4152890 2023-12-08T08:39:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.636+0000 7f77d9bb8640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f77c006ae70 0x7f77c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.636+0000 7f77d9bb8640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f77c006ae70 0x7f77c006d330 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009a40 tx=0x7f77cc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:19.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.640+0000 7f77d27fc640 1 -- 172.21.15.16:0/373624904 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f77c8054550 con 0x7f77d4152890 2023-12-08T08:39:19.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.818+0000 7f77dbe1c640 1 -- 172.21.15.16:0/373624904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77d4067430 con 0x7f77d4152890 2023-12-08T08:39:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.819+0000 7f77d27fc640 1 -- 172.21.15.16:0/373624904 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77c8057b70 con 0x7f77d4152890 2023-12-08T08:39:19.821 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:39:19.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.821+0000 7f77dbe1c640 1 -- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f77c006ae70 msgr2=0x7f77c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.821+0000 7f77dbe1c640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f77c006ae70 0x7f77c006d330 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009a40 tx=0x7f77cc008040 comp rx=0 tx=0).stop 2023-12-08T08:39:19.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.821+0000 7f77dbe1c640 1 -- 172.21.15.16:0/373624904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d4152890 msgr2=0x7f77d4154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.821+0000 7f77dbe1c640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d4152890 0x7f77d4154310 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f77c800e600 tx=0x7f77c800eac0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.821+0000 7f77dbe1c640 1 -- 172.21.15.16:0/373624904 shutdown_connections 2023-12-08T08:39:19.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.821+0000 7f77dbe1c640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f77c006ae70 0x7f77c006d330 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.821+0000 7f77dbe1c640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f77d4154850 0x7f77d40caee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.821+0000 7f77dbe1c640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d4152890 0x7f77d4154310 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.821+0000 7f77dbe1c640 1 --2- 172.21.15.16:0/373624904 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f77d41524b0 0x7f77d4155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.821+0000 7f77dbe1c640 1 -- 172.21.15.16:0/373624904 >> 172.21.15.16:0/373624904 conn(0x7f77d40d3090 msgr2=0x7f77d40b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:19.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.821+0000 7f77dbe1c640 1 -- 172.21.15.16:0/373624904 shutdown_connections 2023-12-08T08:39:19.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.821+0000 7f77dbe1c640 1 -- 172.21.15.16:0/373624904 wait complete. 2023-12-08T08:39:19.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:39:19.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.942+0000 7f065e968640 1 Processor -- start 2023-12-08T08:39:19.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.942+0000 7f065e968640 1 -- start start 2023-12-08T08:39:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.942+0000 7f065e968640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06580cfa30 0x7f06580b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.942+0000 7f065e968640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06580b4150 0x7f06580b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.942+0000 7f065e968640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f06580cfe10 0x7f0658161260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.942+0000 7f065e968640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06580c6be0 con 0x7f06580cfa30 2023-12-08T08:39:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.942+0000 7f065e968640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06580d3ff0 con 0x7f06580cfe10 2023-12-08T08:39:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.942+0000 7f065e968640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06580b4860 con 0x7f06580b4150 2023-12-08T08:39:19.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.943+0000 7f0657fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06580cfa30 0x7f06580b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.943+0000 7f0657fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06580cfa30 0x7f06580b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49356/0 (socket says 172.21.15.16:49356) 2023-12-08T08:39:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.943+0000 7f0657fff640 1 -- 172.21.15.16:0/3924468309 learned_addr learned my addr 172.21.15.16:0/3924468309 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.943+0000 7f065cf05640 1 --2- 172.21.15.16:0/3924468309 >> v2:172.21.15.112:3300/0 conn(0x7f06580cfe10 0x7f0658161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.943+0000 7f06577fe640 1 --2- 172.21.15.16:0/3924468309 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06580b4150 0x7f06580b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.943+0000 7f0657fff640 1 -- 172.21.15.16:0/3924468309 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06580b4150 msgr2=0x7f06580b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.943+0000 7f0657fff640 1 --2- 172.21.15.16:0/3924468309 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06580b4150 0x7f06580b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.943+0000 7f0657fff640 1 -- 172.21.15.16:0/3924468309 >> v2:172.21.15.112:3300/0 conn(0x7f06580cfe10 msgr2=0x7f0658161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.943+0000 7f0657fff640 1 --2- 172.21.15.16:0/3924468309 >> v2:172.21.15.112:3300/0 conn(0x7f06580cfe10 0x7f0658161260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.943+0000 7f0657fff640 1 -- 172.21.15.16:0/3924468309 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06580d3d50 con 0x7f06580cfa30 2023-12-08T08:39:19.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.944+0000 7f0657fff640 1 --2- 172.21.15.16:0/3924468309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06580cfa30 0x7f06580b2f20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f06580d02a0 tx=0x7f064802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fdfbf07d56f04b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.944+0000 7f0656ffd640 1 -- 172.21.15.16:0/3924468309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06480317e0 con 0x7f06580cfa30 2023-12-08T08:39:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.944+0000 7f0656ffd640 1 -- 172.21.15.16:0/3924468309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0648031940 con 0x7f06580cfa30 2023-12-08T08:39:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.944+0000 7f0656ffd640 1 -- 172.21.15.16:0/3924468309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0648031cd0 con 0x7f06580cfa30 2023-12-08T08:39:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.944+0000 7f065e968640 1 -- 172.21.15.16:0/3924468309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06580cfa30 msgr2=0x7f06580b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.944+0000 7f065e968640 1 --2- 172.21.15.16:0/3924468309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06580cfa30 0x7f06580b2f20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f06580d02a0 tx=0x7f064802b880 comp rx=0 tx=0).stop 2023-12-08T08:39:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.944+0000 7f065e968640 1 -- 172.21.15.16:0/3924468309 shutdown_connections 2023-12-08T08:39:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.944+0000 7f065e968640 1 --2- 172.21.15.16:0/3924468309 >> v2:172.21.15.112:3300/0 conn(0x7f06580cfe10 0x7f0658161260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.944+0000 7f065e968640 1 --2- 172.21.15.16:0/3924468309 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06580b4150 0x7f06580b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.944+0000 7f065e968640 1 --2- 172.21.15.16:0/3924468309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06580cfa30 0x7f06580b2f20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.944+0000 7f065e968640 1 -- 172.21.15.16:0/3924468309 >> 172.21.15.16:0/3924468309 conn(0x7f06580d3090 msgr2=0x7f06580d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.944+0000 7f065e968640 1 -- 172.21.15.16:0/3924468309 shutdown_connections 2023-12-08T08:39:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.945+0000 7f065e968640 1 -- 172.21.15.16:0/3924468309 wait complete. 2023-12-08T08:39:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.946+0000 7f065e968640 1 Processor -- start 2023-12-08T08:39:19.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.946+0000 7f065e968640 1 -- start start 2023-12-08T08:39:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065e968640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06580b4150 0x7f0658147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065e968640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06580cfa30 0x7f06580b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065e968640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06580cfe10 0x7f06581f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065e968640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06580b2e20 con 0x7f06580cfe10 2023-12-08T08:39:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065e968640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06580b2210 con 0x7f06580cfa30 2023-12-08T08:39:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065e968640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06580b24f0 con 0x7f06580b4150 2023-12-08T08:39:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065cf05640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06580cfe10 0x7f06581f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065cf05640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06580cfe10 0x7f06581f6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49364/0 (socket says 172.21.15.16:49364) 2023-12-08T08:39:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065cf05640 1 -- 172.21.15.16:0/1572092013 learned_addr learned my addr 172.21.15.16:0/1572092013 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f0657fff640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06580b4150 0x7f0658147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f06577fe640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06580cfa30 0x7f06580b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065cf05640 1 -- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06580b4150 msgr2=0x7f0658147b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065cf05640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06580b4150 0x7f0658147b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065cf05640 1 -- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06580cfa30 msgr2=0x7f06580b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065cf05640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06580cfa30 0x7f06580b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:19.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065cf05640 1 -- 172.21.15.16:0/1572092013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06480094d0 con 0x7f06580cfe10 2023-12-08T08:39:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f06577fe640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06580cfa30 0x7f06580b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f0657fff640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06580b4150 0x7f0658147b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065cf05640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06580cfe10 0x7f06581f6790 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0650007e60 tx=0x7f065000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f0654ff9640 1 -- 172.21.15.16:0/1572092013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650014420 con 0x7f06580cfe10 2023-12-08T08:39:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f0654ff9640 1 -- 172.21.15.16:0/1572092013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06500145a0 con 0x7f06580cfe10 2023-12-08T08:39:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f065e968640 1 -- 172.21.15.16:0/1572092013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06580b27d0 con 0x7f06580cfe10 2023-12-08T08:39:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.947+0000 7f0654ff9640 1 -- 172.21.15.16:0/1572092013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650014950 con 0x7f06580cfe10 2023-12-08T08:39:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.948+0000 7f065e968640 1 -- 172.21.15.16:0/1572092013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06580b1e70 con 0x7f06580cfe10 2023-12-08T08:39:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.948+0000 7f065e968640 1 -- 172.21.15.16:0/1572092013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0658067430 con 0x7f06580cfe10 2023-12-08T08:39:19.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.949+0000 7f0654ff9640 1 -- 172.21.15.16:0/1572092013 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0650014ab0 con 0x7f06580cfe10 2023-12-08T08:39:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.949+0000 7f0654ff9640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f064406ae40 0x7f064406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.949+0000 7f0654ff9640 1 -- 172.21.15.16:0/1572092013 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f065008a340 con 0x7f06580cfe10 2023-12-08T08:39:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.950+0000 7f0657fff640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f064406ae40 0x7f064406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:19.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.950+0000 7f0657fff640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f064406ae40 0x7f064406d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0648036000 tx=0x7f064803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:19.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:19.954+0000 7f0654ff9640 1 -- 172.21.15.16:0/1572092013 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0650055010 con 0x7f06580cfe10 2023-12-08T08:39:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.128+0000 7f065e968640 1 -- 172.21.15.16:0/1572092013 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f06580b2f60 con 0x7f06580cfe10 2023-12-08T08:39:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.128+0000 7f0654ff9640 1 -- 172.21.15.16:0/1572092013 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0650058630 con 0x7f06580cfe10 2023-12-08T08:39:20.130 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.130+0000 7f065e968640 1 -- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f064406ae40 msgr2=0x7f064406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.130+0000 7f065e968640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f064406ae40 0x7f064406d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0648036000 tx=0x7f064803d040 comp rx=0 tx=0).stop 2023-12-08T08:39:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.130+0000 7f065e968640 1 -- 172.21.15.16:0/1572092013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06580cfe10 msgr2=0x7f06581f6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.130+0000 7f065e968640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06580cfe10 0x7f06581f6790 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0650007e60 tx=0x7f065000e4b0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.130+0000 7f065e968640 1 -- 172.21.15.16:0/1572092013 shutdown_connections 2023-12-08T08:39:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.130+0000 7f065e968640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f064406ae40 0x7f064406d300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.130+0000 7f065e968640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06580cfe10 0x7f06581f6790 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.130+0000 7f065e968640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f06580cfa30 0x7f06580b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.130+0000 7f065e968640 1 --2- 172.21.15.16:0/1572092013 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06580b4150 0x7f0658147b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.130+0000 7f065e968640 1 -- 172.21.15.16:0/1572092013 >> 172.21.15.16:0/1572092013 conn(0x7f06580d3090 msgr2=0x7f06580d2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.130+0000 7f065e968640 1 -- 172.21.15.16:0/1572092013 shutdown_connections 2023-12-08T08:39:20.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.130+0000 7f065e968640 1 -- 172.21.15.16:0/1572092013 wait complete. 2023-12-08T08:39:20.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-08T08:39:20.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.248+0000 7fd0a68c9640 1 Processor -- start 2023-12-08T08:39:20.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.248+0000 7fd0a68c9640 1 -- start start 2023-12-08T08:39:20.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.249+0000 7fd0a68c9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd0a015d580 0x7fd0a00d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.249+0000 7fd0a68c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0a015f890 0x7fd0a015d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.249+0000 7fd0a68c9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a015ee90 con 0x7fd0a00d3ad0 2023-12-08T08:39:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.249+0000 7fd0a68c9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a00d0fe0 con 0x7fd0a015d580 2023-12-08T08:39:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.249+0000 7fd0a68c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a00d1280 con 0x7fd0a015f890 2023-12-08T08:39:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.249+0000 7fd09ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd0a015d580 0x7fd0a00d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.249+0000 7fd0a4e66640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0a015f890 0x7fd0a015d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.249+0000 7fd09ffff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd0a015d580 0x7fd0a00d1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:40306/0 (socket says 172.21.15.16:40306) 2023-12-08T08:39:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.249+0000 7fd0a4e66640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0a015f890 0x7fd0a015d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:52648/0 (socket says 172.21.15.16:52648) 2023-12-08T08:39:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.249+0000 7fd09ffff640 1 -- 172.21.15.16:0/108968026 learned_addr learned my addr 172.21.15.16:0/108968026 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.249+0000 7fd09f7fe640 1 --1- 172.21.15.16:0/108968026 >> v1:172.21.15.112:6789/0 conn(0x7fd0a00d3ad0 0x7fd0a015f180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:60114/0 (socket says 172.21.15.16:60114) 2023-12-08T08:39:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.250+0000 7fd09ffff640 1 -- 172.21.15.16:0/108968026 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0a015f890 msgr2=0x7fd0a015d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:20.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.250+0000 7fd09ffff640 1 --2- 172.21.15.16:0/108968026 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0a015f890 0x7fd0a015d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.250+0000 7fd09ffff640 1 -- 172.21.15.16:0/108968026 >> v1:172.21.15.112:6789/0 conn(0x7fd0a00d3ad0 legacy=0x7fd0a015f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.250+0000 7fd09ffff640 1 -- 172.21.15.16:0/108968026 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a00d1560 con 0x7fd0a015d580 2023-12-08T08:39:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.250+0000 7fd0a4e66640 1 --2- 172.21.15.16:0/108968026 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0a015f890 0x7fd0a015d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:39:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.250+0000 7fd09ffff640 1 --2- 172.21.15.16:0/108968026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a015d580 0x7fd0a00d1d60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd0940098a0 tx=0x7fd09402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99e215e9ac937d08 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.251+0000 7fd09effd640 1 -- 172.21.15.16:0/108968026 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094031640 con 0x7fd0a015d580 2023-12-08T08:39:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.251+0000 7fd09effd640 1 -- 172.21.15.16:0/108968026 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0940317a0 con 0x7fd0a015d580 2023-12-08T08:39:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.251+0000 7fd09effd640 1 -- 172.21.15.16:0/108968026 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094031b30 con 0x7fd0a015d580 2023-12-08T08:39:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.251+0000 7fd0a68c9640 1 -- 172.21.15.16:0/108968026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a015d580 msgr2=0x7fd0a00d1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.251+0000 7fd0a68c9640 1 --2- 172.21.15.16:0/108968026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a015d580 0x7fd0a00d1d60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd0940098a0 tx=0x7fd09402b8f0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.251+0000 7fd0a68c9640 1 -- 172.21.15.16:0/108968026 shutdown_connections 2023-12-08T08:39:20.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.251+0000 7fd0a68c9640 1 --2- 172.21.15.16:0/108968026 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0a015f890 0x7fd0a015d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.251+0000 7fd0a68c9640 1 --2- 172.21.15.16:0/108968026 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a015d580 0x7fd0a00d1d60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.251+0000 7fd0a68c9640 1 -- 172.21.15.16:0/108968026 >> 172.21.15.16:0/108968026 conn(0x7fd0a00d2fb0 msgr2=0x7fd0a00d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:20.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.251+0000 7fd0a68c9640 1 -- 172.21.15.16:0/108968026 shutdown_connections 2023-12-08T08:39:20.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.251+0000 7fd0a68c9640 1 -- 172.21.15.16:0/108968026 wait complete. 2023-12-08T08:39:20.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.252+0000 7fd0a68c9640 1 Processor -- start 2023-12-08T08:39:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.253+0000 7fd0a68c9640 1 -- start start 2023-12-08T08:39:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.253+0000 7fd0a68c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0a00d3ad0 0x7fd0a015e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.253+0000 7fd0a68c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a015d580 0x7fd0a00b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.253+0000 7fd0a68c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0a015f890 0x7fd0a00b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.253+0000 7fd0a68c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a00b2b30 con 0x7fd0a00d3ad0 2023-12-08T08:39:20.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.253+0000 7fd0a68c9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a00b7f70 con 0x7fd0a015d580 2023-12-08T08:39:20.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.253+0000 7fd0a68c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a01f3ea0 con 0x7fd0a015f890 2023-12-08T08:39:20.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.253+0000 7fd09ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0a00d3ad0 0x7fd0a015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd09ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0a00d3ad0 0x7fd0a015e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49368/0 (socket says 172.21.15.16:49368) 2023-12-08T08:39:20.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd09ffff640 1 -- 172.21.15.16:0/242113651 learned_addr learned my addr 172.21.15.16:0/242113651 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:20.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd09f7fe640 1 --2- 172.21.15.16:0/242113651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a015d580 0x7fd0a00b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd0a4e66640 1 --2- 172.21.15.16:0/242113651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0a015f890 0x7fd0a00b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd09ffff640 1 -- 172.21.15.16:0/242113651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0a015f890 msgr2=0x7fd0a00b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd09ffff640 1 --2- 172.21.15.16:0/242113651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0a015f890 0x7fd0a00b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd09ffff640 1 -- 172.21.15.16:0/242113651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a015d580 msgr2=0x7fd0a00b73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd09ffff640 1 --2- 172.21.15.16:0/242113651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a015d580 0x7fd0a00b73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd09ffff640 1 -- 172.21.15.16:0/242113651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd098009a70 con 0x7fd0a00d3ad0 2023-12-08T08:39:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd09ffff640 1 --2- 172.21.15.16:0/242113651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0a00d3ad0 0x7fd0a015e170 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd09402b930 tx=0x7fd0940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd09cff9640 1 -- 172.21.15.16:0/242113651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09403a930 con 0x7fd0a00d3ad0 2023-12-08T08:39:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd09cff9640 1 -- 172.21.15.16:0/242113651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09403aa90 con 0x7fd0a00d3ad0 2023-12-08T08:39:20.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd0a68c9640 1 -- 172.21.15.16:0/242113651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0940094d0 con 0x7fd0a00d3ad0 2023-12-08T08:39:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd09cff9640 1 -- 172.21.15.16:0/242113651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09403ae40 con 0x7fd0a00d3ad0 2023-12-08T08:39:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.254+0000 7fd0a68c9640 1 -- 172.21.15.16:0/242113651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a01f33c0 con 0x7fd0a00d3ad0 2023-12-08T08:39:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.255+0000 7fd0a68c9640 1 -- 172.21.15.16:0/242113651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a00d0980 con 0x7fd0a00d3ad0 2023-12-08T08:39:20.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.256+0000 7fd09cff9640 1 -- 172.21.15.16:0/242113651 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd09402f050 con 0x7fd0a00d3ad0 2023-12-08T08:39:20.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.256+0000 7fd09cff9640 1 --2- 172.21.15.16:0/242113651 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd08c06ae40 0x7fd08c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.256+0000 7fd09cff9640 1 -- 172.21.15.16:0/242113651 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fd0940ab800 con 0x7fd0a00d3ad0 2023-12-08T08:39:20.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.256+0000 7fd09f7fe640 1 --2- 172.21.15.16:0/242113651 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd08c06ae40 0x7fd08c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.257+0000 7fd09f7fe640 1 --2- 172.21.15.16:0/242113651 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd08c06ae40 0x7fd08c06d300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd090000f60 tx=0x7fd090007ca0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:20.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.260+0000 7fd09cff9640 1 -- 172.21.15.16:0/242113651 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0940764d0 con 0x7fd0a00d3ad0 2023-12-08T08:39:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.448+0000 7fd0a68c9640 1 -- 172.21.15.16:0/242113651 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd0a0067430 con 0x7fd0a00d3ad0 2023-12-08T08:39:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.449+0000 7fd09cff9640 1 -- 172.21.15.16:0/242113651 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd094079af0 con 0x7fd0a00d3ad0 2023-12-08T08:39:20.451 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-12-08T08:39:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.450+0000 7fd0a68c9640 1 -- 172.21.15.16:0/242113651 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd08c06ae40 msgr2=0x7fd08c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.450+0000 7fd0a68c9640 1 --2- 172.21.15.16:0/242113651 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd08c06ae40 0x7fd08c06d300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd090000f60 tx=0x7fd090007ca0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.450+0000 7fd0a68c9640 1 -- 172.21.15.16:0/242113651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0a00d3ad0 msgr2=0x7fd0a015e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.450+0000 7fd0a68c9640 1 --2- 172.21.15.16:0/242113651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0a00d3ad0 0x7fd0a015e170 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd09402b930 tx=0x7fd0940046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.451+0000 7fd0a68c9640 1 -- 172.21.15.16:0/242113651 shutdown_connections 2023-12-08T08:39:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.451+0000 7fd0a68c9640 1 --2- 172.21.15.16:0/242113651 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd08c06ae40 0x7fd08c06d300 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.451+0000 7fd0a68c9640 1 --2- 172.21.15.16:0/242113651 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd0a015f890 0x7fd0a00b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.451+0000 7fd0a68c9640 1 --2- 172.21.15.16:0/242113651 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd0a015d580 0x7fd0a00b73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.451+0000 7fd0a68c9640 1 --2- 172.21.15.16:0/242113651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0a00d3ad0 0x7fd0a015e170 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.451+0000 7fd0a68c9640 1 -- 172.21.15.16:0/242113651 >> 172.21.15.16:0/242113651 conn(0x7fd0a00d2fb0 msgr2=0x7fd0a006b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.451+0000 7fd0a68c9640 1 -- 172.21.15.16:0/242113651 shutdown_connections 2023-12-08T08:39:20.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.451+0000 7fd0a68c9640 1 -- 172.21.15.16:0/242113651 wait complete. 2023-12-08T08:39:20.463 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:39:20.463 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T08:39:20.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:20.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.623+0000 7f6e7f105640 1 Processor -- start 2023-12-08T08:39:20.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.623+0000 7f6e7f105640 1 -- start start 2023-12-08T08:39:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.624+0000 7f6e7f105640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6e7815b750 0x7f6e78070e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.624+0000 7f6e7f105640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e780720d0 con 0x7f6e78158ee0 2023-12-08T08:39:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.624+0000 7f6e7f105640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e780cc890 con 0x7f6e780cbd10 2023-12-08T08:39:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.624+0000 7f6e7f105640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e780714c0 con 0x7f6e7815b750 2023-12-08T08:39:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.624+0000 7f6e7cea1640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6e78158ee0 0x7f6e780cb600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34296/0 (socket says 172.21.15.16:34296) 2023-12-08T08:39:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.624+0000 7f6e7cea1640 1 -- 172.21.15.16:0/163408986 learned_addr learned my addr 172.21.15.16:0/163408986 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.624+0000 7f6e777fe640 1 -- 172.21.15.16:0/163408986 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093244578 0 0) 0x7f6e780720d0 con 0x7f6e78158ee0 2023-12-08T08:39:20.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.624+0000 7f6e7d6a2640 1 --2- 172.21.15.16:0/163408986 >> v2:172.21.15.112:3300/0 conn(0x7f6e7815b750 0x7f6e78070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.624+0000 7f6e777fe640 1 -- 172.21.15.16:0/163408986 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e64003520 con 0x7f6e78158ee0 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e777fe640 1 -- 172.21.15.16:0/163408986 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 149063944 0 0) 0x7f6e64003520 con 0x7f6e78158ee0 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e777fe640 1 -- 172.21.15.16:0/163408986 >> v2:172.21.15.112:3300/0 conn(0x7f6e7815b750 msgr2=0x7f6e78070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e777fe640 1 --2- 172.21.15.16:0/163408986 >> v2:172.21.15.112:3300/0 conn(0x7f6e7815b750 0x7f6e78070e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e777fe640 1 -- 172.21.15.16:0/163408986 >> v1:172.21.15.112:6789/0 conn(0x7f6e780cbd10 legacy=0x7f6e7806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e777fe640 1 -- 172.21.15.16:0/163408986 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e780717a0 con 0x7f6e78158ee0 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e777fe640 1 -- 172.21.15.16:0/163408986 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f6e700019f0 con 0x7f6e78158ee0 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e777fe640 1 -- 172.21.15.16:0/163408986 >> v1:172.21.15.16:6789/0 conn(0x7f6e78158ee0 legacy=0x7f6e780cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e777fe640 1 --2- 172.21.15.16:0/163408986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e64003970 0x7f6e64003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e777fe640 1 --2- 172.21.15.16:0/163408986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e640047c0 0x7f6e64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e777fe640 1 --2- 172.21.15.16:0/163408986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64005180 0x7f6e6400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e777fe640 1 -- 172.21.15.16:0/163408986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780714c0 con 0x7f6e64003970 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e777fe640 1 -- 172.21.15.16:0/163408986 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6400c120 con 0x7f6e64005180 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e777fe640 1 -- 172.21.15.16:0/163408986 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6400c2e0 con 0x7f6e640047c0 2023-12-08T08:39:20.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e7cea1640 1 --2- 172.21.15.16:0/163408986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e64003970 0x7f6e64003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e7d6a2640 1 --2- 172.21.15.16:0/163408986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64005180 0x7f6e6400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e77fff640 1 --2- 172.21.15.16:0/163408986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e640047c0 0x7f6e64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e7cea1640 1 -- 172.21.15.16:0/163408986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e640047c0 msgr2=0x7f6e64004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:20.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e7cea1640 1 --2- 172.21.15.16:0/163408986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e640047c0 0x7f6e64004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e7cea1640 1 -- 172.21.15.16:0/163408986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64005180 msgr2=0x7f6e6400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:20.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e7cea1640 1 --2- 172.21.15.16:0/163408986 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e64005180 0x7f6e6400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e7cea1640 1 -- 172.21.15.16:0/163408986 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e64003520 con 0x7f6e64003970 2023-12-08T08:39:20.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e7d6a2640 1 -- 172.21.15.16:0/163408986 reap_dead start 2023-12-08T08:39:20.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.625+0000 7f6e77fff640 1 --2- 172.21.15.16:0/163408986 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e640047c0 0x7f6e64004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:20.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.626+0000 7f6e7cea1640 1 --2- 172.21.15.16:0/163408986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e64003970 0x7f6e64003d90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000a830 tx=0x7f6e640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbea1179589dabb0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:20.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.626+0000 7f6e777fe640 1 -- 172.21.15.16:0/163408986 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e700072d0 con 0x7f6e64003970 2023-12-08T08:39:20.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.626+0000 7f6e777fe640 1 -- 172.21.15.16:0/163408986 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7002f070 con 0x7f6e64003970 2023-12-08T08:39:20.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.626+0000 7f6e777fe640 1 -- 172.21.15.16:0/163408986 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70007690 con 0x7f6e64003970 2023-12-08T08:39:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.627+0000 7f6e7f105640 1 -- 172.21.15.16:0/163408986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e64003970 msgr2=0x7f6e64003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.627+0000 7f6e7f105640 1 --2- 172.21.15.16:0/163408986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e64003970 0x7f6e64003d90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000a830 tx=0x7f6e640057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.627+0000 7f6e7f105640 1 -- 172.21.15.16:0/163408986 shutdown_connections 2023-12-08T08:39:20.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.627+0000 7f6e7f105640 1 --2- 172.21.15.16:0/163408986 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e64003970 0x7f6e64003d90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.627+0000 7f6e7f105640 1 -- 172.21.15.16:0/163408986 >> 172.21.15.16:0/163408986 conn(0x7f6e780b09f0 msgr2=0x7f6e780b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.627+0000 7f6e7f105640 1 -- 172.21.15.16:0/163408986 shutdown_connections 2023-12-08T08:39:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.627+0000 7f6e7f105640 1 -- 172.21.15.16:0/163408986 wait complete. 2023-12-08T08:39:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.628+0000 7f6e7f105640 1 Processor -- start 2023-12-08T08:39:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.628+0000 7f6e7f105640 1 -- start start 2023-12-08T08:39:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7f105640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e64003970 0x7f6e781f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7f105640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e780cf3b0 0x7f6e7809a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7f105640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e78099a10 0x7f6e7815a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7f105640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e781f0ed0 con 0x7f6e64003970 2023-12-08T08:39:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7f105640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e781f48d0 con 0x7f6e780cf3b0 2023-12-08T08:39:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7f105640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e781f4bb0 con 0x7f6e78099a10 2023-12-08T08:39:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7cea1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e64003970 0x7f6e781f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7cea1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e64003970 0x7f6e781f42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49388/0 (socket says 172.21.15.16:49388) 2023-12-08T08:39:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7cea1640 1 -- 172.21.15.16:0/2061671181 learned_addr learned my addr 172.21.15.16:0/2061671181 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e77fff640 1 --2- 172.21.15.16:0/2061671181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e780cf3b0 0x7f6e7809a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7cea1640 1 -- 172.21.15.16:0/2061671181 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e78099a10 msgr2=0x7f6e7815a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7d6a2640 1 --2- 172.21.15.16:0/2061671181 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e78099a10 0x7f6e7815a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7cea1640 1 --2- 172.21.15.16:0/2061671181 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e78099a10 0x7f6e7815a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7cea1640 1 -- 172.21.15.16:0/2061671181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e780cf3b0 msgr2=0x7f6e7809a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7cea1640 1 --2- 172.21.15.16:0/2061671181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e780cf3b0 0x7f6e7809a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7cea1640 1 -- 172.21.15.16:0/2061671181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e781f4e90 con 0x7f6e64003970 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.629+0000 7f6e7cea1640 1 --2- 172.21.15.16:0/2061671181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e64003970 0x7f6e781f42b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6e7002cb20 tx=0x7f6e70001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.630+0000 7f6e757fa640 1 -- 172.21.15.16:0/2061671181 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e700070d0 con 0x7f6e64003970 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.630+0000 7f6e757fa640 1 -- 172.21.15.16:0/2061671181 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e70007230 con 0x7f6e64003970 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.630+0000 7f6e7f105640 1 -- 172.21.15.16:0/2061671181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e781f5170 con 0x7f6e64003970 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.630+0000 7f6e757fa640 1 -- 172.21.15.16:0/2061671181 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e700075a0 con 0x7f6e64003970 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.630+0000 7f6e7f105640 1 -- 172.21.15.16:0/2061671181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7809ae80 con 0x7f6e64003970 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.631+0000 7f6e7f105640 1 -- 172.21.15.16:0/2061671181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e78067430 con 0x7f6e64003970 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.631+0000 7f6e757fa640 1 -- 172.21.15.16:0/2061671181 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e70031030 con 0x7f6e64003970 2023-12-08T08:39:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.632+0000 7f6e757fa640 1 --2- 172.21.15.16:0/2061671181 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6e5806ae70 0x7f6e5806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.632+0000 7f6e757fa640 1 -- 172.21.15.16:0/2061671181 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f6e700ab9a0 con 0x7f6e64003970 2023-12-08T08:39:20.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.632+0000 7f6e77fff640 1 --2- 172.21.15.16:0/2061671181 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6e5806ae70 0x7f6e5806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.632+0000 7f6e77fff640 1 --2- 172.21.15.16:0/2061671181 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6e5806ae70 0x7f6e5806d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6e60007fd0 tx=0x7f6e60009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.636+0000 7f6e757fa640 1 -- 172.21.15.16:0/2061671181 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e70076670 con 0x7f6e64003970 2023-12-08T08:39:20.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.814+0000 7f6e7f105640 1 -- 172.21.15.16:0/2061671181 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e78067430 con 0x7f6e64003970 2023-12-08T08:39:20.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.815+0000 7f6e757fa640 1 -- 172.21.15.16:0/2061671181 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7f6e70079c90 con 0x7f6e64003970 2023-12-08T08:39:20.818 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:20.818 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:20.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.817+0000 7f6e7f105640 1 -- 172.21.15.16:0/2061671181 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6e5806ae70 msgr2=0x7f6e5806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.817+0000 7f6e7f105640 1 --2- 172.21.15.16:0/2061671181 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6e5806ae70 0x7f6e5806d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6e60007fd0 tx=0x7f6e60009040 comp rx=0 tx=0).stop 2023-12-08T08:39:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.817+0000 7f6e7f105640 1 -- 172.21.15.16:0/2061671181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e64003970 msgr2=0x7f6e781f42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.817+0000 7f6e7f105640 1 --2- 172.21.15.16:0/2061671181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e64003970 0x7f6e781f42b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6e7002cb20 tx=0x7f6e70001d80 comp rx=0 tx=0).stop 2023-12-08T08:39:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.817+0000 7f6e7f105640 1 -- 172.21.15.16:0/2061671181 shutdown_connections 2023-12-08T08:39:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.817+0000 7f6e7f105640 1 --2- 172.21.15.16:0/2061671181 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6e5806ae70 0x7f6e5806d330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.817+0000 7f6e7f105640 1 --2- 172.21.15.16:0/2061671181 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6e78099a10 0x7f6e7815a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.817+0000 7f6e7f105640 1 --2- 172.21.15.16:0/2061671181 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6e780cf3b0 0x7f6e7809a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.817+0000 7f6e7f105640 1 --2- 172.21.15.16:0/2061671181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e64003970 0x7f6e781f42b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.817+0000 7f6e7f105640 1 -- 172.21.15.16:0/2061671181 >> 172.21.15.16:0/2061671181 conn(0x7f6e780b09f0 msgr2=0x7f6e780aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.817+0000 7f6e7f105640 1 -- 172.21.15.16:0/2061671181 shutdown_connections 2023-12-08T08:39:20.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.817+0000 7f6e7f105640 1 -- 172.21.15.16:0/2061671181 wait complete. 2023-12-08T08:39:20.830 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:39:20.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:20.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.938+0000 7f47da5a9640 1 Processor -- start 2023-12-08T08:39:20.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.938+0000 7f47da5a9640 1 -- start start 2023-12-08T08:39:20.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.939+0000 7f47da5a9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47d40cfa30 0x7f47d40b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.939+0000 7f47da5a9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f47d40cfe10 0x7f47d40cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.939+0000 7f47da5a9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d40c6be0 con 0x7f47d40b4150 2023-12-08T08:39:20.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.939+0000 7f47da5a9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d40d3ff0 con 0x7f47d40cfe10 2023-12-08T08:39:20.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.939+0000 7f47da5a9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40b4860 con 0x7f47d40cfa30 2023-12-08T08:39:20.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.940+0000 7f47d8b46640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f47d40cfe10 0x7f47d40cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.940+0000 7f47d3fff640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47d40cfa30 0x7f47d40b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.940+0000 7f47d8b46640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f47d40cfe10 0x7f47d40cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:40350/0 (socket says 172.21.15.16:40350) 2023-12-08T08:39:20.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.940+0000 7f47d8b46640 1 -- 172.21.15.16:0/3005885332 learned_addr learned my addr 172.21.15.16:0/3005885332 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:20.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.940+0000 7f47d8b46640 1 -- 172.21.15.16:0/3005885332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47d40cfa30 msgr2=0x7f47d40b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:20.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.940+0000 7f47d8b46640 1 --2- 172.21.15.16:0/3005885332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47d40cfa30 0x7f47d40b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.940+0000 7f47d8b46640 1 -- 172.21.15.16:0/3005885332 >> v1:172.21.15.112:6789/0 conn(0x7f47d40b4150 legacy=0x7f47d40b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.940+0000 7f47d8b46640 1 -- 172.21.15.16:0/3005885332 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d40d3d50 con 0x7f47d40cfe10 2023-12-08T08:39:20.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.941+0000 7f47d8b46640 1 --2- 172.21.15.16:0/3005885332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d40cfe10 0x7f47d40cdf50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009e50 tx=0x7f47cc02bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd7dfd103eed67d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:20.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.941+0000 7f47d2ffd640 1 -- 172.21.15.16:0/3005885332 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc02ca70 con 0x7f47d40cfe10 2023-12-08T08:39:20.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.941+0000 7f47d2ffd640 1 -- 172.21.15.16:0/3005885332 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc02cbd0 con 0x7f47d40cfe10 2023-12-08T08:39:20.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.941+0000 7f47d2ffd640 1 -- 172.21.15.16:0/3005885332 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc0343f0 con 0x7f47d40cfe10 2023-12-08T08:39:20.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.942+0000 7f47da5a9640 1 -- 172.21.15.16:0/3005885332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d40cfe10 msgr2=0x7f47d40cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.942+0000 7f47da5a9640 1 --2- 172.21.15.16:0/3005885332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d40cfe10 0x7f47d40cdf50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009e50 tx=0x7f47cc02bed0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.942+0000 7f47da5a9640 1 -- 172.21.15.16:0/3005885332 shutdown_connections 2023-12-08T08:39:20.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.942+0000 7f47da5a9640 1 --2- 172.21.15.16:0/3005885332 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d40cfe10 0x7f47d40cdf50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.942+0000 7f47da5a9640 1 --2- 172.21.15.16:0/3005885332 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47d40cfa30 0x7f47d40b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.942+0000 7f47da5a9640 1 -- 172.21.15.16:0/3005885332 >> 172.21.15.16:0/3005885332 conn(0x7f47d40d3090 msgr2=0x7f47d40d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:20.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.942+0000 7f47da5a9640 1 -- 172.21.15.16:0/3005885332 shutdown_connections 2023-12-08T08:39:20.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.942+0000 7f47da5a9640 1 -- 172.21.15.16:0/3005885332 wait complete. 2023-12-08T08:39:20.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.943+0000 7f47da5a9640 1 Processor -- start 2023-12-08T08:39:20.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.943+0000 7f47da5a9640 1 -- start start 2023-12-08T08:39:20.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47da5a9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d40b4150 0x7f47d4159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47da5a9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d40cfa30 0x7f47d4161580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47da5a9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47d40cfe10 0x7f47d41f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47da5a9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40cced0 con 0x7f47d40cfa30 2023-12-08T08:39:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47da5a9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40cb190 con 0x7f47d40b4150 2023-12-08T08:39:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47da5a9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40cb470 con 0x7f47d40cfe10 2023-12-08T08:39:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d40cfa30 0x7f47d4161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47d37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d40cfa30 0x7f47d4161580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49392/0 (socket says 172.21.15.16:49392) 2023-12-08T08:39:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47d37fe640 1 -- 172.21.15.16:0/3077568677 learned_addr learned my addr 172.21.15.16:0/3077568677 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47d3fff640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d40b4150 0x7f47d4159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47d8b46640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47d40cfe10 0x7f47d41f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47d37fe640 1 -- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47d40cfe10 msgr2=0x7f47d41f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47d37fe640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47d40cfe10 0x7f47d41f39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47d37fe640 1 -- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d40b4150 msgr2=0x7f47d4159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47d37fe640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d40b4150 0x7f47d4159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47d37fe640 1 -- 172.21.15.16:0/3077568677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc009b30 con 0x7f47d40cfa30 2023-12-08T08:39:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47d3fff640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d40b4150 0x7f47d4159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.944+0000 7f47d8b46640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47d40cfe10 0x7f47d41f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.945+0000 7f47d37fe640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d40cfa30 0x7f47d4161580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f47c800c400 tx=0x7f47c800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.945+0000 7f47d0ff9640 1 -- 172.21.15.16:0/3077568677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c80052d0 con 0x7f47d40cfa30 2023-12-08T08:39:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.945+0000 7f47d0ff9640 1 -- 172.21.15.16:0/3077568677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c8005430 con 0x7f47d40cfa30 2023-12-08T08:39:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.945+0000 7f47da5a9640 1 -- 172.21.15.16:0/3077568677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d40b19b0 con 0x7f47d40cfa30 2023-12-08T08:39:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.945+0000 7f47d0ff9640 1 -- 172.21.15.16:0/3077568677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8003830 con 0x7f47d40cfa30 2023-12-08T08:39:20.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.945+0000 7f47da5a9640 1 -- 172.21.15.16:0/3077568677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d4159d60 con 0x7f47d40cfa30 2023-12-08T08:39:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.946+0000 7f47da5a9640 1 -- 172.21.15.16:0/3077568677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d40b1ba0 con 0x7f47d40cfa30 2023-12-08T08:39:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.946+0000 7f47d0ff9640 1 -- 172.21.15.16:0/3077568677 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f47c8003990 con 0x7f47d40cfa30 2023-12-08T08:39:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.947+0000 7f47d0ff9640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f47c006ae40 0x7f47c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.947+0000 7f47d0ff9640 1 -- 172.21.15.16:0/3077568677 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f47c8088ff0 con 0x7f47d40cfa30 2023-12-08T08:39:20.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.947+0000 7f47d3fff640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f47c006ae40 0x7f47c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:20.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.947+0000 7f47d3fff640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f47c006ae40 0x7f47c006d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009a20 tx=0x7f47c4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:20.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:20.951+0000 7f47d0ff9640 1 -- 172.21.15.16:0/3077568677 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f47c8053c40 con 0x7f47d40cfa30 2023-12-08T08:39:21.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.127+0000 7f47da5a9640 1 -- 172.21.15.16:0/3077568677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47d4067430 con 0x7f47d40cfa30 2023-12-08T08:39:21.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.128+0000 7f47d0ff9640 1 -- 172.21.15.16:0/3077568677 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f47c8074e60 con 0x7f47d40cfa30 2023-12-08T08:39:21.130 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:21.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.130+0000 7f47da5a9640 1 -- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f47c006ae40 msgr2=0x7f47c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.130+0000 7f47da5a9640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f47c006ae40 0x7f47c006d300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009a20 tx=0x7f47c4008040 comp rx=0 tx=0).stop 2023-12-08T08:39:21.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.130+0000 7f47da5a9640 1 -- 172.21.15.16:0/3077568677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d40cfa30 msgr2=0x7f47d4161580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.130+0000 7f47da5a9640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d40cfa30 0x7f47d4161580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f47c800c400 tx=0x7f47c800c8c0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.130+0000 7f47da5a9640 1 -- 172.21.15.16:0/3077568677 shutdown_connections 2023-12-08T08:39:21.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.130+0000 7f47da5a9640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f47c006ae40 0x7f47c006d300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.130+0000 7f47da5a9640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f47d40cfe10 0x7f47d41f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.130+0000 7f47da5a9640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d40cfa30 0x7f47d4161580 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.130+0000 7f47da5a9640 1 --2- 172.21.15.16:0/3077568677 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f47d40b4150 0x7f47d4159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.130+0000 7f47da5a9640 1 -- 172.21.15.16:0/3077568677 >> 172.21.15.16:0/3077568677 conn(0x7f47d40d3090 msgr2=0x7f47d415cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:21.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.130+0000 7f47da5a9640 1 -- 172.21.15.16:0/3077568677 shutdown_connections 2023-12-08T08:39:21.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.130+0000 7f47da5a9640 1 -- 172.21.15.16:0/3077568677 wait complete. 2023-12-08T08:39:21.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:39:21.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.250+0000 7f6689ef3640 1 Processor -- start 2023-12-08T08:39:21.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.250+0000 7f6689ef3640 1 -- start start 2023-12-08T08:39:21.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.251+0000 7f6689ef3640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f66840cde00 0x7f66840ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.251+0000 7f6689ef3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66840cce00 con 0x7f66840cc380 2023-12-08T08:39:21.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.251+0000 7f6689ef3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66840b0f00 con 0x7f668415e790 2023-12-08T08:39:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.251+0000 7f6689ef3640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66840ccb20 con 0x7f66840cde00 2023-12-08T08:39:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.251+0000 7f6683fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f66840cc380 0x7f66840b1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34312/0 (socket says 172.21.15.16:34312) 2023-12-08T08:39:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.251+0000 7f6683fff640 1 -- 172.21.15.16:0/1280428790 learned_addr learned my addr 172.21.15.16:0/1280428790 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.251+0000 7f6682ffd640 1 --2- 172.21.15.16:0/1280428790 >> v2:172.21.15.112:3300/0 conn(0x7f66840cde00 0x7f66840ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.251+0000 7f66827fc640 1 -- 172.21.15.16:0/1280428790 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856373600 0 0) 0x7f66840cce00 con 0x7f66840cc380 2023-12-08T08:39:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.251+0000 7f66827fc640 1 -- 172.21.15.16:0/1280428790 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f666c003520 con 0x7f66840cc380 2023-12-08T08:39:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.251+0000 7f6682ffd640 1 -- 172.21.15.16:0/1280428790 >> v1:172.21.15.112:6789/0 conn(0x7f668415e790 legacy=0x7f66840cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.251+0000 7f6682ffd640 1 -- 172.21.15.16:0/1280428790 >> v1:172.21.15.16:6789/0 conn(0x7f66840cc380 legacy=0x7f66840b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.251+0000 7f66827fc640 1 -- 172.21.15.16:0/1280428790 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2320840809 0 0) 0x7f666c003520 con 0x7f66840cc380 2023-12-08T08:39:21.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.251+0000 7f6682ffd640 1 -- 172.21.15.16:0/1280428790 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66840b08b0 con 0x7f66840cde00 2023-12-08T08:39:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.252+0000 7f6682ffd640 1 --2- 172.21.15.16:0/1280428790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66840cde00 0x7f66840ce300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f66780097a0 tx=0x7f667802b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a05ef880f4b3840 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.252+0000 7f66827fc640 1 -- 172.21.15.16:0/1280428790 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0019f0 con 0x7f66840cde00 2023-12-08T08:39:21.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.252+0000 7f66827fc640 1 -- 172.21.15.16:0/1280428790 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667802e050 con 0x7f66840cde00 2023-12-08T08:39:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.252+0000 7f66827fc640 1 -- 172.21.15.16:0/1280428790 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678030730 con 0x7f66840cde00 2023-12-08T08:39:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.253+0000 7f6689ef3640 1 -- 172.21.15.16:0/1280428790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66840cde00 msgr2=0x7f66840ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.253+0000 7f6689ef3640 1 --2- 172.21.15.16:0/1280428790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66840cde00 0x7f66840ce300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f66780097a0 tx=0x7f667802b790 comp rx=0 tx=0).stop 2023-12-08T08:39:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.253+0000 7f6689ef3640 1 -- 172.21.15.16:0/1280428790 shutdown_connections 2023-12-08T08:39:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.253+0000 7f6689ef3640 1 --2- 172.21.15.16:0/1280428790 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66840cde00 0x7f66840ce300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.253+0000 7f6689ef3640 1 -- 172.21.15.16:0/1280428790 >> 172.21.15.16:0/1280428790 conn(0x7f66840d0e00 msgr2=0x7f66840d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.253+0000 7f6689ef3640 1 -- 172.21.15.16:0/1280428790 shutdown_connections 2023-12-08T08:39:21.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.253+0000 7f6689ef3640 1 -- 172.21.15.16:0/1280428790 wait complete. 2023-12-08T08:39:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.254+0000 7f6689ef3640 1 Processor -- start 2023-12-08T08:39:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.254+0000 7f6689ef3640 1 -- start start 2023-12-08T08:39:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f6689ef3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66840cc380 0x7f668415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f6689ef3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66840cde00 0x7f668406b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f6689ef3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f668415e790 0x7f66840b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f6689ef3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66840b41a0 con 0x7f668415e790 2023-12-08T08:39:21.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f6689ef3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668406c510 con 0x7f66840cde00 2023-12-08T08:39:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f6689ef3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668406bf50 con 0x7f66840cc380 2023-12-08T08:39:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f6683fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f668415e790 0x7f66840b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f6683fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f668415e790 0x7f66840b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49400/0 (socket says 172.21.15.16:49400) 2023-12-08T08:39:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f6683fff640 1 -- 172.21.15.16:0/3634756187 learned_addr learned my addr 172.21.15.16:0/3634756187 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f6682ffd640 1 --2- 172.21.15.16:0/3634756187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66840cde00 0x7f668406b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f66837fe640 1 --2- 172.21.15.16:0/3634756187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66840cc380 0x7f668415aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f6683fff640 1 -- 172.21.15.16:0/3634756187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66840cc380 msgr2=0x7f668415aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f6683fff640 1 --2- 172.21.15.16:0/3634756187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66840cc380 0x7f668415aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f6683fff640 1 -- 172.21.15.16:0/3634756187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66840cde00 msgr2=0x7f668406b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.255+0000 7f6683fff640 1 --2- 172.21.15.16:0/3634756187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66840cde00 0x7f668406b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.256+0000 7f6683fff640 1 -- 172.21.15.16:0/3634756187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6678009400 con 0x7f668415e790 2023-12-08T08:39:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.256+0000 7f6683fff640 1 --2- 172.21.15.16:0/3634756187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f668415e790 0x7f66840b4630 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f667c00cc90 tx=0x7f667c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.256+0000 7f666bfff640 1 -- 172.21.15.16:0/3634756187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c00cea0 con 0x7f668415e790 2023-12-08T08:39:21.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.256+0000 7f666bfff640 1 -- 172.21.15.16:0/3634756187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f667c012400 con 0x7f668415e790 2023-12-08T08:39:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.256+0000 7f6689ef3640 1 -- 172.21.15.16:0/3634756187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668406c230 con 0x7f668415e790 2023-12-08T08:39:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.256+0000 7f666bfff640 1 -- 172.21.15.16:0/3634756187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c0127b0 con 0x7f668415e790 2023-12-08T08:39:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.256+0000 7f6689ef3640 1 -- 172.21.15.16:0/3634756187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668415b430 con 0x7f668415e790 2023-12-08T08:39:21.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.257+0000 7f6689ef3640 1 -- 172.21.15.16:0/3634756187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66840b4310 con 0x7f668415e790 2023-12-08T08:39:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.257+0000 7f666bfff640 1 -- 172.21.15.16:0/3634756187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f667c007070 con 0x7f668415e790 2023-12-08T08:39:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.258+0000 7f666bfff640 1 --2- 172.21.15.16:0/3634756187 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f667006ae40 0x7f667006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.258+0000 7f666bfff640 1 -- 172.21.15.16:0/3634756187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f667c089ae0 con 0x7f668415e790 2023-12-08T08:39:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.258+0000 7f66837fe640 1 --2- 172.21.15.16:0/3634756187 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f667006ae40 0x7f667006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.259+0000 7f66837fe640 1 --2- 172.21.15.16:0/3634756187 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f667006ae40 0x7f667006d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6674000f60 tx=0x7f6674009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:21.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.262+0000 7f666bfff640 1 -- 172.21.15.16:0/3634756187 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f667c0548f0 con 0x7f668415e790 2023-12-08T08:39:21.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.438+0000 7f6689ef3640 1 -- 172.21.15.16:0/3634756187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6684067430 con 0x7f668415e790 2023-12-08T08:39:21.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.438+0000 7f666bfff640 1 -- 172.21.15.16:0/3634756187 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f667c017020 con 0x7f668415e790 2023-12-08T08:39:21.440 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.440+0000 7f6689ef3640 1 -- 172.21.15.16:0/3634756187 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f667006ae40 msgr2=0x7f667006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.440+0000 7f6689ef3640 1 --2- 172.21.15.16:0/3634756187 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f667006ae40 0x7f667006d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6674000f60 tx=0x7f6674009040 comp rx=0 tx=0).stop 2023-12-08T08:39:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.441+0000 7f6689ef3640 1 -- 172.21.15.16:0/3634756187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f668415e790 msgr2=0x7f66840b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.441+0000 7f6689ef3640 1 --2- 172.21.15.16:0/3634756187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f668415e790 0x7f66840b4630 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f667c00cc90 tx=0x7f667c00a430 comp rx=0 tx=0).stop 2023-12-08T08:39:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.441+0000 7f6689ef3640 1 -- 172.21.15.16:0/3634756187 shutdown_connections 2023-12-08T08:39:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.441+0000 7f6689ef3640 1 --2- 172.21.15.16:0/3634756187 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f667006ae40 0x7f667006d300 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.441+0000 7f6689ef3640 1 --2- 172.21.15.16:0/3634756187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f668415e790 0x7f66840b4630 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.441+0000 7f6689ef3640 1 --2- 172.21.15.16:0/3634756187 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f66840cde00 0x7f668406b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.441+0000 7f6689ef3640 1 --2- 172.21.15.16:0/3634756187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f66840cc380 0x7f668415aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.441+0000 7f6689ef3640 1 -- 172.21.15.16:0/3634756187 >> 172.21.15.16:0/3634756187 conn(0x7f66840d0e00 msgr2=0x7f668415d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.441+0000 7f6689ef3640 1 -- 172.21.15.16:0/3634756187 shutdown_connections 2023-12-08T08:39:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.441+0000 7f6689ef3640 1 -- 172.21.15.16:0/3634756187 wait complete. 2023-12-08T08:39:21.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:39:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.561+0000 7f3111597640 1 Processor -- start 2023-12-08T08:39:21.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.561+0000 7f3111597640 1 -- start start 2023-12-08T08:39:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.562+0000 7f3111597640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f310c15e6a0 0x7f310c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.562+0000 7f3111597640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f310c0b3c80 0x7f310c0b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.562+0000 7f3111597640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f310c0ce790 con 0x7f310c0d1950 2023-12-08T08:39:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.562+0000 7f3111597640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310c0ceba0 con 0x7f310c0b3c80 2023-12-08T08:39:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.562+0000 7f3111597640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f310c0cee80 con 0x7f310c15e6a0 2023-12-08T08:39:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.562+0000 7f310b7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f310c0b3c80 0x7f310c0b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.562+0000 7f310a7fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f310c0d1950 0x7f310c0cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34328/0 (socket says 172.21.15.16:34328) 2023-12-08T08:39:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.562+0000 7f310a7fc640 1 -- 172.21.15.16:0/2789584667 learned_addr learned my addr 172.21.15.16:0/2789584667 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.562+0000 7f310affd640 1 --2- 172.21.15.16:0/2789584667 >> v2:172.21.15.112:3300/0 conn(0x7f310c15e6a0 0x7f310c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.562+0000 7f310b7fe640 1 -- 172.21.15.16:0/2789584667 >> v2:172.21.15.112:3300/0 conn(0x7f310c15e6a0 msgr2=0x7f310c0ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.562+0000 7f310b7fe640 1 --2- 172.21.15.16:0/2789584667 >> v2:172.21.15.112:3300/0 conn(0x7f310c15e6a0 0x7f310c0ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.562+0000 7f310b7fe640 1 -- 172.21.15.16:0/2789584667 >> v1:172.21.15.16:6789/0 conn(0x7f310c0d1950 legacy=0x7f310c0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.562+0000 7f3109ffb640 1 -- 172.21.15.16:0/2789584667 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555480226 0 0) 0x7f310c0ce790 con 0x7f310c0d1950 2023-12-08T08:39:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.562+0000 7f310b7fe640 1 -- 172.21.15.16:0/2789584667 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c0cbcb0 con 0x7f310c0b3c80 2023-12-08T08:39:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.563+0000 7f310b7fe640 1 --2- 172.21.15.16:0/2789584667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c0b3c80 0x7f310c0b4110 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f310c071230 tx=0x7f310402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4caf1312c15f501a server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.563+0000 7f3109ffb640 1 -- 172.21.15.16:0/2789584667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310402cc80 con 0x7f310c0b3c80 2023-12-08T08:39:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.563+0000 7f3109ffb640 1 -- 172.21.15.16:0/2789584667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310402cde0 con 0x7f310c0b3c80 2023-12-08T08:39:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.563+0000 7f3109ffb640 1 -- 172.21.15.16:0/2789584667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104034660 con 0x7f310c0b3c80 2023-12-08T08:39:21.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.564+0000 7f3111597640 1 -- 172.21.15.16:0/2789584667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c0b3c80 msgr2=0x7f310c0b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.564+0000 7f3111597640 1 --2- 172.21.15.16:0/2789584667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c0b3c80 0x7f310c0b4110 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f310c071230 tx=0x7f310402bed0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.564+0000 7f3111597640 1 -- 172.21.15.16:0/2789584667 shutdown_connections 2023-12-08T08:39:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.564+0000 7f3111597640 1 --2- 172.21.15.16:0/2789584667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c0b3c80 0x7f310c0b4110 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.564+0000 7f3111597640 1 --2- 172.21.15.16:0/2789584667 >> v2:172.21.15.112:3300/0 conn(0x7f310c15e6a0 0x7f310c0ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.564+0000 7f3111597640 1 -- 172.21.15.16:0/2789584667 >> 172.21.15.16:0/2789584667 conn(0x7f310c0d0e30 msgr2=0x7f310c0d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.564+0000 7f3111597640 1 -- 172.21.15.16:0/2789584667 shutdown_connections 2023-12-08T08:39:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.564+0000 7f3111597640 1 -- 172.21.15.16:0/2789584667 wait complete. 2023-12-08T08:39:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.565+0000 7f3111597640 1 Processor -- start 2023-12-08T08:39:21.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.565+0000 7f3111597640 1 -- start start 2023-12-08T08:39:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f3111597640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c0b3c80 0x7f310c0c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f3111597640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c0d1950 0x7f310c0c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f3111597640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f310c15e6a0 0x7f310c0ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f3111597640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c0cadf0 con 0x7f310c0d1950 2023-12-08T08:39:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f3111597640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310c14ee10 con 0x7f310c0b3c80 2023-12-08T08:39:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f3111597640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310c14f2c0 con 0x7f310c15e6a0 2023-12-08T08:39:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f310a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c0d1950 0x7f310c0c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f310a7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c0d1950 0x7f310c0c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49418/0 (socket says 172.21.15.16:49418) 2023-12-08T08:39:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f310a7fc640 1 -- 172.21.15.16:0/3313151211 learned_addr learned my addr 172.21.15.16:0/3313151211 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f310affd640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c0b3c80 0x7f310c0c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f310b7fe640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f310c15e6a0 0x7f310c0ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f310a7fc640 1 -- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f310c15e6a0 msgr2=0x7f310c0ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f310a7fc640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f310c15e6a0 0x7f310c0ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f310a7fc640 1 -- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c0b3c80 msgr2=0x7f310c0c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f310a7fc640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c0b3c80 0x7f310c0c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f310a7fc640 1 -- 172.21.15.16:0/3313151211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3104009a70 con 0x7f310c0d1950 2023-12-08T08:39:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f310b7fe640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f310c15e6a0 0x7f310c0ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.566+0000 7f310a7fc640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c0d1950 0x7f310c0c9fa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f310000c460 tx=0x7f310000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.567+0000 7f30f37fe640 1 -- 172.21.15.16:0/3313151211 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31000074e0 con 0x7f310c0d1950 2023-12-08T08:39:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.567+0000 7f30f37fe640 1 -- 172.21.15.16:0/3313151211 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3100007660 con 0x7f310c0d1950 2023-12-08T08:39:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.567+0000 7f3111597640 1 -- 172.21.15.16:0/3313151211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310c0cab10 con 0x7f310c0d1950 2023-12-08T08:39:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.567+0000 7f30f37fe640 1 -- 172.21.15.16:0/3313151211 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100004a20 con 0x7f310c0d1950 2023-12-08T08:39:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.567+0000 7f3111597640 1 -- 172.21.15.16:0/3313151211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310c0c4da0 con 0x7f310c0d1950 2023-12-08T08:39:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.568+0000 7f3111597640 1 -- 172.21.15.16:0/3313151211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310c0c4920 con 0x7f310c0d1950 2023-12-08T08:39:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.568+0000 7f30f37fe640 1 -- 172.21.15.16:0/3313151211 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3100004b80 con 0x7f310c0d1950 2023-12-08T08:39:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.568+0000 7f30f37fe640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f30f806ae40 0x7f30f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.568+0000 7f30f37fe640 1 -- 172.21.15.16:0/3313151211 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f3100088df0 con 0x7f310c0d1950 2023-12-08T08:39:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.569+0000 7f310affd640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f30f806ae40 0x7f30f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.569+0000 7f310affd640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f30f806ae40 0x7f30f806d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f30fc006fd0 tx=0x7f30fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.573+0000 7f30f37fe640 1 -- 172.21.15.16:0/3313151211 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f310c0c4920 con 0x7f310c0d1950 2023-12-08T08:39:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.747+0000 7f3111597640 1 -- 172.21.15.16:0/3313151211 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f310c0c4920 con 0x7f310c0d1950 2023-12-08T08:39:21.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.748+0000 7f30f37fe640 1 -- 172.21.15.16:0/3313151211 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f310c0c4920 con 0x7f310c0d1950 2023-12-08T08:39:21.750 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:39:21.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.750+0000 7f3111597640 1 -- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f30f806ae40 msgr2=0x7f30f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.750+0000 7f3111597640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f30f806ae40 0x7f30f806d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f30fc006fd0 tx=0x7f30fc008040 comp rx=0 tx=0).stop 2023-12-08T08:39:21.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.750+0000 7f3111597640 1 -- 172.21.15.16:0/3313151211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c0d1950 msgr2=0x7f310c0c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.750+0000 7f3111597640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c0d1950 0x7f310c0c9fa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f310000c460 tx=0x7f310000c920 comp rx=0 tx=0).stop 2023-12-08T08:39:21.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.750+0000 7f3111597640 1 -- 172.21.15.16:0/3313151211 shutdown_connections 2023-12-08T08:39:21.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.750+0000 7f3111597640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f30f806ae40 0x7f30f806d300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.750+0000 7f3111597640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f310c15e6a0 0x7f310c0ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.750+0000 7f3111597640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f310c0d1950 0x7f310c0c9fa0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.750+0000 7f3111597640 1 --2- 172.21.15.16:0/3313151211 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f310c0b3c80 0x7f310c0c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.750+0000 7f3111597640 1 -- 172.21.15.16:0/3313151211 >> 172.21.15.16:0/3313151211 conn(0x7f310c0d0e30 msgr2=0x7f310c0d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:21.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.750+0000 7f3111597640 1 -- 172.21.15.16:0/3313151211 shutdown_connections 2023-12-08T08:39:21.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.750+0000 7f3111597640 1 -- 172.21.15.16:0/3313151211 wait complete. 2023-12-08T08:39:21.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:39:21.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.870+0000 7f08a30b4640 1 Processor -- start 2023-12-08T08:39:21.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.871+0000 7f08a30b4640 1 -- start start 2023-12-08T08:39:21.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.871+0000 7f08a30b4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f089c0cfa30 0x7f089c0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.871+0000 7f08a30b4640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f089c0b4150 0x7f089c0b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.871+0000 7f08a30b4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f089c0cfe10 0x7f089c161260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.871+0000 7f08a30b4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089c0c6be0 con 0x7f089c0cfa30 2023-12-08T08:39:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.871+0000 7f08a30b4640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089c0d3ff0 con 0x7f089c0b4150 2023-12-08T08:39:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.871+0000 7f08a30b4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0b4860 con 0x7f089c0cfe10 2023-12-08T08:39:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.871+0000 7f08a0e50640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f089c0cfa30 0x7f089c0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.871+0000 7f08a0e50640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f089c0cfa30 0x7f089c0b2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49424/0 (socket says 172.21.15.16:49424) 2023-12-08T08:39:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.872+0000 7f08a0e50640 1 -- 172.21.15.16:0/706190995 learned_addr learned my addr 172.21.15.16:0/706190995 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.872+0000 7f08a1651640 1 --2- 172.21.15.16:0/706190995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f089c0cfe10 0x7f089c161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.872+0000 7f08a0e50640 1 -- 172.21.15.16:0/706190995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f089c0cfe10 msgr2=0x7f089c161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.872+0000 7f08a0e50640 1 --2- 172.21.15.16:0/706190995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f089c0cfe10 0x7f089c161260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.872+0000 7f089bfff640 1 --2- 172.21.15.16:0/706190995 >> v2:172.21.15.112:3300/0 conn(0x7f089c0b4150 0x7f089c0b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.872+0000 7f08a0e50640 1 -- 172.21.15.16:0/706190995 >> v2:172.21.15.112:3300/0 conn(0x7f089c0b4150 msgr2=0x7f089c0b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.872+0000 7f08a0e50640 1 --2- 172.21.15.16:0/706190995 >> v2:172.21.15.112:3300/0 conn(0x7f089c0b4150 0x7f089c0b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.872+0000 7f08a0e50640 1 -- 172.21.15.16:0/706190995 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c0d3d50 con 0x7f089c0cfa30 2023-12-08T08:39:21.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.873+0000 7f08a0e50640 1 --2- 172.21.15.16:0/706190995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f089c0cfa30 0x7f089c0b2f20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f089c162300 tx=0x7f089402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0461cd3552dc8f server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.873+0000 7f089b7fe640 1 -- 172.21.15.16:0/706190995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089402c950 con 0x7f089c0cfa30 2023-12-08T08:39:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.873+0000 7f089b7fe640 1 -- 172.21.15.16:0/706190995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089402cab0 con 0x7f089c0cfa30 2023-12-08T08:39:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.873+0000 7f089b7fe640 1 -- 172.21.15.16:0/706190995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089402ce40 con 0x7f089c0cfa30 2023-12-08T08:39:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.873+0000 7f08a30b4640 1 -- 172.21.15.16:0/706190995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f089c0cfa30 msgr2=0x7f089c0b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.873+0000 7f08a30b4640 1 --2- 172.21.15.16:0/706190995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f089c0cfa30 0x7f089c0b2f20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f089c162300 tx=0x7f089402bdb0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.873+0000 7f08a30b4640 1 -- 172.21.15.16:0/706190995 shutdown_connections 2023-12-08T08:39:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.873+0000 7f08a30b4640 1 --2- 172.21.15.16:0/706190995 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f089c0cfe10 0x7f089c161260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.873+0000 7f08a30b4640 1 --2- 172.21.15.16:0/706190995 >> v2:172.21.15.112:3300/0 conn(0x7f089c0b4150 0x7f089c0b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.873+0000 7f08a30b4640 1 --2- 172.21.15.16:0/706190995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f089c0cfa30 0x7f089c0b2f20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.873+0000 7f08a30b4640 1 -- 172.21.15.16:0/706190995 >> 172.21.15.16:0/706190995 conn(0x7f089c0d3090 msgr2=0x7f089c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.873+0000 7f08a30b4640 1 -- 172.21.15.16:0/706190995 shutdown_connections 2023-12-08T08:39:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.873+0000 7f08a30b4640 1 -- 172.21.15.16:0/706190995 wait complete. 2023-12-08T08:39:21.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.874+0000 7f08a30b4640 1 Processor -- start 2023-12-08T08:39:21.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.875+0000 7f08a30b4640 1 -- start start 2023-12-08T08:39:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.875+0000 7f08a30b4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f089c0b4150 0x7f089c0cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.875+0000 7f08a30b4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f089c0cfa30 0x7f089c156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.875+0000 7f08a30b4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f089c0cfe10 0x7f089c15b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.875+0000 7f08a30b4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0cd290 con 0x7f089c0cfe10 2023-12-08T08:39:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.875+0000 7f08a30b4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0cc1f0 con 0x7f089c0cfa30 2023-12-08T08:39:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.875+0000 7f08a30b4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0cc4d0 con 0x7f089c0b4150 2023-12-08T08:39:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08a1651640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f089c0cfe10 0x7f089c15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08a1651640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f089c0cfe10 0x7f089c15b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49438/0 (socket says 172.21.15.16:49438) 2023-12-08T08:39:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08a1651640 1 -- 172.21.15.16:0/3714570303 learned_addr learned my addr 172.21.15.16:0/3714570303 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:21.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f089bfff640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f089c0cfa30 0x7f089c156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08a0e50640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f089c0b4150 0x7f089c0cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08a1651640 1 -- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f089c0b4150 msgr2=0x7f089c0cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08a1651640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f089c0b4150 0x7f089c0cb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08a1651640 1 -- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f089c0cfa30 msgr2=0x7f089c156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08a1651640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f089c0cfa30 0x7f089c156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08a1651640 1 -- 172.21.15.16:0/3714570303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0894009b30 con 0x7f089c0cfe10 2023-12-08T08:39:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08a0e50640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f089c0b4150 0x7f089c0cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f089bfff640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f089c0cfa30 0x7f089c156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08a1651640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f089c0cfe10 0x7f089c15b810 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f088400c550 tx=0x7f088400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08997fa640 1 -- 172.21.15.16:0/3714570303 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884004220 con 0x7f089c0cfe10 2023-12-08T08:39:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08997fa640 1 -- 172.21.15.16:0/3714570303 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0884004380 con 0x7f089c0cfe10 2023-12-08T08:39:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08a30b4640 1 -- 172.21.15.16:0/3714570303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f089c0cc7b0 con 0x7f089c0cfe10 2023-12-08T08:39:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08997fa640 1 -- 172.21.15.16:0/3714570303 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884003830 con 0x7f089c0cfe10 2023-12-08T08:39:21.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.876+0000 7f08a30b4640 1 -- 172.21.15.16:0/3714570303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089c0ccd70 con 0x7f089c0cfe10 2023-12-08T08:39:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.877+0000 7f08a30b4640 1 -- 172.21.15.16:0/3714570303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f089c0ccf90 con 0x7f089c0cfe10 2023-12-08T08:39:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.878+0000 7f08997fa640 1 -- 172.21.15.16:0/3714570303 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0884003990 con 0x7f089c0cfe10 2023-12-08T08:39:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.878+0000 7f08997fa640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f087c06ae40 0x7f087c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:21.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.878+0000 7f08997fa640 1 -- 172.21.15.16:0/3714570303 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f0884089700 con 0x7f089c0cfe10 2023-12-08T08:39:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.878+0000 7f08a0e50640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f087c06ae40 0x7f087c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.879+0000 7f08a0e50640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f087c06ae40 0x7f087c06d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0894035000 tx=0x7f0894030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:21.883+0000 7f08997fa640 1 -- 172.21.15.16:0/3714570303 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08840543c0 con 0x7f089c0cfe10 2023-12-08T08:39:22.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.059+0000 7f08a30b4640 1 -- 172.21.15.16:0/3714570303 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f089c067430 con 0x7f089c0cfe10 2023-12-08T08:39:22.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.060+0000 7f08997fa640 1 -- 172.21.15.16:0/3714570303 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f08840579e0 con 0x7f089c0cfe10 2023-12-08T08:39:22.062 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:22.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.061+0000 7f08a30b4640 1 -- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f087c06ae40 msgr2=0x7f087c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.061+0000 7f08a30b4640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f087c06ae40 0x7f087c06d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0894035000 tx=0x7f0894030000 comp rx=0 tx=0).stop 2023-12-08T08:39:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.061+0000 7f08a30b4640 1 -- 172.21.15.16:0/3714570303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f089c0cfe10 msgr2=0x7f089c15b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.061+0000 7f08a30b4640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f089c0cfe10 0x7f089c15b810 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f088400c550 tx=0x7f088400ca10 comp rx=0 tx=0).stop 2023-12-08T08:39:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.061+0000 7f08a30b4640 1 -- 172.21.15.16:0/3714570303 shutdown_connections 2023-12-08T08:39:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.061+0000 7f08a30b4640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f087c06ae40 0x7f087c06d300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.061+0000 7f08a30b4640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f089c0cfe10 0x7f089c15b810 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.061+0000 7f08a30b4640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f089c0cfa30 0x7f089c156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.061+0000 7f08a30b4640 1 --2- 172.21.15.16:0/3714570303 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f089c0b4150 0x7f089c0cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.061+0000 7f08a30b4640 1 -- 172.21.15.16:0/3714570303 >> 172.21.15.16:0/3714570303 conn(0x7f089c0d3090 msgr2=0x7f089c15d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.062+0000 7f08a30b4640 1 -- 172.21.15.16:0/3714570303 shutdown_connections 2023-12-08T08:39:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.062+0000 7f08a30b4640 1 -- 172.21.15.16:0/3714570303 wait complete. 2023-12-08T08:39:22.075 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-08T08:39:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.180+0000 7f258ac7c640 1 Processor -- start 2023-12-08T08:39:22.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.180+0000 7f258ac7c640 1 -- start start 2023-12-08T08:39:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.181+0000 7f258ac7c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f25840b0cf0 0x7f25840d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.181+0000 7f258ac7c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f25840b1d70 0x7f25840cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.181+0000 7f258ac7c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25840d20b0 con 0x7f25841608a0 2023-12-08T08:39:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.181+0000 7f258ac7c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25840d0620 con 0x7f25840b0cf0 2023-12-08T08:39:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.181+0000 7f258ac7c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25840d1b80 con 0x7f25840b1d70 2023-12-08T08:39:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.181+0000 7f2588a18640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f25841608a0 0x7f25840d0f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34342/0 (socket says 172.21.15.16:34342) 2023-12-08T08:39:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.181+0000 7f2588a18640 1 -- 172.21.15.16:0/3088214195 learned_addr learned my addr 172.21.15.16:0/3088214195 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.181+0000 7f2583fff640 1 --2- 172.21.15.16:0/3088214195 >> v2:172.21.15.16:3300/0 conn(0x7f25840b0cf0 0x7f25840d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.181+0000 7f25837fe640 1 -- 172.21.15.16:0/3088214195 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989333210 0 0) 0x7f25840d20b0 con 0x7f25841608a0 2023-12-08T08:39:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.181+0000 7f25837fe640 1 -- 172.21.15.16:0/3088214195 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2574003520 con 0x7f25841608a0 2023-12-08T08:39:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.181+0000 7f2589219640 1 --2- 172.21.15.16:0/3088214195 >> v2:172.21.15.112:3300/0 conn(0x7f25840b1d70 0x7f25840cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.182+0000 7f2583fff640 1 -- 172.21.15.16:0/3088214195 >> v2:172.21.15.112:3300/0 conn(0x7f25840b1d70 msgr2=0x7f25840cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.182+0000 7f2583fff640 1 --2- 172.21.15.16:0/3088214195 >> v2:172.21.15.112:3300/0 conn(0x7f25840b1d70 0x7f25840cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.182+0000 7f2583fff640 1 -- 172.21.15.16:0/3088214195 >> v1:172.21.15.16:6789/0 conn(0x7f25841608a0 legacy=0x7f25840d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.182+0000 7f2583fff640 1 -- 172.21.15.16:0/3088214195 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25840d1dd0 con 0x7f25840b0cf0 2023-12-08T08:39:22.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.182+0000 7f2583fff640 1 --2- 172.21.15.16:0/3088214195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25840b0cf0 0x7f25840d1640 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2584073100 tx=0x7f257002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a60cd48e912f4e server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:22.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.182+0000 7f25837fe640 1 -- 172.21.15.16:0/3088214195 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257002e070 con 0x7f25840b0cf0 2023-12-08T08:39:22.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.182+0000 7f25837fe640 1 -- 172.21.15.16:0/3088214195 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25700315a0 con 0x7f25840b0cf0 2023-12-08T08:39:22.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.182+0000 7f25837fe640 1 -- 172.21.15.16:0/3088214195 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570031910 con 0x7f25840b0cf0 2023-12-08T08:39:22.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.183+0000 7f258ac7c640 1 -- 172.21.15.16:0/3088214195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25840b0cf0 msgr2=0x7f25840d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.183+0000 7f258ac7c640 1 --2- 172.21.15.16:0/3088214195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25840b0cf0 0x7f25840d1640 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2584073100 tx=0x7f257002b840 comp rx=0 tx=0).stop 2023-12-08T08:39:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.183+0000 7f258ac7c640 1 -- 172.21.15.16:0/3088214195 shutdown_connections 2023-12-08T08:39:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.183+0000 7f258ac7c640 1 --2- 172.21.15.16:0/3088214195 >> v2:172.21.15.112:3300/0 conn(0x7f25840b1d70 0x7f25840cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.183+0000 7f258ac7c640 1 --2- 172.21.15.16:0/3088214195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25840b0cf0 0x7f25840d1640 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.183+0000 7f258ac7c640 1 -- 172.21.15.16:0/3088214195 >> 172.21.15.16:0/3088214195 conn(0x7f25840d3030 msgr2=0x7f25840d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.183+0000 7f258ac7c640 1 -- 172.21.15.16:0/3088214195 shutdown_connections 2023-12-08T08:39:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.183+0000 7f258ac7c640 1 -- 172.21.15.16:0/3088214195 wait complete. 2023-12-08T08:39:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.184+0000 7f258ac7c640 1 Processor -- start 2023-12-08T08:39:22.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.184+0000 7f258ac7c640 1 -- start start 2023-12-08T08:39:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.185+0000 7f258ac7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25840b0cf0 0x7f258414d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.185+0000 7f258ac7c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b1d70 0x7f258414fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.185+0000 7f258ac7c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25841608a0 0x7f258414ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.185+0000 7f258ac7c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25840cbe90 con 0x7f25840b0cf0 2023-12-08T08:39:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.185+0000 7f258ac7c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2584150250 con 0x7f25840b1d70 2023-12-08T08:39:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.185+0000 7f258ac7c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25840b8750 con 0x7f25841608a0 2023-12-08T08:39:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.185+0000 7f2588a18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25840b0cf0 0x7f258414d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.185+0000 7f2588a18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25840b0cf0 0x7f258414d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49458/0 (socket says 172.21.15.16:49458) 2023-12-08T08:39:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.185+0000 7f2588a18640 1 -- 172.21.15.16:0/2666692926 learned_addr learned my addr 172.21.15.16:0/2666692926 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.185+0000 7f2583fff640 1 --2- 172.21.15.16:0/2666692926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b1d70 0x7f258414fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.185+0000 7f2589219640 1 --2- 172.21.15.16:0/2666692926 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25841608a0 0x7f258414ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.185+0000 7f2588a18640 1 -- 172.21.15.16:0/2666692926 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25841608a0 msgr2=0x7f258414ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.185+0000 7f2588a18640 1 --2- 172.21.15.16:0/2666692926 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25841608a0 0x7f258414ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.186+0000 7f2588a18640 1 -- 172.21.15.16:0/2666692926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b1d70 msgr2=0x7f258414fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.186+0000 7f2588a18640 1 --2- 172.21.15.16:0/2666692926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b1d70 0x7f258414fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.186+0000 7f2588a18640 1 -- 172.21.15.16:0/2666692926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2570009400 con 0x7f25840b0cf0 2023-12-08T08:39:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.186+0000 7f2588a18640 1 --2- 172.21.15.16:0/2666692926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25840b0cf0 0x7f258414d2b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f257800c390 tx=0x7f257800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.186+0000 7f25817fa640 1 -- 172.21.15.16:0/2666692926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578005ce0 con 0x7f25840b0cf0 2023-12-08T08:39:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.186+0000 7f25817fa640 1 -- 172.21.15.16:0/2666692926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2578005e40 con 0x7f25840b0cf0 2023-12-08T08:39:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.186+0000 7f258ac7c640 1 -- 172.21.15.16:0/2666692926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25840b8a30 con 0x7f25840b0cf0 2023-12-08T08:39:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.186+0000 7f25817fa640 1 -- 172.21.15.16:0/2666692926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25780072f0 con 0x7f25840b0cf0 2023-12-08T08:39:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.186+0000 7f258ac7c640 1 -- 172.21.15.16:0/2666692926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258414cf60 con 0x7f25840b0cf0 2023-12-08T08:39:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.187+0000 7f258ac7c640 1 -- 172.21.15.16:0/2666692926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2584067430 con 0x7f25840b0cf0 2023-12-08T08:39:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.187+0000 7f25817fa640 1 -- 172.21.15.16:0/2666692926 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2578007450 con 0x7f25840b0cf0 2023-12-08T08:39:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.188+0000 7f25817fa640 1 --2- 172.21.15.16:0/2666692926 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f256806ae70 0x7f256806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.188+0000 7f25817fa640 1 -- 172.21.15.16:0/2666692926 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f2578088fd0 con 0x7f25840b0cf0 2023-12-08T08:39:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.188+0000 7f2583fff640 1 --2- 172.21.15.16:0/2666692926 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f256806ae70 0x7f256806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.188+0000 7f2583fff640 1 --2- 172.21.15.16:0/2666692926 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f256806ae70 0x7f256806d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2570036000 tx=0x7f257003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:22.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.192+0000 7f25817fa640 1 -- 172.21.15.16:0/2666692926 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2578053ca0 con 0x7f25840b0cf0 2023-12-08T08:39:22.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.382+0000 7f258ac7c640 1 -- 172.21.15.16:0/2666692926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f25840d18e0 con 0x7f25840b0cf0 2023-12-08T08:39:22.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.383+0000 7f25817fa640 1 -- 172.21.15.16:0/2666692926 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2578017080 con 0x7f25840b0cf0 2023-12-08T08:39:22.385 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-12-08T08:39:22.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.384+0000 7f258ac7c640 1 -- 172.21.15.16:0/2666692926 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f256806ae70 msgr2=0x7f256806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.384+0000 7f258ac7c640 1 --2- 172.21.15.16:0/2666692926 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f256806ae70 0x7f256806d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2570036000 tx=0x7f257003d040 comp rx=0 tx=0).stop 2023-12-08T08:39:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.384+0000 7f258ac7c640 1 -- 172.21.15.16:0/2666692926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25840b0cf0 msgr2=0x7f258414d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.384+0000 7f258ac7c640 1 --2- 172.21.15.16:0/2666692926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25840b0cf0 0x7f258414d2b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f257800c390 tx=0x7f257800c850 comp rx=0 tx=0).stop 2023-12-08T08:39:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.384+0000 7f258ac7c640 1 -- 172.21.15.16:0/2666692926 shutdown_connections 2023-12-08T08:39:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.384+0000 7f258ac7c640 1 --2- 172.21.15.16:0/2666692926 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f256806ae70 0x7f256806d330 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.384+0000 7f258ac7c640 1 --2- 172.21.15.16:0/2666692926 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f25841608a0 0x7f258414ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.384+0000 7f258ac7c640 1 --2- 172.21.15.16:0/2666692926 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f25840b1d70 0x7f258414fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.384+0000 7f258ac7c640 1 --2- 172.21.15.16:0/2666692926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25840b0cf0 0x7f258414d2b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.384+0000 7f258ac7c640 1 -- 172.21.15.16:0/2666692926 >> 172.21.15.16:0/2666692926 conn(0x7f25840d3030 msgr2=0x7f25840b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.384+0000 7f258ac7c640 1 -- 172.21.15.16:0/2666692926 shutdown_connections 2023-12-08T08:39:22.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.384+0000 7f258ac7c640 1 -- 172.21.15.16:0/2666692926 wait complete. 2023-12-08T08:39:22.397 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi016.front.sepia.ceph.com... 2023-12-08T08:39:22.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:22.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-12-08T08:39:22.429 INFO:teuthology.orchestra.run:waiting for 300 2023-12-08T08:39:22.429 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-12-08T08:39:22.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:22.429 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-08T08:39:22.484 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:39:22.484 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T08:39:22.503 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:39:22.504 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T08:39:22.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:22.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.667+0000 7f063663d640 1 Processor -- start 2023-12-08T08:39:22.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.667+0000 7f063663d640 1 -- start start 2023-12-08T08:39:22.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f063663d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06300ccd70 0x7f06300ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f063663d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f063006c650 0x7f063015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f063663d640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06300cf9c0 con 0x7f06300cf1e0 2023-12-08T08:39:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f063663d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06300d0a60 con 0x7f063006c650 2023-12-08T08:39:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f063663d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06300ce7b0 con 0x7f06300ccd70 2023-12-08T08:39:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f0634bda640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f063006c650 0x7f063015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f062f7fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f06300cf1e0 0x7f063006bf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34354/0 (socket says 172.21.15.16:34354) 2023-12-08T08:39:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f062f7fe640 1 -- 172.21.15.16:0/3943853484 learned_addr learned my addr 172.21.15.16:0/3943853484 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f0634bda640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f063006c650 0x7f063015ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49474/0 (socket says 172.21.15.16:49474) 2023-12-08T08:39:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f062ffff640 1 --2- 172.21.15.16:0/3943853484 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06300ccd70 0x7f06300ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f0634bda640 1 -- 172.21.15.16:0/3943853484 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06300ccd70 msgr2=0x7f06300ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f0634bda640 1 --2- 172.21.15.16:0/3943853484 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06300ccd70 0x7f06300ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f0634bda640 1 -- 172.21.15.16:0/3943853484 >> v1:172.21.15.16:6789/0 conn(0x7f06300cf1e0 legacy=0x7f063006bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f0634bda640 1 -- 172.21.15.16:0/3943853484 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06300cfca0 con 0x7f063006c650 2023-12-08T08:39:22.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.668+0000 7f062effd640 1 -- 172.21.15.16:0/3943853484 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282858375 0 0) 0x7f06300cf9c0 con 0x7f06300cf1e0 2023-12-08T08:39:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.669+0000 7f0634bda640 1 --2- 172.21.15.16:0/3943853484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063006c650 0x7f063015ca50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0628009d90 tx=0x7f062802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e38e8f8dbab23a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.669+0000 7f062effd640 1 -- 172.21.15.16:0/3943853484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062802cc50 con 0x7f063006c650 2023-12-08T08:39:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.669+0000 7f062effd640 1 -- 172.21.15.16:0/3943853484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062802cdb0 con 0x7f063006c650 2023-12-08T08:39:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.669+0000 7f062effd640 1 -- 172.21.15.16:0/3943853484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628034660 con 0x7f063006c650 2023-12-08T08:39:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.670+0000 7f063663d640 1 -- 172.21.15.16:0/3943853484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063006c650 msgr2=0x7f063015ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.670+0000 7f063663d640 1 --2- 172.21.15.16:0/3943853484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063006c650 0x7f063015ca50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0628009d90 tx=0x7f062802bea0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.670+0000 7f063663d640 1 -- 172.21.15.16:0/3943853484 shutdown_connections 2023-12-08T08:39:22.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.670+0000 7f063663d640 1 --2- 172.21.15.16:0/3943853484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063006c650 0x7f063015ca50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.670+0000 7f063663d640 1 --2- 172.21.15.16:0/3943853484 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06300ccd70 0x7f06300ce270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.670+0000 7f063663d640 1 -- 172.21.15.16:0/3943853484 >> 172.21.15.16:0/3943853484 conn(0x7f06300d0de0 msgr2=0x7f06300d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.670+0000 7f063663d640 1 -- 172.21.15.16:0/3943853484 shutdown_connections 2023-12-08T08:39:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.670+0000 7f063663d640 1 -- 172.21.15.16:0/3943853484 wait complete. 2023-12-08T08:39:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.671+0000 7f063663d640 1 Processor -- start 2023-12-08T08:39:22.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.671+0000 7f063663d640 1 -- start start 2023-12-08T08:39:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f063663d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f063006c650 0x7f06300c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f063663d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06300ccd70 0x7f063015b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f063663d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06300cf1e0 0x7f063015b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f063663d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06300c87b0 con 0x7f06300ccd70 2023-12-08T08:39:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f063663d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06300c8a90 con 0x7f063006c650 2023-12-08T08:39:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f063663d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06300c8d70 con 0x7f06300cf1e0 2023-12-08T08:39:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f062f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06300ccd70 0x7f063015b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f062f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06300ccd70 0x7f063015b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49490/0 (socket says 172.21.15.16:49490) 2023-12-08T08:39:22.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f062f7fe640 1 -- 172.21.15.16:0/956495730 learned_addr learned my addr 172.21.15.16:0/956495730 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f0634bda640 1 --2- 172.21.15.16:0/956495730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06300cf1e0 0x7f063015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f062ffff640 1 --2- 172.21.15.16:0/956495730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f063006c650 0x7f06300c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f062f7fe640 1 -- 172.21.15.16:0/956495730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06300cf1e0 msgr2=0x7f063015b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f062f7fe640 1 --2- 172.21.15.16:0/956495730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06300cf1e0 0x7f063015b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f062f7fe640 1 -- 172.21.15.16:0/956495730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f063006c650 msgr2=0x7f06300c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f062f7fe640 1 --2- 172.21.15.16:0/956495730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f063006c650 0x7f06300c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.672+0000 7f062f7fe640 1 -- 172.21.15.16:0/956495730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0628009a70 con 0x7f06300ccd70 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.673+0000 7f062f7fe640 1 --2- 172.21.15.16:0/956495730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06300ccd70 0x7f063015b110 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f062000c430 tx=0x7f062000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.673+0000 7f062cff9640 1 -- 172.21.15.16:0/956495730 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06200074e0 con 0x7f06300ccd70 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.673+0000 7f062cff9640 1 -- 172.21.15.16:0/956495730 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0620007660 con 0x7f06300ccd70 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.673+0000 7f063663d640 1 -- 172.21.15.16:0/956495730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06300c9270 con 0x7f06300ccd70 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.673+0000 7f062cff9640 1 -- 172.21.15.16:0/956495730 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0620004a20 con 0x7f06300ccd70 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.673+0000 7f063663d640 1 -- 172.21.15.16:0/956495730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06300c83f0 con 0x7f06300ccd70 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.674+0000 7f063663d640 1 -- 172.21.15.16:0/956495730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0630067430 con 0x7f06300ccd70 2023-12-08T08:39:22.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.674+0000 7f062cff9640 1 -- 172.21.15.16:0/956495730 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0620004b80 con 0x7f06300ccd70 2023-12-08T08:39:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.675+0000 7f062cff9640 1 --2- 172.21.15.16:0/956495730 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f061c06ae40 0x7f061c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.675+0000 7f062cff9640 1 -- 172.21.15.16:0/956495730 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f06200896d0 con 0x7f06300ccd70 2023-12-08T08:39:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.675+0000 7f062ffff640 1 --2- 172.21.15.16:0/956495730 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f061c06ae40 0x7f061c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.675+0000 7f062ffff640 1 --2- 172.21.15.16:0/956495730 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f061c06ae40 0x7f061c06d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f06301600a0 tx=0x7f0624008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:22.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.679+0000 7f062cff9640 1 -- 172.21.15.16:0/956495730 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0620054450 con 0x7f06300ccd70 2023-12-08T08:39:22.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.856+0000 7f063663d640 1 -- 172.21.15.16:0/956495730 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0630072520 con 0x7f06300ccd70 2023-12-08T08:39:22.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.857+0000 7f062cff9640 1 -- 172.21.15.16:0/956495730 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7f0620057a70 con 0x7f06300ccd70 2023-12-08T08:39:22.859 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:22.859 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.859+0000 7f063663d640 1 -- 172.21.15.16:0/956495730 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f061c06ae40 msgr2=0x7f061c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.859+0000 7f063663d640 1 --2- 172.21.15.16:0/956495730 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f061c06ae40 0x7f061c06d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f06301600a0 tx=0x7f0624008040 comp rx=0 tx=0).stop 2023-12-08T08:39:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.859+0000 7f063663d640 1 -- 172.21.15.16:0/956495730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06300ccd70 msgr2=0x7f063015b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.859+0000 7f063663d640 1 --2- 172.21.15.16:0/956495730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06300ccd70 0x7f063015b110 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f062000c430 tx=0x7f062000c8f0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.860+0000 7f063663d640 1 -- 172.21.15.16:0/956495730 shutdown_connections 2023-12-08T08:39:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.860+0000 7f063663d640 1 --2- 172.21.15.16:0/956495730 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f061c06ae40 0x7f061c06d300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.860+0000 7f063663d640 1 --2- 172.21.15.16:0/956495730 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f06300cf1e0 0x7f063015b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.860+0000 7f063663d640 1 --2- 172.21.15.16:0/956495730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06300ccd70 0x7f063015b110 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.860+0000 7f063663d640 1 --2- 172.21.15.16:0/956495730 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f063006c650 0x7f06300c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.860+0000 7f063663d640 1 -- 172.21.15.16:0/956495730 >> 172.21.15.16:0/956495730 conn(0x7f06300d0de0 msgr2=0x7f063006f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.860+0000 7f063663d640 1 -- 172.21.15.16:0/956495730 shutdown_connections 2023-12-08T08:39:22.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.860+0000 7f063663d640 1 -- 172.21.15.16:0/956495730 wait complete. 2023-12-08T08:39:22.873 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:39:22.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:22.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.981+0000 7f88c1490640 1 Processor -- start 2023-12-08T08:39:22.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.981+0000 7f88c1490640 1 -- start start 2023-12-08T08:39:22.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88c1490640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f88bc15d4f0 0x7f88bc0b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88c1490640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88bc0b0920 0x7f88bc0b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88c1490640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88bc0adb50 0x7f88bc0adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88c1490640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88bc06bb40 con 0x7f88bc0adb50 2023-12-08T08:39:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88c1490640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88bc0b2280 con 0x7f88bc15d4f0 2023-12-08T08:39:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88c1490640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc0b2710 con 0x7f88bc0b0920 2023-12-08T08:39:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88bb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88bc0adb50 0x7f88bc0adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88bb7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88bc0adb50 0x7f88bc0adfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49498/0 (socket says 172.21.15.16:49498) 2023-12-08T08:39:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88bb7fe640 1 -- 172.21.15.16:0/199029064 learned_addr learned my addr 172.21.15.16:0/199029064 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88b3fff640 1 --2- 172.21.15.16:0/199029064 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88bc0b0920 0x7f88bc0b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88baffd640 1 --2- 172.21.15.16:0/199029064 >> v2:172.21.15.112:3300/0 conn(0x7f88bc15d4f0 0x7f88bc0b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88bb7fe640 1 -- 172.21.15.16:0/199029064 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88bc0b0920 msgr2=0x7f88bc0b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88bb7fe640 1 --2- 172.21.15.16:0/199029064 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88bc0b0920 0x7f88bc0b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88bb7fe640 1 -- 172.21.15.16:0/199029064 >> v2:172.21.15.112:3300/0 conn(0x7f88bc15d4f0 msgr2=0x7f88bc0b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:22.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88bb7fe640 1 --2- 172.21.15.16:0/199029064 >> v2:172.21.15.112:3300/0 conn(0x7f88bc15d4f0 0x7f88bc0b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88bb7fe640 1 -- 172.21.15.16:0/199029064 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc0b1690 con 0x7f88bc0adb50 2023-12-08T08:39:22.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.982+0000 7f88baffd640 1 --2- 172.21.15.16:0/199029064 >> v2:172.21.15.112:3300/0 conn(0x7f88bc15d4f0 0x7f88bc0b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:22.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.983+0000 7f88bb7fe640 1 --2- 172.21.15.16:0/199029064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88bc0adb50 0x7f88bc0adfe0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f88bc071970 tx=0x7f88b402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43583c299e4bec2c server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:22.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.983+0000 7f88ba7fc640 1 -- 172.21.15.16:0/199029064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b402ca70 con 0x7f88bc0adb50 2023-12-08T08:39:22.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.983+0000 7f88ba7fc640 1 -- 172.21.15.16:0/199029064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b402cbd0 con 0x7f88bc0adb50 2023-12-08T08:39:22.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.983+0000 7f88ba7fc640 1 -- 172.21.15.16:0/199029064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b40343f0 con 0x7f88bc0adb50 2023-12-08T08:39:22.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.984+0000 7f88c1490640 1 -- 172.21.15.16:0/199029064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88bc0adb50 msgr2=0x7f88bc0adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.984+0000 7f88c1490640 1 --2- 172.21.15.16:0/199029064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88bc0adb50 0x7f88bc0adfe0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f88bc071970 tx=0x7f88b402bed0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.984+0000 7f88c1490640 1 -- 172.21.15.16:0/199029064 shutdown_connections 2023-12-08T08:39:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.984+0000 7f88c1490640 1 --2- 172.21.15.16:0/199029064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88bc0adb50 0x7f88bc0adfe0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.984+0000 7f88c1490640 1 --2- 172.21.15.16:0/199029064 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88bc0b0920 0x7f88bc0b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.984+0000 7f88c1490640 1 --2- 172.21.15.16:0/199029064 >> v2:172.21.15.112:3300/0 conn(0x7f88bc15d4f0 0x7f88bc0b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.984+0000 7f88c1490640 1 -- 172.21.15.16:0/199029064 >> 172.21.15.16:0/199029064 conn(0x7f88bc0d0e00 msgr2=0x7f88bc0d1210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.984+0000 7f88c1490640 1 -- 172.21.15.16:0/199029064 shutdown_connections 2023-12-08T08:39:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.984+0000 7f88c1490640 1 -- 172.21.15.16:0/199029064 wait complete. 2023-12-08T08:39:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.985+0000 7f88c1490640 1 Processor -- start 2023-12-08T08:39:22.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.985+0000 7f88c1490640 1 -- start start 2023-12-08T08:39:22.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88c1490640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88bc0adb50 0x7f88bc15a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88c1490640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88bc0b0920 0x7f88bc1f3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88c1490640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88bc15d4f0 0x7f88bc0c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88c1490640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc1f3740 con 0x7f88bc15d4f0 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88c1490640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc0ca660 con 0x7f88bc0adb50 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88c1490640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88bc0c8810 con 0x7f88bc0b0920 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88bc15d4f0 0x7f88bc0c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88bb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88bc15d4f0 0x7f88bc0c8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49508/0 (socket says 172.21.15.16:49508) 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88bb7fe640 1 -- 172.21.15.16:0/1130597202 learned_addr learned my addr 172.21.15.16:0/1130597202 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88b3fff640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88bc0b0920 0x7f88bc1f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88baffd640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88bc0adb50 0x7f88bc15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88bb7fe640 1 -- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88bc0b0920 msgr2=0x7f88bc1f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88bb7fe640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88bc0b0920 0x7f88bc1f3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88bb7fe640 1 -- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88bc0adb50 msgr2=0x7f88bc15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88bb7fe640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88bc0adb50 0x7f88bc15a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:22.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88bb7fe640 1 -- 172.21.15.16:0/1130597202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b4009b30 con 0x7f88bc15d4f0 2023-12-08T08:39:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88b3fff640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88bc0b0920 0x7f88bc1f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88baffd640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88bc0adb50 0x7f88bc15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.986+0000 7f88bb7fe640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88bc15d4f0 0x7f88bc0c8ff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f88b4036040 tx=0x7f88b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.987+0000 7f88b37fe640 1 -- 172.21.15.16:0/1130597202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4009510 con 0x7f88bc15d4f0 2023-12-08T08:39:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.987+0000 7f88b37fe640 1 -- 172.21.15.16:0/1130597202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b403e450 con 0x7f88bc15d4f0 2023-12-08T08:39:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.987+0000 7f88c1490640 1 -- 172.21.15.16:0/1130597202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88bc0ca920 con 0x7f88bc15d4f0 2023-12-08T08:39:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.987+0000 7f88b37fe640 1 -- 172.21.15.16:0/1130597202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b403e7e0 con 0x7f88bc15d4f0 2023-12-08T08:39:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.987+0000 7f88c1490640 1 -- 172.21.15.16:0/1130597202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88bc0c8ad0 con 0x7f88bc15d4f0 2023-12-08T08:39:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.988+0000 7f88c1490640 1 -- 172.21.15.16:0/1130597202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88bc067430 con 0x7f88bc15d4f0 2023-12-08T08:39:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.988+0000 7f88b37fe640 1 -- 172.21.15.16:0/1130597202 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f88b403e940 con 0x7f88bc15d4f0 2023-12-08T08:39:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.988+0000 7f88b37fe640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f889c06ae40 0x7f889c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.989+0000 7f88b37fe640 1 -- 172.21.15.16:0/1130597202 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f88b40ac820 con 0x7f88bc15d4f0 2023-12-08T08:39:22.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.989+0000 7f88baffd640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f889c06ae40 0x7f889c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:22.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.989+0000 7f88baffd640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f889c06ae40 0x7f889c06d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0b6b60 tx=0x7f88ac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:22.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:22.993+0000 7f88b37fe640 1 -- 172.21.15.16:0/1130597202 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88b40774e0 con 0x7f88bc15d4f0 2023-12-08T08:39:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.167+0000 7f88c1490640 1 -- 172.21.15.16:0/1130597202 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88bc0cade0 con 0x7f88bc15d4f0 2023-12-08T08:39:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.167+0000 7f88b37fe640 1 -- 172.21.15.16:0/1130597202 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88b407ab00 con 0x7f88bc15d4f0 2023-12-08T08:39:23.169 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.169+0000 7f88c1490640 1 -- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f889c06ae40 msgr2=0x7f889c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.169+0000 7f88c1490640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f889c06ae40 0x7f889c06d300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0b6b60 tx=0x7f88ac008040 comp rx=0 tx=0).stop 2023-12-08T08:39:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.169+0000 7f88c1490640 1 -- 172.21.15.16:0/1130597202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88bc15d4f0 msgr2=0x7f88bc0c8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.169+0000 7f88c1490640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88bc15d4f0 0x7f88bc0c8ff0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f88b4036040 tx=0x7f88b4004a40 comp rx=0 tx=0).stop 2023-12-08T08:39:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.169+0000 7f88c1490640 1 -- 172.21.15.16:0/1130597202 shutdown_connections 2023-12-08T08:39:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.169+0000 7f88c1490640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f889c06ae40 0x7f889c06d300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.169+0000 7f88c1490640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88bc15d4f0 0x7f88bc0c8ff0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.169+0000 7f88c1490640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f88bc0b0920 0x7f88bc1f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.169+0000 7f88c1490640 1 --2- 172.21.15.16:0/1130597202 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f88bc0adb50 0x7f88bc15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.169+0000 7f88c1490640 1 -- 172.21.15.16:0/1130597202 >> 172.21.15.16:0/1130597202 conn(0x7f88bc0d0e00 msgr2=0x7f88bc06ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.169+0000 7f88c1490640 1 -- 172.21.15.16:0/1130597202 shutdown_connections 2023-12-08T08:39:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.169+0000 7f88c1490640 1 -- 172.21.15.16:0/1130597202 wait complete. 2023-12-08T08:39:23.182 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:39:23.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.289+0000 7f03a97dc640 1 Processor -- start 2023-12-08T08:39:23.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.289+0000 7f03a97dc640 1 -- start start 2023-12-08T08:39:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a97dc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f03a40cfa80 0x7f03a40d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a97dc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03a40ce690 0x7f03a40cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a97dc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03a40d10f0 0x7f03a40d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a97dc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a40b5ec0 con 0x7f03a40ce690 2023-12-08T08:39:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a97dc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a40d03a0 con 0x7f03a40cfa80 2023-12-08T08:39:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a97dc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40cede0 con 0x7f03a40d10f0 2023-12-08T08:39:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a27fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03a40ce690 0x7f03a40cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a27fc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03a40ce690 0x7f03a40cfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49514/0 (socket says 172.21.15.16:49514) 2023-12-08T08:39:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a27fc640 1 -- 172.21.15.16:0/1245227325 learned_addr learned my addr 172.21.15.16:0/1245227325 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a2ffd640 1 --2- 172.21.15.16:0/1245227325 >> v2:172.21.15.112:3300/0 conn(0x7f03a40cfa80 0x7f03a40d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a27fc640 1 -- 172.21.15.16:0/1245227325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03a40d10f0 msgr2=0x7f03a40d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a37fe640 1 --2- 172.21.15.16:0/1245227325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03a40d10f0 0x7f03a40d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a27fc640 1 --2- 172.21.15.16:0/1245227325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03a40d10f0 0x7f03a40d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a27fc640 1 -- 172.21.15.16:0/1245227325 >> v2:172.21.15.112:3300/0 conn(0x7f03a40cfa80 msgr2=0x7f03a40d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a27fc640 1 --2- 172.21.15.16:0/1245227325 >> v2:172.21.15.112:3300/0 conn(0x7f03a40cfa80 0x7f03a40d3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a27fc640 1 -- 172.21.15.16:0/1245227325 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a40b6110 con 0x7f03a40ce690 2023-12-08T08:39:23.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.290+0000 7f03a2ffd640 1 --2- 172.21.15.16:0/1245227325 >> v2:172.21.15.112:3300/0 conn(0x7f03a40cfa80 0x7f03a40d3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.291+0000 7f03a27fc640 1 --2- 172.21.15.16:0/1245227325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40ce690 0x7f03a40cfe60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f03a406ad40 tx=0x7f038c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d444c7ce8e37966 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.291+0000 7f03a1ffb640 1 -- 172.21.15.16:0/1245227325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0317e0 con 0x7f03a40ce690 2023-12-08T08:39:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.291+0000 7f03a1ffb640 1 -- 172.21.15.16:0/1245227325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c031940 con 0x7f03a40ce690 2023-12-08T08:39:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.291+0000 7f03a1ffb640 1 -- 172.21.15.16:0/1245227325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c031cd0 con 0x7f03a40ce690 2023-12-08T08:39:23.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.292+0000 7f03a97dc640 1 -- 172.21.15.16:0/1245227325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40ce690 msgr2=0x7f03a40cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.292+0000 7f03a97dc640 1 --2- 172.21.15.16:0/1245227325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40ce690 0x7f03a40cfe60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f03a406ad40 tx=0x7f038c02b880 comp rx=0 tx=0).stop 2023-12-08T08:39:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.292+0000 7f03a97dc640 1 -- 172.21.15.16:0/1245227325 shutdown_connections 2023-12-08T08:39:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.292+0000 7f03a97dc640 1 --2- 172.21.15.16:0/1245227325 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03a40d10f0 0x7f03a40d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.292+0000 7f03a97dc640 1 --2- 172.21.15.16:0/1245227325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40ce690 0x7f03a40cfe60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.292+0000 7f03a97dc640 1 --2- 172.21.15.16:0/1245227325 >> v2:172.21.15.112:3300/0 conn(0x7f03a40cfa80 0x7f03a40d3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.292+0000 7f03a97dc640 1 -- 172.21.15.16:0/1245227325 >> 172.21.15.16:0/1245227325 conn(0x7f03a409c340 msgr2=0x7f03a40d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.292+0000 7f03a97dc640 1 -- 172.21.15.16:0/1245227325 shutdown_connections 2023-12-08T08:39:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.292+0000 7f03a97dc640 1 -- 172.21.15.16:0/1245227325 wait complete. 2023-12-08T08:39:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.293+0000 7f03a97dc640 1 Processor -- start 2023-12-08T08:39:23.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.293+0000 7f03a97dc640 1 -- start start 2023-12-08T08:39:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.294+0000 7f03a97dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40ce690 0x7f03a415add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.294+0000 7f03a97dc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a40cfa80 0x7f03a40cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.294+0000 7f03a97dc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03a40d10f0 0x7f03a41598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.294+0000 7f03a97dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a4158850 con 0x7f03a40ce690 2023-12-08T08:39:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.294+0000 7f03a97dc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a4158b30 con 0x7f03a40cfa80 2023-12-08T08:39:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.294+0000 7f03a97dc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a4158e10 con 0x7f03a40d10f0 2023-12-08T08:39:23.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.294+0000 7f03a2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40ce690 0x7f03a415add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.294+0000 7f03a2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40ce690 0x7f03a415add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49520/0 (socket says 172.21.15.16:49520) 2023-12-08T08:39:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.294+0000 7f03a2ffd640 1 -- 172.21.15.16:0/1768339192 learned_addr learned my addr 172.21.15.16:0/1768339192 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.294+0000 7f03a27fc640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a40cfa80 0x7f03a40cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.294+0000 7f03a37fe640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03a40d10f0 0x7f03a41598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.295+0000 7f03a2ffd640 1 -- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03a40d10f0 msgr2=0x7f03a41598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.295+0000 7f03a2ffd640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03a40d10f0 0x7f03a41598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.295+0000 7f03a2ffd640 1 -- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a40cfa80 msgr2=0x7f03a40cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.295+0000 7f03a2ffd640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a40cfa80 0x7f03a40cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.295+0000 7f03a2ffd640 1 -- 172.21.15.16:0/1768339192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c0094d0 con 0x7f03a40ce690 2023-12-08T08:39:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.295+0000 7f03a37fe640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03a40d10f0 0x7f03a41598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.295+0000 7f03a27fc640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a40cfa80 0x7f03a40cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.295+0000 7f03a2ffd640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40ce690 0x7f03a415add0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f039c00c7f0 tx=0x7f039c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.295+0000 7f039b7fe640 1 -- 172.21.15.16:0/1768339192 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c007820 con 0x7f03a40ce690 2023-12-08T08:39:23.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.295+0000 7f039b7fe640 1 -- 172.21.15.16:0/1768339192 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c007980 con 0x7f03a40ce690 2023-12-08T08:39:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.295+0000 7f03a97dc640 1 -- 172.21.15.16:0/1768339192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a41590f0 con 0x7f03a40ce690 2023-12-08T08:39:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.295+0000 7f039b7fe640 1 -- 172.21.15.16:0/1768339192 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c007d30 con 0x7f03a40ce690 2023-12-08T08:39:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.295+0000 7f03a97dc640 1 -- 172.21.15.16:0/1768339192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a415b310 con 0x7f03a40ce690 2023-12-08T08:39:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.296+0000 7f03a97dc640 1 -- 172.21.15.16:0/1768339192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a4067430 con 0x7f03a40ce690 2023-12-08T08:39:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.296+0000 7f039b7fe640 1 -- 172.21.15.16:0/1768339192 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f039c0113f0 con 0x7f03a40ce690 2023-12-08T08:39:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.297+0000 7f039b7fe640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f038406ae40 0x7f038406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.297+0000 7f039b7fe640 1 -- 172.21.15.16:0/1768339192 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f039c089e40 con 0x7f03a40ce690 2023-12-08T08:39:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.297+0000 7f03a27fc640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f038406ae40 0x7f038406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.298+0000 7f03a27fc640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f038406ae40 0x7f038406d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f038c036000 tx=0x7f038c03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:23.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.301+0000 7f039b7fe640 1 -- 172.21.15.16:0/1768339192 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f039c0549f0 con 0x7f03a40ce690 2023-12-08T08:39:23.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.475+0000 7f03a97dc640 1 -- 172.21.15.16:0/1768339192 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03a40b6b80 con 0x7f03a40ce690 2023-12-08T08:39:23.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.475+0000 7f039b7fe640 1 -- 172.21.15.16:0/1768339192 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f039c058010 con 0x7f03a40ce690 2023-12-08T08:39:23.478 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.477+0000 7f03a97dc640 1 -- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f038406ae40 msgr2=0x7f038406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.477+0000 7f03a97dc640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f038406ae40 0x7f038406d300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f038c036000 tx=0x7f038c03d040 comp rx=0 tx=0).stop 2023-12-08T08:39:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.477+0000 7f03a97dc640 1 -- 172.21.15.16:0/1768339192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40ce690 msgr2=0x7f03a415add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.477+0000 7f03a97dc640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40ce690 0x7f03a415add0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f039c00c7f0 tx=0x7f039c00ccb0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.478+0000 7f03a97dc640 1 -- 172.21.15.16:0/1768339192 shutdown_connections 2023-12-08T08:39:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.478+0000 7f03a97dc640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f038406ae40 0x7f038406d300 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.478+0000 7f03a97dc640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f03a40d10f0 0x7f03a41598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.478+0000 7f03a97dc640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f03a40cfa80 0x7f03a40cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.478+0000 7f03a97dc640 1 --2- 172.21.15.16:0/1768339192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03a40ce690 0x7f03a415add0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.478+0000 7f03a97dc640 1 -- 172.21.15.16:0/1768339192 >> 172.21.15.16:0/1768339192 conn(0x7f03a409c340 msgr2=0x7f03a40d3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.478+0000 7f03a97dc640 1 -- 172.21.15.16:0/1768339192 shutdown_connections 2023-12-08T08:39:23.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.478+0000 7f03a97dc640 1 -- 172.21.15.16:0/1768339192 wait complete. 2023-12-08T08:39:23.491 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:39:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.598+0000 7f2f26dec640 1 Processor -- start 2023-12-08T08:39:23.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.598+0000 7f2f26dec640 1 -- start start 2023-12-08T08:39:23.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.598+0000 7f2f26dec640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f20145d80 0x7f2f20071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.598+0000 7f2f26dec640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f2006a4d0 0x7f2f2006a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.598+0000 7f2f26dec640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2006ae60 con 0x7f2f200b4920 2023-12-08T08:39:23.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.599+0000 7f2f26dec640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f200b4e00 con 0x7f2f20145d80 2023-12-08T08:39:23.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.599+0000 7f2f26dec640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f200b50e0 con 0x7f2f2006a4d0 2023-12-08T08:39:23.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.599+0000 7f2f24b88640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f20145d80 0x7f2f20071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.599+0000 7f2f24b88640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f20145d80 0x7f2f20071ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49524/0 (socket says 172.21.15.16:49524) 2023-12-08T08:39:23.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.599+0000 7f2f24b88640 1 -- 172.21.15.16:0/1931695902 learned_addr learned my addr 172.21.15.16:0/1931695902 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:23.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.599+0000 7f2f25389640 1 --2- 172.21.15.16:0/1931695902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f2006a4d0 0x7f2f2006a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.599+0000 7f2f24b88640 1 -- 172.21.15.16:0/1931695902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f2006a4d0 msgr2=0x7f2f2006a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:23.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.599+0000 7f2f24b88640 1 --2- 172.21.15.16:0/1931695902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f2006a4d0 0x7f2f2006a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.599+0000 7f2f24b88640 1 -- 172.21.15.16:0/1931695902 >> v1:172.21.15.112:6789/0 conn(0x7f2f200b4920 legacy=0x7f2f200acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.599+0000 7f2f24b88640 1 -- 172.21.15.16:0/1931695902 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2006cfb0 con 0x7f2f20145d80 2023-12-08T08:39:23.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.600+0000 7f2f24b88640 1 --2- 172.21.15.16:0/1931695902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f20145d80 0x7f2f20071ae0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2f14009870 tx=0x7f2f1402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd4ef8d6c28fd0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:23.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.600+0000 7f2f1f7fe640 1 -- 172.21.15.16:0/1931695902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14030840 con 0x7f2f20145d80 2023-12-08T08:39:23.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.600+0000 7f2f1f7fe640 1 -- 172.21.15.16:0/1931695902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f140309a0 con 0x7f2f20145d80 2023-12-08T08:39:23.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.600+0000 7f2f1f7fe640 1 -- 172.21.15.16:0/1931695902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f14030d30 con 0x7f2f20145d80 2023-12-08T08:39:23.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.600+0000 7f2f26dec640 1 -- 172.21.15.16:0/1931695902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f20145d80 msgr2=0x7f2f20071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.600+0000 7f2f26dec640 1 --2- 172.21.15.16:0/1931695902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f20145d80 0x7f2f20071ae0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2f14009870 tx=0x7f2f1402b980 comp rx=0 tx=0).stop 2023-12-08T08:39:23.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.600+0000 7f2f26dec640 1 -- 172.21.15.16:0/1931695902 shutdown_connections 2023-12-08T08:39:23.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.600+0000 7f2f26dec640 1 --2- 172.21.15.16:0/1931695902 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f2006a4d0 0x7f2f2006a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.600+0000 7f2f26dec640 1 --2- 172.21.15.16:0/1931695902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f20145d80 0x7f2f20071ae0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.600+0000 7f2f26dec640 1 -- 172.21.15.16:0/1931695902 >> 172.21.15.16:0/1931695902 conn(0x7f2f2006b7c0 msgr2=0x7f2f2006bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:23.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.601+0000 7f2f26dec640 1 -- 172.21.15.16:0/1931695902 shutdown_connections 2023-12-08T08:39:23.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.601+0000 7f2f26dec640 1 -- 172.21.15.16:0/1931695902 wait complete. 2023-12-08T08:39:23.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.602+0000 7f2f26dec640 1 Processor -- start 2023-12-08T08:39:23.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.602+0000 7f2f26dec640 1 -- start start 2023-12-08T08:39:23.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f26dec640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f2006a4d0 0x7f2f2015c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f26dec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f200b4920 0x7f2f200b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f26dec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f20145d80 0x7f2f200aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f26dec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f200aeb60 con 0x7f2f200b4920 2023-12-08T08:39:23.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f26dec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2015d8e0 con 0x7f2f20145d80 2023-12-08T08:39:23.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f26dec640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f200ae5a0 con 0x7f2f2006a4d0 2023-12-08T08:39:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f200b4920 0x7f2f200b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f1ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f200b4920 0x7f2f200b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49532/0 (socket says 172.21.15.16:49532) 2023-12-08T08:39:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f1ffff640 1 -- 172.21.15.16:0/1889578158 learned_addr learned my addr 172.21.15.16:0/1889578158 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f24b88640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f2006a4d0 0x7f2f2015c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f25389640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f20145d80 0x7f2f200aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f1ffff640 1 -- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f2006a4d0 msgr2=0x7f2f2015c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f1ffff640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f2006a4d0 0x7f2f2015c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f1ffff640 1 -- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f20145d80 msgr2=0x7f2f200aeec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f1ffff640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f20145d80 0x7f2f200aeec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f1ffff640 1 -- 172.21.15.16:0/1889578158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f140094d0 con 0x7f2f200b4920 2023-12-08T08:39:23.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f25389640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f20145d80 0x7f2f200aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:23.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f1ffff640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f200b4920 0x7f2f200b0ee0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00c400 tx=0x7f2f0c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:23.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f1d7fa640 1 -- 172.21.15.16:0/1889578158 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c0052d0 con 0x7f2f200b4920 2023-12-08T08:39:23.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f1d7fa640 1 -- 172.21.15.16:0/1889578158 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0c005430 con 0x7f2f200b4920 2023-12-08T08:39:23.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.603+0000 7f2f1d7fa640 1 -- 172.21.15.16:0/1889578158 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c003850 con 0x7f2f200b4920 2023-12-08T08:39:23.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.604+0000 7f2f26dec640 1 -- 172.21.15.16:0/1889578158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f200ae880 con 0x7f2f200b4920 2023-12-08T08:39:23.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.604+0000 7f2f26dec640 1 -- 172.21.15.16:0/1889578158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2015cec0 con 0x7f2f200b4920 2023-12-08T08:39:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.605+0000 7f2f26dec640 1 -- 172.21.15.16:0/1889578158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f20071460 con 0x7f2f200b4920 2023-12-08T08:39:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.605+0000 7f2f1d7fa640 1 -- 172.21.15.16:0/1889578158 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f0c0039b0 con 0x7f2f200b4920 2023-12-08T08:39:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.605+0000 7f2f1d7fa640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f0406ae70 0x7f2f0406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.606+0000 7f2f1d7fa640 1 -- 172.21.15.16:0/1889578158 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f2f0c088db0 con 0x7f2f200b4920 2023-12-08T08:39:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.606+0000 7f2f24b88640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f0406ae70 0x7f2f0406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.606+0000 7f2f24b88640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f0406ae70 0x7f2f0406d330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2f14036000 tx=0x7f2f1403b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:23.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.610+0000 7f2f1d7fa640 1 -- 172.21.15.16:0/1889578158 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f0c053b40 con 0x7f2f200b4920 2023-12-08T08:39:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.788+0000 7f2f26dec640 1 -- 172.21.15.16:0/1889578158 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f2006ae60 con 0x7f2f200b4920 2023-12-08T08:39:23.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.788+0000 7f2f1d7fa640 1 -- 172.21.15.16:0/1889578158 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f2006ae60 con 0x7f2f200b4920 2023-12-08T08:39:23.791 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:39:23.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.790+0000 7f2f26dec640 1 -- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f0406ae70 msgr2=0x7f2f0406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.790+0000 7f2f26dec640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f0406ae70 0x7f2f0406d330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2f14036000 tx=0x7f2f1403b040 comp rx=0 tx=0).stop 2023-12-08T08:39:23.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.790+0000 7f2f26dec640 1 -- 172.21.15.16:0/1889578158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f200b4920 msgr2=0x7f2f200b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.790+0000 7f2f26dec640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f200b4920 0x7f2f200b0ee0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00c400 tx=0x7f2f0c00c8c0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.790+0000 7f2f26dec640 1 -- 172.21.15.16:0/1889578158 shutdown_connections 2023-12-08T08:39:23.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.790+0000 7f2f26dec640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f0406ae70 0x7f2f0406d330 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.790+0000 7f2f26dec640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2f20145d80 0x7f2f200aeec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.790+0000 7f2f26dec640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f200b4920 0x7f2f200b0ee0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.790+0000 7f2f26dec640 1 --2- 172.21.15.16:0/1889578158 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2f2006a4d0 0x7f2f2015c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.790+0000 7f2f26dec640 1 -- 172.21.15.16:0/1889578158 >> 172.21.15.16:0/1889578158 conn(0x7f2f2006b7c0 msgr2=0x7f2f200719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:23.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.790+0000 7f2f26dec640 1 -- 172.21.15.16:0/1889578158 shutdown_connections 2023-12-08T08:39:23.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.790+0000 7f2f26dec640 1 -- 172.21.15.16:0/1889578158 wait complete. 2023-12-08T08:39:23.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:39:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.911+0000 7fd600061640 1 Processor -- start 2023-12-08T08:39:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.911+0000 7fd600061640 1 -- start start 2023-12-08T08:39:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.912+0000 7fd600061640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5f815d480 0x7fd5f80cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.912+0000 7fd600061640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f806b910 0x7fd5f806bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.912+0000 7fd600061640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f80d00d0 con 0x7fd5f806c530 2023-12-08T08:39:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.912+0000 7fd600061640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f806c9c0 con 0x7fd5f815d480 2023-12-08T08:39:23.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.912+0000 7fd600061640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f80d0310 con 0x7fd5f806b910 2023-12-08T08:39:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.912+0000 7fd5fddfd640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fd5f815d480 0x7fd5f80cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.912+0000 7fd5fd5fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f806b910 0x7fd5f806bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.912+0000 7fd5fe5fe640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fd5f806c530 0x7fd5f80cd710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:60156/0 (socket says 172.21.15.16:60156) 2023-12-08T08:39:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.912+0000 7fd5fe5fe640 1 -- 172.21.15.16:0/393611431 learned_addr learned my addr 172.21.15.16:0/393611431 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.913+0000 7fd5fd5fc640 1 -- 172.21.15.16:0/393611431 >> v2:172.21.15.112:3300/0 conn(0x7fd5f815d480 msgr2=0x7fd5f80cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.913+0000 7fd5fd5fc640 1 --2- 172.21.15.16:0/393611431 >> v2:172.21.15.112:3300/0 conn(0x7fd5f815d480 0x7fd5f80cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.913+0000 7fd5fd5fc640 1 -- 172.21.15.16:0/393611431 >> v1:172.21.15.112:6789/0 conn(0x7fd5f806c530 legacy=0x7fd5f80cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.913+0000 7fd5fd5fc640 1 -- 172.21.15.16:0/393611431 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f80d05f0 con 0x7fd5f806b910 2023-12-08T08:39:23.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.913+0000 7fd5fddfd640 1 --2- 172.21.15.16:0/393611431 >> v2:172.21.15.112:3300/0 conn(0x7fd5f815d480 0x7fd5f80cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:39:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.913+0000 7fd5fd5fc640 1 --2- 172.21.15.16:0/393611431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f806b910 0x7fd5f806bff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80097a0 tx=0x7fd5e802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47dfcfd24e0bd906 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.914+0000 7fd5fcdfb640 1 -- 172.21.15.16:0/393611431 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8031830 con 0x7fd5f806b910 2023-12-08T08:39:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.914+0000 7fd5fcdfb640 1 -- 172.21.15.16:0/393611431 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e8031990 con 0x7fd5f806b910 2023-12-08T08:39:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.914+0000 7fd5fcdfb640 1 -- 172.21.15.16:0/393611431 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8031d20 con 0x7fd5f806b910 2023-12-08T08:39:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.915+0000 7fd600061640 1 -- 172.21.15.16:0/393611431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f806b910 msgr2=0x7fd5f806bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.915+0000 7fd600061640 1 --2- 172.21.15.16:0/393611431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f806b910 0x7fd5f806bff0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80097a0 tx=0x7fd5e802b8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.915+0000 7fd600061640 1 -- 172.21.15.16:0/393611431 shutdown_connections 2023-12-08T08:39:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.915+0000 7fd600061640 1 --2- 172.21.15.16:0/393611431 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f806b910 0x7fd5f806bff0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.915+0000 7fd600061640 1 --2- 172.21.15.16:0/393611431 >> v2:172.21.15.112:3300/0 conn(0x7fd5f815d480 0x7fd5f80cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.915+0000 7fd600061640 1 -- 172.21.15.16:0/393611431 >> 172.21.15.16:0/393611431 conn(0x7fd5f80d0e90 msgr2=0x7fd5f80d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.915+0000 7fd600061640 1 -- 172.21.15.16:0/393611431 shutdown_connections 2023-12-08T08:39:23.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.915+0000 7fd600061640 1 -- 172.21.15.16:0/393611431 wait complete. 2023-12-08T08:39:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.916+0000 7fd600061640 1 Processor -- start 2023-12-08T08:39:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.916+0000 7fd600061640 1 -- start start 2023-12-08T08:39:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd600061640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f806b910 0x7fd5f81605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd600061640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f806c530 0x7fd5f814d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd600061640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f815d480 0x7fd5f815f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd600061640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f815e2f0 con 0x7fd5f806c530 2023-12-08T08:39:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd600061640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f815e5d0 con 0x7fd5f815d480 2023-12-08T08:39:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd600061640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f815e8b0 con 0x7fd5f806b910 2023-12-08T08:39:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd5fd5fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f806c530 0x7fd5f814d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd5fd5fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f806c530 0x7fd5f814d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49538/0 (socket says 172.21.15.16:49538) 2023-12-08T08:39:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd5fd5fc640 1 -- 172.21.15.16:0/3354216096 learned_addr learned my addr 172.21.15.16:0/3354216096 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:23.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd5fd5fc640 1 -- 172.21.15.16:0/3354216096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f806b910 msgr2=0x7fd5f81605c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd5fd5fc640 1 --2- 172.21.15.16:0/3354216096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f806b910 0x7fd5f81605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd5fd5fc640 1 -- 172.21.15.16:0/3354216096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f815d480 msgr2=0x7fd5f815f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:23.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd5fe5fe640 1 --2- 172.21.15.16:0/3354216096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f815d480 0x7fd5f815f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd5fd5fc640 1 --2- 172.21.15.16:0/3354216096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f815d480 0x7fd5f815f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:23.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd5fd5fc640 1 -- 172.21.15.16:0/3354216096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f4009400 con 0x7fd5f806c530 2023-12-08T08:39:23.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd5fe5fe640 1 --2- 172.21.15.16:0/3354216096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f815d480 0x7fd5f815f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:23.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd5fd5fc640 1 --2- 172.21.15.16:0/3354216096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f806c530 0x7fd5f814d3f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8000c00 tx=0x7fd5e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:23.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd5ee7fc640 1 -- 172.21.15.16:0/3354216096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e803a8f0 con 0x7fd5f806c530 2023-12-08T08:39:23.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd5ee7fc640 1 -- 172.21.15.16:0/3354216096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e803aa50 con 0x7fd5f806c530 2023-12-08T08:39:23.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd5ee7fc640 1 -- 172.21.15.16:0/3354216096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e803ae00 con 0x7fd5f806c530 2023-12-08T08:39:23.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.917+0000 7fd600061640 1 -- 172.21.15.16:0/3354216096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e8009400 con 0x7fd5f806c530 2023-12-08T08:39:23.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.918+0000 7fd600061640 1 -- 172.21.15.16:0/3354216096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f815ee70 con 0x7fd5f806c530 2023-12-08T08:39:23.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.919+0000 7fd600061640 1 -- 172.21.15.16:0/3354216096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f815f060 con 0x7fd5f806c530 2023-12-08T08:39:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.919+0000 7fd5ee7fc640 1 -- 172.21.15.16:0/3354216096 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd5e804d450 con 0x7fd5f806c530 2023-12-08T08:39:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.919+0000 7fd5ee7fc640 1 --2- 172.21.15.16:0/3354216096 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd5dc06ae40 0x7fd5dc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.920+0000 7fd5ee7fc640 1 -- 172.21.15.16:0/3354216096 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fd5e80abb00 con 0x7fd5f806c530 2023-12-08T08:39:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.920+0000 7fd5fddfd640 1 --2- 172.21.15.16:0/3354216096 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd5dc06ae40 0x7fd5dc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:23.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.920+0000 7fd5fddfd640 1 --2- 172.21.15.16:0/3354216096 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd5dc06ae40 0x7fd5dc06d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4006e80 tx=0x7fd5f4006dd0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:23.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:23.924+0000 7fd5ee7fc640 1 -- 172.21.15.16:0/3354216096 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd5e80ade60 con 0x7fd5f806c530 2023-12-08T08:39:24.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.099+0000 7fd600061640 1 -- 172.21.15.16:0/3354216096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd5f8067430 con 0x7fd5f806c530 2023-12-08T08:39:24.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.099+0000 7fd5ee7fc640 1 -- 172.21.15.16:0/3354216096 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd5e8076730 con 0x7fd5f806c530 2023-12-08T08:39:24.101 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:24.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.101+0000 7fd600061640 1 -- 172.21.15.16:0/3354216096 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd5dc06ae40 msgr2=0x7fd5dc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.101+0000 7fd600061640 1 --2- 172.21.15.16:0/3354216096 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd5dc06ae40 0x7fd5dc06d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4006e80 tx=0x7fd5f4006dd0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.101+0000 7fd600061640 1 -- 172.21.15.16:0/3354216096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f806c530 msgr2=0x7fd5f814d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.101+0000 7fd600061640 1 --2- 172.21.15.16:0/3354216096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f806c530 0x7fd5f814d3f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8000c00 tx=0x7fd5e80046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.101+0000 7fd600061640 1 -- 172.21.15.16:0/3354216096 shutdown_connections 2023-12-08T08:39:24.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.101+0000 7fd600061640 1 --2- 172.21.15.16:0/3354216096 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd5dc06ae40 0x7fd5dc06d300 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.101+0000 7fd600061640 1 --2- 172.21.15.16:0/3354216096 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd5f815d480 0x7fd5f815f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.101+0000 7fd600061640 1 --2- 172.21.15.16:0/3354216096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f806c530 0x7fd5f814d3f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.101+0000 7fd600061640 1 --2- 172.21.15.16:0/3354216096 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd5f806b910 0x7fd5f81605c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.101+0000 7fd600061640 1 -- 172.21.15.16:0/3354216096 >> 172.21.15.16:0/3354216096 conn(0x7fd5f80d0e90 msgr2=0x7fd5f80ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:24.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.101+0000 7fd600061640 1 -- 172.21.15.16:0/3354216096 shutdown_connections 2023-12-08T08:39:24.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.101+0000 7fd600061640 1 -- 172.21.15.16:0/3354216096 wait complete. 2023-12-08T08:39:24.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-08T08:39:24.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.219+0000 7f7b239e3640 1 Processor -- start 2023-12-08T08:39:24.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.220+0000 7f7b239e3640 1 -- start start 2023-12-08T08:39:24.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.220+0000 7f7b239e3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b1c0b1d70 0x7f7b1c0d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.220+0000 7f7b239e3640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1c0d20b0 con 0x7f7b1c1608a0 2023-12-08T08:39:24.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.220+0000 7f7b239e3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b1c0d1b80 con 0x7f7b1c0b0cf0 2023-12-08T08:39:24.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.220+0000 7f7b239e3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1c0d0620 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.221+0000 7f7b21f80640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b1c0b1d70 0x7f7b1c0d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.221+0000 7f7b2177f640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7b1c1608a0 0x7f7b1c0d0f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34358/0 (socket says 172.21.15.16:34358) 2023-12-08T08:39:24.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.221+0000 7f7b2177f640 1 -- 172.21.15.16:0/550772768 learned_addr learned my addr 172.21.15.16:0/550772768 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.221+0000 7f7b21f80640 1 -- 172.21.15.16:0/550772768 >> v1:172.21.15.112:6789/0 conn(0x7f7b1c0b0cf0 legacy=0x7f7b1c0cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.221+0000 7f7b21f80640 1 -- 172.21.15.16:0/550772768 >> v1:172.21.15.16:6789/0 conn(0x7f7b1c1608a0 legacy=0x7f7b1c0d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.221+0000 7f7b13fff640 1 -- 172.21.15.16:0/550772768 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286547726 0 0) 0x7f7b1c0d20b0 con 0x7f7b1c1608a0 2023-12-08T08:39:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.221+0000 7f7b21f80640 1 -- 172.21.15.16:0/550772768 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c0d1dd0 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.222+0000 7f7b21f80640 1 --2- 172.21.15.16:0/550772768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c0b1d70 0x7f7b1c0d1640 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009d90 tx=0x7f7b1402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c813a56d635837df server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.222+0000 7f7b13fff640 1 -- 172.21.15.16:0/550772768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1402cc60 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.222+0000 7f7b13fff640 1 -- 172.21.15.16:0/550772768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1402cdc0 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.222+0000 7f7b13fff640 1 -- 172.21.15.16:0/550772768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b14034660 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.222+0000 7f7b239e3640 1 -- 172.21.15.16:0/550772768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c0b1d70 msgr2=0x7f7b1c0d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.222+0000 7f7b239e3640 1 --2- 172.21.15.16:0/550772768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c0b1d70 0x7f7b1c0d1640 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7b14009d90 tx=0x7f7b1402beb0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.222+0000 7f7b239e3640 1 -- 172.21.15.16:0/550772768 shutdown_connections 2023-12-08T08:39:24.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.222+0000 7f7b239e3640 1 --2- 172.21.15.16:0/550772768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c0b1d70 0x7f7b1c0d1640 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.222+0000 7f7b239e3640 1 -- 172.21.15.16:0/550772768 >> 172.21.15.16:0/550772768 conn(0x7f7b1c0d3030 msgr2=0x7f7b1c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:24.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.223+0000 7f7b239e3640 1 -- 172.21.15.16:0/550772768 shutdown_connections 2023-12-08T08:39:24.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.223+0000 7f7b239e3640 1 -- 172.21.15.16:0/550772768 wait complete. 2023-12-08T08:39:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.224+0000 7f7b239e3640 1 Processor -- start 2023-12-08T08:39:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.224+0000 7f7b239e3640 1 -- start start 2023-12-08T08:39:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.224+0000 7f7b239e3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b1c0b0cf0 0x7f7b1c1512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.224+0000 7f7b239e3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c0b1d70 0x7f7b1c0caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.224+0000 7f7b239e3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b1c1608a0 0x7f7b1c1f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.224+0000 7f7b239e3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c0cb5d0 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.224+0000 7f7b239e3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c0cb8b0 con 0x7f7b1c1608a0 2023-12-08T08:39:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.224+0000 7f7b239e3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c0cbb90 con 0x7f7b1c0b0cf0 2023-12-08T08:39:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b20f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c0b1d70 0x7f7b1c0caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b20f7e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c0b1d70 0x7f7b1c0caae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49560/0 (socket says 172.21.15.16:49560) 2023-12-08T08:39:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b20f7e640 1 -- 172.21.15.16:0/2668067563 learned_addr learned my addr 172.21.15.16:0/2668067563 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b2177f640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b1c0b0cf0 0x7f7b1c1512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b21f80640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b1c1608a0 0x7f7b1c1f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b20f7e640 1 -- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b1c0b0cf0 msgr2=0x7f7b1c1512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b20f7e640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b1c0b0cf0 0x7f7b1c1512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b20f7e640 1 -- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b1c1608a0 msgr2=0x7f7b1c1f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b20f7e640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b1c1608a0 0x7f7b1c1f2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b20f7e640 1 -- 172.21.15.16:0/2668067563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b14009a70 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b21f80640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b1c1608a0 0x7f7b1c1f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b20f7e640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c0b1d70 0x7f7b1c0caae0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00c450 tx=0x7f7b0c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b11ffb640 1 -- 172.21.15.16:0/2668067563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c0052d0 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b11ffb640 1 -- 172.21.15.16:0/2668067563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0c005430 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b239e3640 1 -- 172.21.15.16:0/2668067563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1c0cbe70 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.225+0000 7f7b11ffb640 1 -- 172.21.15.16:0/2668067563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c012870 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.226+0000 7f7b239e3640 1 -- 172.21.15.16:0/2668067563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1c151830 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.227+0000 7f7b239e3640 1 -- 172.21.15.16:0/2668067563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1c1519c0 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.227+0000 7f7b11ffb640 1 -- 172.21.15.16:0/2668067563 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b0c0129d0 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.227+0000 7f7b11ffb640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7b0006ae40 0x7f7b0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.228+0000 7f7b11ffb640 1 -- 172.21.15.16:0/2668067563 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f7b0c089530 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.228+0000 7f7b2177f640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7b0006ae40 0x7f7b0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.228+0000 7f7b2177f640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7b0006ae40 0x7f7b0006d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7b18000f60 tx=0x7f7b18007e40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.232+0000 7f7b11ffb640 1 -- 172.21.15.16:0/2668067563 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b0c054160 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.419+0000 7f7b239e3640 1 -- 172.21.15.16:0/2668067563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7b1c067430 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.420+0000 7f7b11ffb640 1 -- 172.21.15.16:0/2668067563 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7b0c057780 con 0x7f7b1c0b1d70 2023-12-08T08:39:24.422 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-12-08T08:39:24.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.422+0000 7f7b239e3640 1 -- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7b0006ae40 msgr2=0x7f7b0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.422+0000 7f7b239e3640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7b0006ae40 0x7f7b0006d300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7b18000f60 tx=0x7f7b18007e40 comp rx=0 tx=0).stop 2023-12-08T08:39:24.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.422+0000 7f7b239e3640 1 -- 172.21.15.16:0/2668067563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c0b1d70 msgr2=0x7f7b1c0caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.422+0000 7f7b239e3640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c0b1d70 0x7f7b1c0caae0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00c450 tx=0x7f7b0c00c910 comp rx=0 tx=0).stop 2023-12-08T08:39:24.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.422+0000 7f7b239e3640 1 -- 172.21.15.16:0/2668067563 shutdown_connections 2023-12-08T08:39:24.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.422+0000 7f7b239e3640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f7b0006ae40 0x7f7b0006d300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.422+0000 7f7b239e3640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7b1c1608a0 0x7f7b1c1f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.422+0000 7f7b239e3640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b1c0b1d70 0x7f7b1c0caae0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.422+0000 7f7b239e3640 1 --2- 172.21.15.16:0/2668067563 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7b1c0b0cf0 0x7f7b1c1512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.422+0000 7f7b239e3640 1 -- 172.21.15.16:0/2668067563 >> 172.21.15.16:0/2668067563 conn(0x7f7b1c0d3030 msgr2=0x7f7b1c0d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:24.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.422+0000 7f7b239e3640 1 -- 172.21.15.16:0/2668067563 shutdown_connections 2023-12-08T08:39:24.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.422+0000 7f7b239e3640 1 -- 172.21.15.16:0/2668067563 wait complete. 2023-12-08T08:39:24.434 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T08:39:24.434 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T08:39:24.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:24.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.557+0000 7f2d0d8c9640 1 Processor -- start 2023-12-08T08:39:24.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.557+0000 7f2d0d8c9640 1 -- start start 2023-12-08T08:39:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.558+0000 7f2d0d8c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d080d1d00 0x7f2d080d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.558+0000 7f2d0d8c9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2d080b3bf0 0x7f2d080b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.558+0000 7f2d0d8c9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d080cdc20 con 0x7f2d080b3340 2023-12-08T08:39:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.558+0000 7f2d0d8c9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d080d11d0 con 0x7f2d080b3bf0 2023-12-08T08:39:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.558+0000 7f2d0d8c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080d3a60 con 0x7f2d080d1d00 2023-12-08T08:39:24.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.558+0000 7f2d067fc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d080d1d00 0x7f2d080d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.558+0000 7f2d077fe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2d080b3bf0 0x7f2d080b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.558+0000 7f2d06ffd640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2d080b3340 0x7f2d080d15f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:60178/0 (socket says 172.21.15.16:60178) 2023-12-08T08:39:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.558+0000 7f2d06ffd640 1 -- 172.21.15.16:0/3432942170 learned_addr learned my addr 172.21.15.16:0/3432942170 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.559+0000 7f2d067fc640 1 -- 172.21.15.16:0/3432942170 >> v2:172.21.15.112:3300/0 conn(0x7f2d080b3bf0 msgr2=0x7f2d080b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.559+0000 7f2d067fc640 1 --2- 172.21.15.16:0/3432942170 >> v2:172.21.15.112:3300/0 conn(0x7f2d080b3bf0 0x7f2d080b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.559+0000 7f2d067fc640 1 -- 172.21.15.16:0/3432942170 >> v1:172.21.15.112:6789/0 conn(0x7f2d080b3340 legacy=0x7f2d080d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.559+0000 7f2d067fc640 1 -- 172.21.15.16:0/3432942170 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080d3d40 con 0x7f2d080d1d00 2023-12-08T08:39:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.560+0000 7f2d067fc640 1 --2- 172.21.15.16:0/3432942170 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d080d1d00 0x7f2d080d00c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00097a0 tx=0x7f2cf002b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc05548413ce1073 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.560+0000 7f2d05ffb640 1 -- 172.21.15.16:0/3432942170 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf002e070 con 0x7f2d080d1d00 2023-12-08T08:39:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.560+0000 7f2d05ffb640 1 -- 172.21.15.16:0/3432942170 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf00315a0 con 0x7f2d080d1d00 2023-12-08T08:39:24.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.560+0000 7f2d05ffb640 1 -- 172.21.15.16:0/3432942170 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0031930 con 0x7f2d080d1d00 2023-12-08T08:39:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.560+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/3432942170 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d080d1d00 msgr2=0x7f2d080d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.560+0000 7f2d0d8c9640 1 --2- 172.21.15.16:0/3432942170 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d080d1d00 0x7f2d080d00c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2cf00097a0 tx=0x7f2cf002b840 comp rx=0 tx=0).stop 2023-12-08T08:39:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.560+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/3432942170 shutdown_connections 2023-12-08T08:39:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.560+0000 7f2d0d8c9640 1 --2- 172.21.15.16:0/3432942170 >> v2:172.21.15.112:3300/0 conn(0x7f2d080b3bf0 0x7f2d080b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.560+0000 7f2d0d8c9640 1 --2- 172.21.15.16:0/3432942170 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d080d1d00 0x7f2d080d00c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.560+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/3432942170 >> 172.21.15.16:0/3432942170 conn(0x7f2d0809c2f0 msgr2=0x7f2d080d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.560+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/3432942170 shutdown_connections 2023-12-08T08:39:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.560+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/3432942170 wait complete. 2023-12-08T08:39:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.561+0000 7f2d0d8c9640 1 Processor -- start 2023-12-08T08:39:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.562+0000 7f2d0d8c9640 1 -- start start 2023-12-08T08:39:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.562+0000 7f2d0d8c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d080b3340 0x7f2d081570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.562+0000 7f2d0d8c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d080b3bf0 0x7f2d080cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.562+0000 7f2d0d8c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d080d1d00 0x7f2d080ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.562+0000 7f2d0d8c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080b09b0 con 0x7f2d080b3bf0 2023-12-08T08:39:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.562+0000 7f2d0d8c9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d081575e0 con 0x7f2d080b3340 2023-12-08T08:39:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.562+0000 7f2d0d8c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080ccc20 con 0x7f2d080d1d00 2023-12-08T08:39:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.562+0000 7f2d067fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d080b3bf0 0x7f2d080cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.562+0000 7f2d067fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d080b3bf0 0x7f2d080cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49576/0 (socket says 172.21.15.16:49576) 2023-12-08T08:39:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.562+0000 7f2d067fc640 1 -- 172.21.15.16:0/1945909243 learned_addr learned my addr 172.21.15.16:0/1945909243 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.563+0000 7f2d06ffd640 1 --2- 172.21.15.16:0/1945909243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d080b3340 0x7f2d081570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.563+0000 7f2d077fe640 1 --2- 172.21.15.16:0/1945909243 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d080d1d00 0x7f2d080ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.563+0000 7f2d067fc640 1 -- 172.21.15.16:0/1945909243 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d080d1d00 msgr2=0x7f2d080ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.563+0000 7f2d067fc640 1 --2- 172.21.15.16:0/1945909243 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d080d1d00 0x7f2d080ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.563+0000 7f2d067fc640 1 -- 172.21.15.16:0/1945909243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d080b3340 msgr2=0x7f2d081570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.563+0000 7f2d067fc640 1 --2- 172.21.15.16:0/1945909243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d080b3340 0x7f2d081570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.563+0000 7f2d067fc640 1 -- 172.21.15.16:0/1945909243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf0009400 con 0x7f2d080b3bf0 2023-12-08T08:39:24.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.563+0000 7f2d067fc640 1 --2- 172.21.15.16:0/1945909243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d080b3bf0 0x7f2d080cc1b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0000c00 tx=0x7f2cf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.563+0000 7f2cfffff640 1 -- 172.21.15.16:0/1945909243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0031f10 con 0x7f2d080b3bf0 2023-12-08T08:39:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.563+0000 7f2cfffff640 1 -- 172.21.15.16:0/1945909243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf0032860 con 0x7f2d080b3bf0 2023-12-08T08:39:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.563+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/1945909243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d080ccf00 con 0x7f2d080b3bf0 2023-12-08T08:39:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.563+0000 7f2cfffff640 1 -- 172.21.15.16:0/1945909243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0032c10 con 0x7f2d080b3bf0 2023-12-08T08:39:24.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.563+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/1945909243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d080b0640 con 0x7f2d080b3bf0 2023-12-08T08:39:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.564+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/1945909243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d080b0b60 con 0x7f2d080b3bf0 2023-12-08T08:39:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.565+0000 7f2cfffff640 1 -- 172.21.15.16:0/1945909243 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2cf004d3f0 con 0x7f2d080b3bf0 2023-12-08T08:39:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.565+0000 7f2cfffff640 1 --2- 172.21.15.16:0/1945909243 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2ce806ae40 0x7f2ce806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.565+0000 7f2cfffff640 1 -- 172.21.15.16:0/1945909243 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f2cf00ab4b0 con 0x7f2d080b3bf0 2023-12-08T08:39:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.565+0000 7f2d06ffd640 1 --2- 172.21.15.16:0/1945909243 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2ce806ae40 0x7f2ce806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.566+0000 7f2d06ffd640 1 --2- 172.21.15.16:0/1945909243 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2ce806ae40 0x7f2ce806d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2d00000f60 tx=0x7f2d00009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:24.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.569+0000 7f2cfffff640 1 -- 172.21.15.16:0/1945909243 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d080b0b60 con 0x7f2d080b3bf0 2023-12-08T08:39:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.751+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/1945909243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d080b0b60 con 0x7f2d080b3bf0 2023-12-08T08:39:24.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.752+0000 7f2cfffff640 1 -- 172.21.15.16:0/1945909243 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7f2d080b0b60 con 0x7f2d080b3bf0 2023-12-08T08:39:24.754 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:24.754 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:24.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.754+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/1945909243 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2ce806ae40 msgr2=0x7f2ce806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.754+0000 7f2d0d8c9640 1 --2- 172.21.15.16:0/1945909243 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2ce806ae40 0x7f2ce806d300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2d00000f60 tx=0x7f2d00009040 comp rx=0 tx=0).stop 2023-12-08T08:39:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.754+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/1945909243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d080b3bf0 msgr2=0x7f2d080cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.754+0000 7f2d0d8c9640 1 --2- 172.21.15.16:0/1945909243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d080b3bf0 0x7f2d080cc1b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0000c00 tx=0x7f2cf00046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.754+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/1945909243 shutdown_connections 2023-12-08T08:39:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.754+0000 7f2d0d8c9640 1 --2- 172.21.15.16:0/1945909243 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2ce806ae40 0x7f2ce806d300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.754+0000 7f2d0d8c9640 1 --2- 172.21.15.16:0/1945909243 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2d080d1d00 0x7f2d080ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.754+0000 7f2d0d8c9640 1 --2- 172.21.15.16:0/1945909243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d080b3bf0 0x7f2d080cc1b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.754+0000 7f2d0d8c9640 1 --2- 172.21.15.16:0/1945909243 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2d080b3340 0x7f2d081570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.754+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/1945909243 >> 172.21.15.16:0/1945909243 conn(0x7f2d0809c2f0 msgr2=0x7f2d080d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.754+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/1945909243 shutdown_connections 2023-12-08T08:39:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.754+0000 7f2d0d8c9640 1 -- 172.21.15.16:0/1945909243 wait complete. 2023-12-08T08:39:24.767 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:39:24.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:24.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.876+0000 7f0d542e0640 1 Processor -- start 2023-12-08T08:39:24.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.876+0000 7f0d542e0640 1 -- start start 2023-12-08T08:39:24.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.877+0000 7f0d542e0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d4c0cfa30 0x7f0d4c0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.877+0000 7f0d542e0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d4c0b4150 0x7f0d4c0b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.877+0000 7f0d542e0640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d4c0c6be0 con 0x7f0d4c0cfe10 2023-12-08T08:39:24.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.877+0000 7f0d542e0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d4c0d3ff0 con 0x7f0d4c0cfa30 2023-12-08T08:39:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.877+0000 7f0d542e0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0b4860 con 0x7f0d4c0b4150 2023-12-08T08:39:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.877+0000 7f0d5207c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d4c0cfa30 0x7f0d4c0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.877+0000 7f0d5287d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0d4c0cfe10 0x7f0d4c161260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34362/0 (socket says 172.21.15.16:34362) 2023-12-08T08:39:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.877+0000 7f0d5287d640 1 -- 172.21.15.16:0/4196754838 learned_addr learned my addr 172.21.15.16:0/4196754838 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.877+0000 7f0d5187b640 1 --2- 172.21.15.16:0/4196754838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d4c0b4150 0x7f0d4c0b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.877+0000 7f0d5207c640 1 -- 172.21.15.16:0/4196754838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d4c0b4150 msgr2=0x7f0d4c0b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.877+0000 7f0d5207c640 1 --2- 172.21.15.16:0/4196754838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d4c0b4150 0x7f0d4c0b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.877+0000 7f0d5207c640 1 -- 172.21.15.16:0/4196754838 >> v1:172.21.15.16:6789/0 conn(0x7f0d4c0cfe10 legacy=0x7f0d4c161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.877+0000 7f0d5207c640 1 -- 172.21.15.16:0/4196754838 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c0d3d50 con 0x7f0d4c0cfa30 2023-12-08T08:39:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.877+0000 7f0d5107a640 1 -- 172.21.15.16:0/4196754838 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909359900 0 0) 0x7f0d4c0c6be0 con 0x7f0d4c0cfe10 2023-12-08T08:39:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.878+0000 7f0d5207c640 1 --2- 172.21.15.16:0/4196754838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4c0cfa30 0x7f0d4c0b2f20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009d90 tx=0x7f0d3c02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae45cad296f9184c server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.878+0000 7f0d5107a640 1 -- 172.21.15.16:0/4196754838 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c02cbc0 con 0x7f0d4c0cfa30 2023-12-08T08:39:24.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.878+0000 7f0d5107a640 1 -- 172.21.15.16:0/4196754838 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d3c02cd20 con 0x7f0d4c0cfa30 2023-12-08T08:39:24.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.878+0000 7f0d5107a640 1 -- 172.21.15.16:0/4196754838 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c034660 con 0x7f0d4c0cfa30 2023-12-08T08:39:24.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.879+0000 7f0d542e0640 1 -- 172.21.15.16:0/4196754838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4c0cfa30 msgr2=0x7f0d4c0b2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.879+0000 7f0d542e0640 1 --2- 172.21.15.16:0/4196754838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4c0cfa30 0x7f0d4c0b2f20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009d90 tx=0x7f0d3c02be10 comp rx=0 tx=0).stop 2023-12-08T08:39:24.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.879+0000 7f0d542e0640 1 -- 172.21.15.16:0/4196754838 shutdown_connections 2023-12-08T08:39:24.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.879+0000 7f0d542e0640 1 --2- 172.21.15.16:0/4196754838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d4c0b4150 0x7f0d4c0b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.879+0000 7f0d542e0640 1 --2- 172.21.15.16:0/4196754838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4c0cfa30 0x7f0d4c0b2f20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.879+0000 7f0d542e0640 1 -- 172.21.15.16:0/4196754838 >> 172.21.15.16:0/4196754838 conn(0x7f0d4c0d3090 msgr2=0x7f0d4c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.879+0000 7f0d542e0640 1 -- 172.21.15.16:0/4196754838 shutdown_connections 2023-12-08T08:39:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.879+0000 7f0d542e0640 1 -- 172.21.15.16:0/4196754838 wait complete. 2023-12-08T08:39:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.880+0000 7f0d542e0640 1 Processor -- start 2023-12-08T08:39:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.880+0000 7f0d542e0640 1 -- start start 2023-12-08T08:39:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d542e0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d4c0b4150 0x7f0d4c0ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d542e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d4c0cfa30 0x7f0d4c0b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d542e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4c0cfe10 0x7f0d4c0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d542e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c153b10 con 0x7f0d4c0cfe10 2023-12-08T08:39:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d542e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0cd0f0 con 0x7f0d4c0cfa30 2023-12-08T08:39:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d542e0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0cafb0 con 0x7f0d4c0b4150 2023-12-08T08:39:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d5287d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4c0cfe10 0x7f0d4c0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d5287d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4c0cfe10 0x7f0d4c0b61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49600/0 (socket says 172.21.15.16:49600) 2023-12-08T08:39:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d5287d640 1 -- 172.21.15.16:0/3115718144 learned_addr learned my addr 172.21.15.16:0/3115718144 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d5207c640 1 --2- 172.21.15.16:0/3115718144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d4c0b4150 0x7f0d4c0ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d5287d640 1 -- 172.21.15.16:0/3115718144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d4c0b4150 msgr2=0x7f0d4c0ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d5287d640 1 --2- 172.21.15.16:0/3115718144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d4c0b4150 0x7f0d4c0ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d5287d640 1 -- 172.21.15.16:0/3115718144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d4c0cfa30 msgr2=0x7f0d4c0b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d5187b640 1 --2- 172.21.15.16:0/3115718144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d4c0cfa30 0x7f0d4c0b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d5287d640 1 --2- 172.21.15.16:0/3115718144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d4c0cfa30 0x7f0d4c0b5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d5287d640 1 -- 172.21.15.16:0/3115718144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3c009a70 con 0x7f0d4c0cfe10 2023-12-08T08:39:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.881+0000 7f0d5287d640 1 --2- 172.21.15.16:0/3115718144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4c0cfe10 0x7f0d4c0b61c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400c450 tx=0x7f0d4400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.882+0000 7f0d4affd640 1 -- 172.21.15.16:0/3115718144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d440074e0 con 0x7f0d4c0cfe10 2023-12-08T08:39:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.882+0000 7f0d4affd640 1 -- 172.21.15.16:0/3115718144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d44007660 con 0x7f0d4c0cfe10 2023-12-08T08:39:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.882+0000 7f0d542e0640 1 -- 172.21.15.16:0/3115718144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4c0cb290 con 0x7f0d4c0cfe10 2023-12-08T08:39:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.882+0000 7f0d4affd640 1 -- 172.21.15.16:0/3115718144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d44004a20 con 0x7f0d4c0cfe10 2023-12-08T08:39:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.882+0000 7f0d542e0640 1 -- 172.21.15.16:0/3115718144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4c0cb850 con 0x7f0d4c0cfe10 2023-12-08T08:39:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.883+0000 7f0d542e0640 1 -- 172.21.15.16:0/3115718144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4c153c80 con 0x7f0d4c0cfe10 2023-12-08T08:39:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.883+0000 7f0d4affd640 1 -- 172.21.15.16:0/3115718144 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0d44004b80 con 0x7f0d4c0cfe10 2023-12-08T08:39:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.883+0000 7f0d4affd640 1 --2- 172.21.15.16:0/3115718144 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0d3806ae40 0x7f0d3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.884+0000 7f0d4affd640 1 -- 172.21.15.16:0/3115718144 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f0d44088f40 con 0x7f0d4c0cfe10 2023-12-08T08:39:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.884+0000 7f0d5207c640 1 --2- 172.21.15.16:0/3115718144 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0d3806ae40 0x7f0d3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.884+0000 7f0d5207c640 1 --2- 172.21.15.16:0/3115718144 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0d3806ae40 0x7f0d3806d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c036000 tx=0x7f0d3c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:24.888+0000 7f0d4affd640 1 -- 172.21.15.16:0/3115718144 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d44053cd0 con 0x7f0d4c0cfe10 2023-12-08T08:39:25.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.062+0000 7f0d542e0640 1 -- 172.21.15.16:0/3115718144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d4c067430 con 0x7f0d4c0cfe10 2023-12-08T08:39:25.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.062+0000 7f0d4affd640 1 -- 172.21.15.16:0/3115718144 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d44018020 con 0x7f0d4c0cfe10 2023-12-08T08:39:25.065 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:25.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.064+0000 7f0d542e0640 1 -- 172.21.15.16:0/3115718144 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0d3806ae40 msgr2=0x7f0d3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.064+0000 7f0d542e0640 1 --2- 172.21.15.16:0/3115718144 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0d3806ae40 0x7f0d3806d300 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c036000 tx=0x7f0d3c030000 comp rx=0 tx=0).stop 2023-12-08T08:39:25.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.064+0000 7f0d542e0640 1 -- 172.21.15.16:0/3115718144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4c0cfe10 msgr2=0x7f0d4c0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.064+0000 7f0d542e0640 1 --2- 172.21.15.16:0/3115718144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4c0cfe10 0x7f0d4c0b61c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400c450 tx=0x7f0d4400c910 comp rx=0 tx=0).stop 2023-12-08T08:39:25.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.064+0000 7f0d542e0640 1 -- 172.21.15.16:0/3115718144 shutdown_connections 2023-12-08T08:39:25.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.064+0000 7f0d542e0640 1 --2- 172.21.15.16:0/3115718144 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0d3806ae40 0x7f0d3806d300 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.064+0000 7f0d542e0640 1 --2- 172.21.15.16:0/3115718144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d4c0cfe10 0x7f0d4c0b61c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.064+0000 7f0d542e0640 1 --2- 172.21.15.16:0/3115718144 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0d4c0cfa30 0x7f0d4c0b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.064+0000 7f0d542e0640 1 --2- 172.21.15.16:0/3115718144 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0d4c0b4150 0x7f0d4c0ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.064+0000 7f0d542e0640 1 -- 172.21.15.16:0/3115718144 >> 172.21.15.16:0/3115718144 conn(0x7f0d4c0d3090 msgr2=0x7f0d4c15b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:25.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.064+0000 7f0d542e0640 1 -- 172.21.15.16:0/3115718144 shutdown_connections 2023-12-08T08:39:25.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.064+0000 7f0d542e0640 1 -- 172.21.15.16:0/3115718144 wait complete. 2023-12-08T08:39:25.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:39:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.186+0000 7fbd97933640 1 Processor -- start 2023-12-08T08:39:25.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.186+0000 7fbd97933640 1 -- start start 2023-12-08T08:39:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.187+0000 7fbd97933640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd9006a4d0 0x7fbd9006a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.187+0000 7fbd97933640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbd9006d1e0 0x7fbd9006d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.187+0000 7fbd97933640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd900acf10 con 0x7fbd90071250 2023-12-08T08:39:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.187+0000 7fbd97933640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9015b220 con 0x7fbd9006d1e0 2023-12-08T08:39:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.187+0000 7fbd97933640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9006adf0 con 0x7fbd9006a4d0 2023-12-08T08:39:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.187+0000 7fbd94ece640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbd9006d1e0 0x7fbd9006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.187+0000 7fbd95ed0640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbd90071250 0x7fbd900716e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34378/0 (socket says 172.21.15.16:34378) 2023-12-08T08:39:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.187+0000 7fbd95ed0640 1 -- 172.21.15.16:0/1509863067 learned_addr learned my addr 172.21.15.16:0/1509863067 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.188+0000 7fbd956cf640 1 --2- 172.21.15.16:0/1509863067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd9006a4d0 0x7fbd9006a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.188+0000 7fbd83fff640 1 -- 172.21.15.16:0/1509863067 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209027419 0 0) 0x7fbd900acf10 con 0x7fbd90071250 2023-12-08T08:39:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.188+0000 7fbd83fff640 1 -- 172.21.15.16:0/1509863067 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd7c003520 con 0x7fbd90071250 2023-12-08T08:39:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.188+0000 7fbd94ece640 1 -- 172.21.15.16:0/1509863067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd9006a4d0 msgr2=0x7fbd9006a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.188+0000 7fbd94ece640 1 --2- 172.21.15.16:0/1509863067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd9006a4d0 0x7fbd9006a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.188+0000 7fbd94ece640 1 -- 172.21.15.16:0/1509863067 >> v1:172.21.15.16:6789/0 conn(0x7fbd90071250 legacy=0x7fbd900716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.188+0000 7fbd94ece640 1 -- 172.21.15.16:0/1509863067 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9006dba0 con 0x7fbd9006d1e0 2023-12-08T08:39:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.189+0000 7fbd94ece640 1 --2- 172.21.15.16:0/1509863067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9006d1e0 0x7fbd9006d660 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0097a0 tx=0x7fbd8c02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edecf6ca88ccbc39 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.189+0000 7fbd83fff640 1 -- 172.21.15.16:0/1509863067 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c02f050 con 0x7fbd9006d1e0 2023-12-08T08:39:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.189+0000 7fbd83fff640 1 -- 172.21.15.16:0/1509863067 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c031700 con 0x7fbd9006d1e0 2023-12-08T08:39:25.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.189+0000 7fbd83fff640 1 -- 172.21.15.16:0/1509863067 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c031a90 con 0x7fbd9006d1e0 2023-12-08T08:39:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.189+0000 7fbd97933640 1 -- 172.21.15.16:0/1509863067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9006d1e0 msgr2=0x7fbd9006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.189+0000 7fbd97933640 1 --2- 172.21.15.16:0/1509863067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9006d1e0 0x7fbd9006d660 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0097a0 tx=0x7fbd8c02b7a0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.189+0000 7fbd97933640 1 -- 172.21.15.16:0/1509863067 shutdown_connections 2023-12-08T08:39:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.189+0000 7fbd97933640 1 --2- 172.21.15.16:0/1509863067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9006d1e0 0x7fbd9006d660 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.189+0000 7fbd97933640 1 --2- 172.21.15.16:0/1509863067 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd9006a4d0 0x7fbd9006a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.189+0000 7fbd97933640 1 -- 172.21.15.16:0/1509863067 >> 172.21.15.16:0/1509863067 conn(0x7fbd900b1a70 msgr2=0x7fbd900b1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.190+0000 7fbd97933640 1 -- 172.21.15.16:0/1509863067 shutdown_connections 2023-12-08T08:39:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.190+0000 7fbd97933640 1 -- 172.21.15.16:0/1509863067 wait complete. 2023-12-08T08:39:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.191+0000 7fbd97933640 1 Processor -- start 2023-12-08T08:39:25.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.191+0000 7fbd97933640 1 -- start start 2023-12-08T08:39:25.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.191+0000 7fbd97933640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd9006a4d0 0x7fbd900c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.191+0000 7fbd97933640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9006d1e0 0x7fbd900cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.191+0000 7fbd97933640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd90071250 0x7fbd900cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.191+0000 7fbd97933640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900c4d80 con 0x7fbd9006d1e0 2023-12-08T08:39:25.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.191+0000 7fbd97933640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9014e3f0 con 0x7fbd90071250 2023-12-08T08:39:25.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.191+0000 7fbd97933640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900c5020 con 0x7fbd9006a4d0 2023-12-08T08:39:25.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd94ece640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9006d1e0 0x7fbd900cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd94ece640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9006d1e0 0x7fbd900cee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49620/0 (socket says 172.21.15.16:49620) 2023-12-08T08:39:25.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd94ece640 1 -- 172.21.15.16:0/3601940203 learned_addr learned my addr 172.21.15.16:0/3601940203 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:25.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd956cf640 1 --2- 172.21.15.16:0/3601940203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd9006a4d0 0x7fbd900c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd95ed0640 1 --2- 172.21.15.16:0/3601940203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd90071250 0x7fbd900cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd94ece640 1 -- 172.21.15.16:0/3601940203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd9006a4d0 msgr2=0x7fbd900c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd94ece640 1 --2- 172.21.15.16:0/3601940203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd9006a4d0 0x7fbd900c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd94ece640 1 -- 172.21.15.16:0/3601940203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd90071250 msgr2=0x7fbd900cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd94ece640 1 --2- 172.21.15.16:0/3601940203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd90071250 0x7fbd900cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd94ece640 1 -- 172.21.15.16:0/3601940203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c009400 con 0x7fbd9006d1e0 2023-12-08T08:39:25.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd94ece640 1 --2- 172.21.15.16:0/3601940203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9006d1e0 0x7fbd900cee10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0097a0 tx=0x7fbd8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:25.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd81ffb640 1 -- 172.21.15.16:0/3601940203 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c0339b0 con 0x7fbd9006d1e0 2023-12-08T08:39:25.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd81ffb640 1 -- 172.21.15.16:0/3601940203 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c033b10 con 0x7fbd9006d1e0 2023-12-08T08:39:25.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd97933640 1 -- 172.21.15.16:0/3601940203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd900c5300 con 0x7fbd9006d1e0 2023-12-08T08:39:25.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.192+0000 7fbd81ffb640 1 -- 172.21.15.16:0/3601940203 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c03d470 con 0x7fbd9006d1e0 2023-12-08T08:39:25.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.193+0000 7fbd97933640 1 -- 172.21.15.16:0/3601940203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd900cfbc0 con 0x7fbd9006d1e0 2023-12-08T08:39:25.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.194+0000 7fbd97933640 1 -- 172.21.15.16:0/3601940203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd90067ad0 con 0x7fbd9006d1e0 2023-12-08T08:39:25.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.194+0000 7fbd81ffb640 1 -- 172.21.15.16:0/3601940203 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd8c03a040 con 0x7fbd9006d1e0 2023-12-08T08:39:25.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.195+0000 7fbd81ffb640 1 --2- 172.21.15.16:0/3601940203 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbd7406ae40 0x7fbd7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.195+0000 7fbd81ffb640 1 -- 172.21.15.16:0/3601940203 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fbd8c0ab460 con 0x7fbd9006d1e0 2023-12-08T08:39:25.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.195+0000 7fbd956cf640 1 --2- 172.21.15.16:0/3601940203 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbd7406ae40 0x7fbd7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.196+0000 7fbd956cf640 1 --2- 172.21.15.16:0/3601940203 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbd7406ae40 0x7fbd7406d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009aa0 tx=0x7fbd84009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:25.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.200+0000 7fbd81ffb640 1 -- 172.21.15.16:0/3601940203 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd8c076270 con 0x7fbd9006d1e0 2023-12-08T08:39:25.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.376+0000 7fbd97933640 1 -- 172.21.15.16:0/3601940203 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbd900cdca0 con 0x7fbd9006d1e0 2023-12-08T08:39:25.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.376+0000 7fbd81ffb640 1 -- 172.21.15.16:0/3601940203 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbd8c079890 con 0x7fbd9006d1e0 2023-12-08T08:39:25.379 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:25.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.378+0000 7fbd97933640 1 -- 172.21.15.16:0/3601940203 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbd7406ae40 msgr2=0x7fbd7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.378+0000 7fbd97933640 1 --2- 172.21.15.16:0/3601940203 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbd7406ae40 0x7fbd7406d300 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009aa0 tx=0x7fbd84009190 comp rx=0 tx=0).stop 2023-12-08T08:39:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.378+0000 7fbd97933640 1 -- 172.21.15.16:0/3601940203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9006d1e0 msgr2=0x7fbd900cee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.378+0000 7fbd97933640 1 --2- 172.21.15.16:0/3601940203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9006d1e0 0x7fbd900cee10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c0097a0 tx=0x7fbd8c0046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.378+0000 7fbd97933640 1 -- 172.21.15.16:0/3601940203 shutdown_connections 2023-12-08T08:39:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.378+0000 7fbd97933640 1 --2- 172.21.15.16:0/3601940203 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbd7406ae40 0x7fbd7406d300 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.378+0000 7fbd97933640 1 --2- 172.21.15.16:0/3601940203 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbd90071250 0x7fbd900cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.378+0000 7fbd97933640 1 --2- 172.21.15.16:0/3601940203 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd9006d1e0 0x7fbd900cee10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.378+0000 7fbd97933640 1 --2- 172.21.15.16:0/3601940203 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbd9006a4d0 0x7fbd900c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.378+0000 7fbd97933640 1 -- 172.21.15.16:0/3601940203 >> 172.21.15.16:0/3601940203 conn(0x7fbd900b1a70 msgr2=0x7fbd900af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.379+0000 7fbd97933640 1 -- 172.21.15.16:0/3601940203 shutdown_connections 2023-12-08T08:39:25.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.379+0000 7fbd97933640 1 -- 172.21.15.16:0/3601940203 wait complete. 2023-12-08T08:39:25.392 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:39:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.502+0000 7f5b477b6640 1 Processor -- start 2023-12-08T08:39:25.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.502+0000 7f5b477b6640 1 -- start start 2023-12-08T08:39:25.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.503+0000 7f5b477b6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b400adf70 0x7f5b400ae350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.503+0000 7f5b477b6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b400ae890 con 0x7f5b4006ec30 2023-12-08T08:39:25.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.503+0000 7f5b477b6640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b400b4a20 con 0x7f5b4006f090 2023-12-08T08:39:25.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.503+0000 7f5b477b6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b4006db90 con 0x7f5b400adf70 2023-12-08T08:39:25.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.503+0000 7f5b45552640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b400adf70 0x7f5b400ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.503+0000 7f5b44d51640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5b4006ec30 0x7f5b400ad290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34394/0 (socket says 172.21.15.16:34394) 2023-12-08T08:39:25.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.503+0000 7f5b44d51640 1 -- 172.21.15.16:0/3069490398 learned_addr learned my addr 172.21.15.16:0/3069490398 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:25.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.503+0000 7f5b45552640 1 -- 172.21.15.16:0/3069490398 >> v1:172.21.15.112:6789/0 conn(0x7f5b4006f090 legacy=0x7f5b400b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:25.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.503+0000 7f5b45552640 1 -- 172.21.15.16:0/3069490398 >> v1:172.21.15.16:6789/0 conn(0x7f5b4006ec30 legacy=0x7f5b400ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.503+0000 7f5b45552640 1 -- 172.21.15.16:0/3069490398 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4006de70 con 0x7f5b400adf70 2023-12-08T08:39:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.504+0000 7f5b45552640 1 --2- 172.21.15.16:0/3069490398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b400adf70 0x7f5b400ae350 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0097d0 tx=0x7f5b3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7db2952cb9eabc server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.505+0000 7f5b37fff640 1 -- 172.21.15.16:0/3069490398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c031830 con 0x7f5b400adf70 2023-12-08T08:39:25.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.505+0000 7f5b37fff640 1 -- 172.21.15.16:0/3069490398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3c031990 con 0x7f5b400adf70 2023-12-08T08:39:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.505+0000 7f5b37fff640 1 -- 172.21.15.16:0/3069490398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c031d20 con 0x7f5b400adf70 2023-12-08T08:39:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.505+0000 7f5b477b6640 1 -- 172.21.15.16:0/3069490398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b400adf70 msgr2=0x7f5b400ae350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.505+0000 7f5b477b6640 1 --2- 172.21.15.16:0/3069490398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b400adf70 0x7f5b400ae350 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c0097d0 tx=0x7f5b3c02b8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.505+0000 7f5b477b6640 1 -- 172.21.15.16:0/3069490398 shutdown_connections 2023-12-08T08:39:25.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.505+0000 7f5b477b6640 1 --2- 172.21.15.16:0/3069490398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b400adf70 0x7f5b400ae350 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.505+0000 7f5b477b6640 1 -- 172.21.15.16:0/3069490398 >> 172.21.15.16:0/3069490398 conn(0x7f5b40071850 msgr2=0x7f5b4006a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.505+0000 7f5b477b6640 1 -- 172.21.15.16:0/3069490398 shutdown_connections 2023-12-08T08:39:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.505+0000 7f5b477b6640 1 -- 172.21.15.16:0/3069490398 wait complete. 2023-12-08T08:39:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.507+0000 7f5b477b6640 1 Processor -- start 2023-12-08T08:39:25.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.507+0000 7f5b477b6640 1 -- start start 2023-12-08T08:39:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.507+0000 7f5b477b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b4006ec30 0x7f5b4015b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.507+0000 7f5b477b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b4006f090 0x7f5b4015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.507+0000 7f5b477b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b400adf70 0x7f5b401425d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.507+0000 7f5b477b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b40154e90 con 0x7f5b400adf70 2023-12-08T08:39:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.507+0000 7f5b477b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b40154280 con 0x7f5b4006f090 2023-12-08T08:39:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.507+0000 7f5b477b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b40154560 con 0x7f5b4006ec30 2023-12-08T08:39:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b45d53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b400adf70 0x7f5b401425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b45d53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b400adf70 0x7f5b401425d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49636/0 (socket says 172.21.15.16:49636) 2023-12-08T08:39:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b45d53640 1 -- 172.21.15.16:0/3911149567 learned_addr learned my addr 172.21.15.16:0/3911149567 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b45552640 1 --2- 172.21.15.16:0/3911149567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b4006ec30 0x7f5b4015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b44d51640 1 --2- 172.21.15.16:0/3911149567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b4006f090 0x7f5b4015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b45d53640 1 -- 172.21.15.16:0/3911149567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b4006ec30 msgr2=0x7f5b4015b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b45d53640 1 --2- 172.21.15.16:0/3911149567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b4006ec30 0x7f5b4015b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b45d53640 1 -- 172.21.15.16:0/3911149567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b4006f090 msgr2=0x7f5b4015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b45d53640 1 --2- 172.21.15.16:0/3911149567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b4006f090 0x7f5b4015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b45d53640 1 -- 172.21.15.16:0/3911149567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3c009400 con 0x7f5b400adf70 2023-12-08T08:39:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b45d53640 1 --2- 172.21.15.16:0/3911149567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b400adf70 0x7f5b401425d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800b970 tx=0x7f5b3800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b35ffb640 1 -- 172.21.15.16:0/3911149567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3800cb40 con 0x7f5b400adf70 2023-12-08T08:39:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b35ffb640 1 -- 172.21.15.16:0/3911149567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3800cca0 con 0x7f5b400adf70 2023-12-08T08:39:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b35ffb640 1 -- 172.21.15.16:0/3911149567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38014680 con 0x7f5b400adf70 2023-12-08T08:39:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.508+0000 7f5b477b6640 1 -- 172.21.15.16:0/3911149567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b40154840 con 0x7f5b400adf70 2023-12-08T08:39:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.509+0000 7f5b477b6640 1 -- 172.21.15.16:0/3911149567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b40153f50 con 0x7f5b400adf70 2023-12-08T08:39:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.510+0000 7f5b477b6640 1 -- 172.21.15.16:0/3911149567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b40067ad0 con 0x7f5b400adf70 2023-12-08T08:39:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.511+0000 7f5b35ffb640 1 -- 172.21.15.16:0/3911149567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5b3801d3b0 con 0x7f5b400adf70 2023-12-08T08:39:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.511+0000 7f5b35ffb640 1 --2- 172.21.15.16:0/3911149567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5b2406ae40 0x7f5b2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.511+0000 7f5b35ffb640 1 -- 172.21.15.16:0/3911149567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f5b38089780 con 0x7f5b400adf70 2023-12-08T08:39:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.511+0000 7f5b45552640 1 --2- 172.21.15.16:0/3911149567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5b2406ae40 0x7f5b2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.512+0000 7f5b45552640 1 --2- 172.21.15.16:0/3911149567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5b2406ae40 0x7f5b2406d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c036000 tx=0x7f5b3c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.516+0000 7f5b35ffb640 1 -- 172.21.15.16:0/3911149567 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5b38054500 con 0x7f5b400adf70 2023-12-08T08:39:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.694+0000 7f5b477b6640 1 -- 172.21.15.16:0/3911149567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b400b30a0 con 0x7f5b400adf70 2023-12-08T08:39:25.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.695+0000 7f5b35ffb640 1 -- 172.21.15.16:0/3911149567 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5b38057b20 con 0x7f5b400adf70 2023-12-08T08:39:25.697 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:39:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.697+0000 7f5b477b6640 1 -- 172.21.15.16:0/3911149567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5b2406ae40 msgr2=0x7f5b2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.697+0000 7f5b477b6640 1 --2- 172.21.15.16:0/3911149567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5b2406ae40 0x7f5b2406d300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c036000 tx=0x7f5b3c03c040 comp rx=0 tx=0).stop 2023-12-08T08:39:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.697+0000 7f5b477b6640 1 -- 172.21.15.16:0/3911149567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b400adf70 msgr2=0x7f5b401425d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.697+0000 7f5b477b6640 1 --2- 172.21.15.16:0/3911149567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b400adf70 0x7f5b401425d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800b970 tx=0x7f5b3800be30 comp rx=0 tx=0).stop 2023-12-08T08:39:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.697+0000 7f5b477b6640 1 -- 172.21.15.16:0/3911149567 shutdown_connections 2023-12-08T08:39:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.697+0000 7f5b477b6640 1 --2- 172.21.15.16:0/3911149567 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5b2406ae40 0x7f5b2406d300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.697+0000 7f5b477b6640 1 --2- 172.21.15.16:0/3911149567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b400adf70 0x7f5b401425d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.697+0000 7f5b477b6640 1 --2- 172.21.15.16:0/3911149567 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5b4006f090 0x7f5b4015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.697+0000 7f5b477b6640 1 --2- 172.21.15.16:0/3911149567 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5b4006ec30 0x7f5b4015b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.697+0000 7f5b477b6640 1 -- 172.21.15.16:0/3911149567 >> 172.21.15.16:0/3911149567 conn(0x7f5b40071850 msgr2=0x7f5b40153320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.697+0000 7f5b477b6640 1 -- 172.21.15.16:0/3911149567 shutdown_connections 2023-12-08T08:39:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.697+0000 7f5b477b6640 1 -- 172.21.15.16:0/3911149567 wait complete. 2023-12-08T08:39:25.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:39:25.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.818+0000 7f00334bb640 1 Processor -- start 2023-12-08T08:39:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.819+0000 7f00334bb640 1 -- start start 2023-12-08T08:39:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.819+0000 7f00334bb640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f002c0cfe10 0x7f002c0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.819+0000 7f00334bb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f002c0c6be0 con 0x7f002c0b4150 2023-12-08T08:39:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.819+0000 7f00334bb640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f002c0d3ff0 con 0x7f002c0cfa30 2023-12-08T08:39:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.819+0000 7f00334bb640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f002c0b4860 con 0x7f002c0cfe10 2023-12-08T08:39:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.819+0000 7f0031a58640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f002c0cfe10 0x7f002c0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.819+0000 7f0030a56640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f002c0b4150 0x7f002c0cdf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34400/0 (socket says 172.21.15.16:34400) 2023-12-08T08:39:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.820+0000 7f0030a56640 1 -- 172.21.15.16:0/2267863012 learned_addr learned my addr 172.21.15.16:0/2267863012 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.820+0000 7f0031a58640 1 -- 172.21.15.16:0/2267863012 >> v1:172.21.15.112:6789/0 conn(0x7f002c0cfa30 legacy=0x7f002c0b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:25.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.820+0000 7f0031a58640 1 -- 172.21.15.16:0/2267863012 >> v1:172.21.15.16:6789/0 conn(0x7f002c0b4150 legacy=0x7f002c0cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.820+0000 7f0031a58640 1 -- 172.21.15.16:0/2267863012 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c0d3d50 con 0x7f002c0cfe10 2023-12-08T08:39:25.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.820+0000 7f002bfff640 1 -- 172.21.15.16:0/2267863012 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143858871 0 0) 0x7f002c0c6be0 con 0x7f002c0b4150 2023-12-08T08:39:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.821+0000 7f0031a58640 1 --2- 172.21.15.16:0/2267863012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c0cfe10 0x7f002c0b3630 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f001c009d90 tx=0x7f001c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8ac6405791b7b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.821+0000 7f002bfff640 1 -- 172.21.15.16:0/2267863012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c02cbd0 con 0x7f002c0cfe10 2023-12-08T08:39:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.821+0000 7f002bfff640 1 -- 172.21.15.16:0/2267863012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c02cd30 con 0x7f002c0cfe10 2023-12-08T08:39:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.821+0000 7f002bfff640 1 -- 172.21.15.16:0/2267863012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c02cea0 con 0x7f002c0cfe10 2023-12-08T08:39:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.821+0000 7f00334bb640 1 -- 172.21.15.16:0/2267863012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c0cfe10 msgr2=0x7f002c0b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.821+0000 7f00334bb640 1 --2- 172.21.15.16:0/2267863012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c0cfe10 0x7f002c0b3630 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f001c009d90 tx=0x7f001c02be20 comp rx=0 tx=0).stop 2023-12-08T08:39:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.821+0000 7f00334bb640 1 -- 172.21.15.16:0/2267863012 shutdown_connections 2023-12-08T08:39:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.821+0000 7f00334bb640 1 --2- 172.21.15.16:0/2267863012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c0cfe10 0x7f002c0b3630 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.821+0000 7f00334bb640 1 -- 172.21.15.16:0/2267863012 >> 172.21.15.16:0/2267863012 conn(0x7f002c0d3090 msgr2=0x7f002c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.821+0000 7f00334bb640 1 -- 172.21.15.16:0/2267863012 shutdown_connections 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.822+0000 7f00334bb640 1 -- 172.21.15.16:0/2267863012 wait complete. 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.823+0000 7f00334bb640 1 Processor -- start 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.823+0000 7f00334bb640 1 -- start start 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.824+0000 7f00334bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c0b4150 0x7f002c1f2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f00334bb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f002c0cfa30 0x7f002c1f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f00334bb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f002c0cfe10 0x7f002c15fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f00334bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c154df0 con 0x7f002c0b4150 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f00334bb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c1f2d90 con 0x7f002c0cfe10 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f00334bb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c1f73b0 con 0x7f002c0cfa30 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0031257640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c0b4150 0x7f002c1f2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0031257640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c0b4150 0x7f002c1f2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49658/0 (socket says 172.21.15.16:49658) 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0031257640 1 -- 172.21.15.16:0/3360382341 learned_addr learned my addr 172.21.15.16:0/3360382341 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0030a56640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f002c0cfa30 0x7f002c1f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0031a58640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f002c0cfe10 0x7f002c15fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0031257640 1 -- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f002c0cfa30 msgr2=0x7f002c1f6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0031257640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f002c0cfa30 0x7f002c1f6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0031257640 1 -- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f002c0cfe10 msgr2=0x7f002c15fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0031257640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f002c0cfe10 0x7f002c15fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0031257640 1 -- 172.21.15.16:0/3360382341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c009a70 con 0x7f002c0b4150 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0030a56640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f002c0cfa30 0x7f002c1f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0031a58640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f002c0cfe10 0x7f002c15fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0031257640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c0b4150 0x7f002c1f2850 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f002400c460 tx=0x7f002400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0029ffb640 1 -- 172.21.15.16:0/3360382341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024012830 con 0x7f002c0b4150 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0029ffb640 1 -- 172.21.15.16:0/3360382341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00240129d0 con 0x7f002c0b4150 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f0029ffb640 1 -- 172.21.15.16:0/3360382341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024012d80 con 0x7f002c0b4150 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.825+0000 7f00334bb640 1 -- 172.21.15.16:0/3360382341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002c15ef50 con 0x7f002c0b4150 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.826+0000 7f00334bb640 1 -- 172.21.15.16:0/3360382341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002c15f510 con 0x7f002c0b4150 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.827+0000 7f00334bb640 1 -- 172.21.15.16:0/3360382341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002c067430 con 0x7f002c0b4150 2023-12-08T08:39:25.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.828+0000 7f0029ffb640 1 -- 172.21.15.16:0/3360382341 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f002401b400 con 0x7f002c0b4150 2023-12-08T08:39:25.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.828+0000 7f0029ffb640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f001806ae40 0x7f001806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:25.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.828+0000 7f0029ffb640 1 -- 172.21.15.16:0/3360382341 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f0024089920 con 0x7f002c0b4150 2023-12-08T08:39:25.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.828+0000 7f0030a56640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f001806ae40 0x7f001806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:25.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.829+0000 7f0030a56640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f001806ae40 0x7f001806d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f002c155420 tx=0x7f0020007ca0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:25.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:25.832+0000 7f0029ffb640 1 -- 172.21.15.16:0/3360382341 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0024054680 con 0x7f002c0b4150 2023-12-08T08:39:26.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.009+0000 7f00334bb640 1 -- 172.21.15.16:0/3360382341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f002c0743e0 con 0x7f002c0b4150 2023-12-08T08:39:26.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.010+0000 7f0029ffb640 1 -- 172.21.15.16:0/3360382341 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0024057ca0 con 0x7f002c0b4150 2023-12-08T08:39:26.012 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.012+0000 7f00334bb640 1 -- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f001806ae40 msgr2=0x7f001806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.012+0000 7f00334bb640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f001806ae40 0x7f001806d300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f002c155420 tx=0x7f0020007ca0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.012+0000 7f00334bb640 1 -- 172.21.15.16:0/3360382341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c0b4150 msgr2=0x7f002c1f2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.012+0000 7f00334bb640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c0b4150 0x7f002c1f2850 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f002400c460 tx=0x7f002400c920 comp rx=0 tx=0).stop 2023-12-08T08:39:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.012+0000 7f00334bb640 1 -- 172.21.15.16:0/3360382341 shutdown_connections 2023-12-08T08:39:26.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.012+0000 7f00334bb640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f001806ae40 0x7f001806d300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.012+0000 7f00334bb640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f002c0cfe10 0x7f002c15fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.012+0000 7f00334bb640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f002c0cfa30 0x7f002c1f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.012+0000 7f00334bb640 1 --2- 172.21.15.16:0/3360382341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f002c0b4150 0x7f002c1f2850 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.012+0000 7f00334bb640 1 -- 172.21.15.16:0/3360382341 >> 172.21.15.16:0/3360382341 conn(0x7f002c0d3090 msgr2=0x7f002c06b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.012+0000 7f00334bb640 1 -- 172.21.15.16:0/3360382341 shutdown_connections 2023-12-08T08:39:26.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.012+0000 7f00334bb640 1 -- 172.21.15.16:0/3360382341 wait complete. 2023-12-08T08:39:26.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-08T08:39:26.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.130+0000 7f28e107c640 1 Processor -- start 2023-12-08T08:39:26.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.130+0000 7f28e107c640 1 -- start start 2023-12-08T08:39:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.130+0000 7f28e107c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28dc144a90 0x7f28dc15b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.130+0000 7f28e107c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28dc15ec10 0x7f28dc06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.130+0000 7f28e107c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28dc0716e0 con 0x7f28dc06a4d0 2023-12-08T08:39:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.130+0000 7f28e107c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28dc144e70 con 0x7f28dc144a90 2023-12-08T08:39:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.130+0000 7f28e107c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc06eff0 con 0x7f28dc15ec10 2023-12-08T08:39:26.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.131+0000 7f28dad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28dc144a90 0x7f28dc15b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.131+0000 7f28dad9d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28dc144a90 0x7f28dc15b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49660/0 (socket says 172.21.15.16:49660) 2023-12-08T08:39:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.131+0000 7f28dad9d640 1 -- 172.21.15.16:0/3453036464 learned_addr learned my addr 172.21.15.16:0/3453036464 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.131+0000 7f28da59c640 1 --2- 172.21.15.16:0/3453036464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28dc15ec10 0x7f28dc06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.131+0000 7f28dad9d640 1 -- 172.21.15.16:0/3453036464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28dc15ec10 msgr2=0x7f28dc06e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.131+0000 7f28dad9d640 1 --2- 172.21.15.16:0/3453036464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28dc15ec10 0x7f28dc06e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.131+0000 7f28dad9d640 1 -- 172.21.15.16:0/3453036464 >> v1:172.21.15.112:6789/0 conn(0x7f28dc06a4d0 legacy=0x7f28dc0ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.131+0000 7f28dad9d640 1 -- 172.21.15.16:0/3453036464 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28dc06f2d0 con 0x7f28dc144a90 2023-12-08T08:39:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.132+0000 7f28dad9d640 1 --2- 172.21.15.16:0/3453036464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc144a90 0x7f28dc15b660 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009870 tx=0x7f28cc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f62802eda2de30 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.132+0000 7f28d9d9b640 1 -- 172.21.15.16:0/3453036464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc030840 con 0x7f28dc144a90 2023-12-08T08:39:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.132+0000 7f28d9d9b640 1 -- 172.21.15.16:0/3453036464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28cc0309a0 con 0x7f28dc144a90 2023-12-08T08:39:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.132+0000 7f28d9d9b640 1 -- 172.21.15.16:0/3453036464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc030d30 con 0x7f28dc144a90 2023-12-08T08:39:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.132+0000 7f28e107c640 1 -- 172.21.15.16:0/3453036464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc144a90 msgr2=0x7f28dc15b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.132+0000 7f28e107c640 1 --2- 172.21.15.16:0/3453036464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc144a90 0x7f28dc15b660 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f28cc009870 tx=0x7f28cc02b980 comp rx=0 tx=0).stop 2023-12-08T08:39:26.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.132+0000 7f28e107c640 1 -- 172.21.15.16:0/3453036464 shutdown_connections 2023-12-08T08:39:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.132+0000 7f28e107c640 1 --2- 172.21.15.16:0/3453036464 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28dc15ec10 0x7f28dc06e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.132+0000 7f28e107c640 1 --2- 172.21.15.16:0/3453036464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc144a90 0x7f28dc15b660 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.132+0000 7f28e107c640 1 -- 172.21.15.16:0/3453036464 >> 172.21.15.16:0/3453036464 conn(0x7f28dc0cff90 msgr2=0x7f28dc0d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.132+0000 7f28e107c640 1 -- 172.21.15.16:0/3453036464 shutdown_connections 2023-12-08T08:39:26.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.132+0000 7f28e107c640 1 -- 172.21.15.16:0/3453036464 wait complete. 2023-12-08T08:39:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.134+0000 7f28e107c640 1 Processor -- start 2023-12-08T08:39:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.134+0000 7f28e107c640 1 -- start start 2023-12-08T08:39:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28e107c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc06a4d0 0x7f28dc0ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28e107c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28dc144a90 0x7f28dc13d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28e107c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28dc15ec10 0x7f28dc0b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28e107c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc0ac920 con 0x7f28dc06a4d0 2023-12-08T08:39:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28e107c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc14f590 con 0x7f28dc15ec10 2023-12-08T08:39:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28e107c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28dc0acbc0 con 0x7f28dc144a90 2023-12-08T08:39:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28dad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc06a4d0 0x7f28dc0ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28dad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc06a4d0 0x7f28dc0ad3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49662/0 (socket says 172.21.15.16:49662) 2023-12-08T08:39:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28dad9d640 1 -- 172.21.15.16:0/3575779947 learned_addr learned my addr 172.21.15.16:0/3575779947 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28db59e640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28dc15ec10 0x7f28dc0b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28da59c640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28dc144a90 0x7f28dc13d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28dad9d640 1 -- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28dc144a90 msgr2=0x7f28dc13d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28dad9d640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28dc144a90 0x7f28dc13d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28dad9d640 1 -- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28dc15ec10 msgr2=0x7f28dc0b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28dad9d640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28dc15ec10 0x7f28dc0b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28dad9d640 1 -- 172.21.15.16:0/3575779947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc0094d0 con 0x7f28dc06a4d0 2023-12-08T08:39:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28db59e640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28dc15ec10 0x7f28dc0b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28dad9d640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc06a4d0 0x7f28dc0ad3c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f28cc000c00 tx=0x7f28cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.135+0000 7f28c37fe640 1 -- 172.21.15.16:0/3575779947 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc039c90 con 0x7f28dc06a4d0 2023-12-08T08:39:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.136+0000 7f28c37fe640 1 -- 172.21.15.16:0/3575779947 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28cc039df0 con 0x7f28dc06a4d0 2023-12-08T08:39:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.136+0000 7f28e107c640 1 -- 172.21.15.16:0/3575779947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28dc0acea0 con 0x7f28dc06a4d0 2023-12-08T08:39:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.136+0000 7f28c37fe640 1 -- 172.21.15.16:0/3575779947 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc043640 con 0x7f28dc06a4d0 2023-12-08T08:39:26.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.136+0000 7f28e107c640 1 -- 172.21.15.16:0/3575779947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28dc13d6d0 con 0x7f28dc06a4d0 2023-12-08T08:39:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.137+0000 7f28e107c640 1 -- 172.21.15.16:0/3575779947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28dc0ad900 con 0x7f28dc06a4d0 2023-12-08T08:39:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.137+0000 7f28c37fe640 1 -- 172.21.15.16:0/3575779947 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f28cc0437a0 con 0x7f28dc06a4d0 2023-12-08T08:39:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.137+0000 7f28c37fe640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f28c806ae70 0x7f28c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.137+0000 7f28c37fe640 1 -- 172.21.15.16:0/3575779947 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f28cc03e070 con 0x7f28dc06a4d0 2023-12-08T08:39:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.138+0000 7f28da59c640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f28c806ae70 0x7f28c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.138+0000 7f28da59c640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f28c806ae70 0x7f28c806d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f28dc14c7e0 tx=0x7f28d0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:26.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.142+0000 7f28c37fe640 1 -- 172.21.15.16:0/3575779947 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28cc077790 con 0x7f28dc06a4d0 2023-12-08T08:39:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.331+0000 7f28e107c640 1 -- 172.21.15.16:0/3575779947 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f28dc0cda30 con 0x7f28dc06a4d0 2023-12-08T08:39:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.332+0000 7f28c37fe640 1 -- 172.21.15.16:0/3575779947 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f28cc034080 con 0x7f28dc06a4d0 2023-12-08T08:39:26.334 INFO:teuthology.orchestra.run.smithi016.stderr:listed 0 entries 2023-12-08T08:39:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.334+0000 7f28e107c640 1 -- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f28c806ae70 msgr2=0x7f28c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.334+0000 7f28e107c640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f28c806ae70 0x7f28c806d330 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f28dc14c7e0 tx=0x7f28d0008040 comp rx=0 tx=0).stop 2023-12-08T08:39:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.334+0000 7f28e107c640 1 -- 172.21.15.16:0/3575779947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc06a4d0 msgr2=0x7f28dc0ad3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.334+0000 7f28e107c640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc06a4d0 0x7f28dc0ad3c0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f28cc000c00 tx=0x7f28cc004730 comp rx=0 tx=0).stop 2023-12-08T08:39:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.334+0000 7f28e107c640 1 -- 172.21.15.16:0/3575779947 shutdown_connections 2023-12-08T08:39:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.334+0000 7f28e107c640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f28c806ae70 0x7f28c806d330 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.334+0000 7f28e107c640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f28dc15ec10 0x7f28dc0b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.334+0000 7f28e107c640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f28dc144a90 0x7f28dc13d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.334+0000 7f28e107c640 1 --2- 172.21.15.16:0/3575779947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc06a4d0 0x7f28dc0ad3c0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.334+0000 7f28e107c640 1 -- 172.21.15.16:0/3575779947 >> 172.21.15.16:0/3575779947 conn(0x7f28dc0cff90 msgr2=0x7f28dc0b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.334+0000 7f28e107c640 1 -- 172.21.15.16:0/3575779947 shutdown_connections 2023-12-08T08:39:26.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.334+0000 7f28e107c640 1 -- 172.21.15.16:0/3575779947 wait complete. 2023-12-08T08:39:26.347 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi112.front.sepia.ceph.com... 2023-12-08T08:39:26.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:26.347 DEBUG:teuthology.orchestra.run.smithi112:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-12-08T08:39:26.379 INFO:teuthology.orchestra.run:waiting for 300 2023-12-08T08:39:26.380 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-12-08T08:39:26.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:26.380 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-08T08:39:26.439 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-08T08:39:26.440 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-08T08:39:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.544+0000 7f54a0c4f640 1 Processor -- start 2023-12-08T08:39:26.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.544+0000 7f54a0c4f640 1 -- start start 2023-12-08T08:39:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.545+0000 7f54a0c4f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f549c0b3340 0x7f549c0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.545+0000 7f54a0c4f640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f549c0b3bf0 0x7f549c0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.545+0000 7f54a0c4f640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c0cdc20 con 0x7f549c0d15f0 2023-12-08T08:39:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.545+0000 7f54a0c4f640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c0d11d0 con 0x7f549c0b3bf0 2023-12-08T08:39:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.545+0000 7f54a0c4f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0d3a60 con 0x7f549c0b3340 2023-12-08T08:39:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.545+0000 7f549ad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f549c0d15f0 0x7f549c0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33290/0 (socket says 172.21.15.16:33290) 2023-12-08T08:39:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.545+0000 7f549ad9d640 1 -- 172.21.15.16:0/3106057532 learned_addr learned my addr 172.21.15.16:0/3106057532 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5493fff640 1 --2- 172.21.15.16:0/3106057532 >> v2:172.21.15.112:3300/0 conn(0x7f549c0b3bf0 0x7f549c0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 -- 172.21.15.16:0/3106057532 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 162810675 0 0) 0x7f549c0cdc20 con 0x7f549c0d15f0 2023-12-08T08:39:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f549a59c640 1 --2- 172.21.15.16:0/3106057532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f549c0b3340 0x7f549c0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 -- 172.21.15.16:0/3106057532 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5488003520 con 0x7f549c0d15f0 2023-12-08T08:39:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 -- 172.21.15.16:0/3106057532 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2874396346 0 0) 0x7f5488003520 con 0x7f549c0d15f0 2023-12-08T08:39:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 -- 172.21.15.16:0/3106057532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f549c0b3340 msgr2=0x7f549c0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 --2- 172.21.15.16:0/3106057532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f549c0b3340 0x7f549c0d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 -- 172.21.15.16:0/3106057532 >> v2:172.21.15.112:3300/0 conn(0x7f549c0b3bf0 msgr2=0x7f549c0b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 --2- 172.21.15.16:0/3106057532 >> v2:172.21.15.112:3300/0 conn(0x7f549c0b3bf0 0x7f549c0b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 -- 172.21.15.16:0/3106057532 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c0d3d40 con 0x7f549c0d15f0 2023-12-08T08:39:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 -- 172.21.15.16:0/3106057532 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f54940019f0 con 0x7f549c0d15f0 2023-12-08T08:39:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 -- 172.21.15.16:0/3106057532 >> v1:172.21.15.16:6789/0 conn(0x7f549c0d15f0 legacy=0x7f549c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 --2- 172.21.15.16:0/3106057532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f549c0b3340 0x7f5488003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 --2- 172.21.15.16:0/3106057532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5488003ef0 0x7f54880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 --2- 172.21.15.16:0/3106057532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488004d00 0x7f5488005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 -- 172.21.15.16:0/3106057532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0d11d0 con 0x7f5488003ef0 2023-12-08T08:39:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 -- 172.21.15.16:0/3106057532 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0d3a60 con 0x7f5488004d00 2023-12-08T08:39:26.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5499d9b640 1 -- 172.21.15.16:0/3106057532 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548800b9d0 con 0x7f549c0b3340 2023-12-08T08:39:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5493fff640 1 --2- 172.21.15.16:0/3106057532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5488003ef0 0x7f54880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f549a59c640 1 --2- 172.21.15.16:0/3106057532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f549c0b3340 0x7f5488003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f549ad9d640 1 --2- 172.21.15.16:0/3106057532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488004d00 0x7f5488005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5493fff640 1 -- 172.21.15.16:0/3106057532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f549c0b3340 msgr2=0x7f5488003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5493fff640 1 --2- 172.21.15.16:0/3106057532 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f549c0b3340 0x7f5488003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5493fff640 1 -- 172.21.15.16:0/3106057532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488004d00 msgr2=0x7f5488005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5493fff640 1 --2- 172.21.15.16:0/3106057532 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488004d00 0x7f5488005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.546+0000 7f5493fff640 1 -- 172.21.15.16:0/3106057532 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488003520 con 0x7f5488003ef0 2023-12-08T08:39:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.547+0000 7f5493fff640 1 --2- 172.21.15.16:0/3106057532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5488003ef0 0x7f54880047c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5484009ac0 tx=0x7f54880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5379c33b704380c server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.547+0000 7f5499d9b640 1 -- 172.21.15.16:0/3106057532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548402c6b0 con 0x7f5488003ef0 2023-12-08T08:39:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.547+0000 7f5499d9b640 1 -- 172.21.15.16:0/3106057532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548402c810 con 0x7f5488003ef0 2023-12-08T08:39:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.547+0000 7f5499d9b640 1 -- 172.21.15.16:0/3106057532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548402cba0 con 0x7f5488003ef0 2023-12-08T08:39:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.548+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3106057532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5488003ef0 msgr2=0x7f54880047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.548+0000 7f54a0c4f640 1 --2- 172.21.15.16:0/3106057532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5488003ef0 0x7f54880047c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5484009ac0 tx=0x7f54880057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.548+0000 7f549ad9d640 1 -- 172.21.15.16:0/3106057532 reap_dead start 2023-12-08T08:39:26.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.548+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3106057532 shutdown_connections 2023-12-08T08:39:26.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.548+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3106057532 >> 172.21.15.16:0/3106057532 conn(0x7f549c09c2f0 msgr2=0x7f549c0d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:26.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.548+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3106057532 shutdown_connections 2023-12-08T08:39:26.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.548+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3106057532 wait complete. 2023-12-08T08:39:26.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.549+0000 7f54a0c4f640 1 Processor -- start 2023-12-08T08:39:26.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.549+0000 7f54a0c4f640 1 -- start start 2023-12-08T08:39:26.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f54a0c4f640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488003ef0 0x7f549c1470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f54a0c4f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c156f20 0x7f549c0b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f54a0c4f640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f549c0b7370 0x7f549c0b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f54a0c4f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c1573e0 con 0x7f549c156f20 2023-12-08T08:39:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f54a0c4f640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c159de0 con 0x7f5488003ef0 2023-12-08T08:39:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f54a0c4f640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c15a0c0 con 0x7f549c0b7370 2023-12-08T08:39:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f5493fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c156f20 0x7f549c0b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f5493fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c156f20 0x7f549c0b6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47518/0 (socket says 172.21.15.16:47518) 2023-12-08T08:39:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f5493fff640 1 -- 172.21.15.16:0/3580105443 learned_addr learned my addr 172.21.15.16:0/3580105443 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f549ad9d640 1 --2- 172.21.15.16:0/3580105443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f549c0b7370 0x7f549c0b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f549a59c640 1 --2- 172.21.15.16:0/3580105443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488003ef0 0x7f549c1470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f5493fff640 1 -- 172.21.15.16:0/3580105443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f549c0b7370 msgr2=0x7f549c0b7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f5493fff640 1 --2- 172.21.15.16:0/3580105443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f549c0b7370 0x7f549c0b7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f5493fff640 1 -- 172.21.15.16:0/3580105443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488003ef0 msgr2=0x7f549c1470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f5493fff640 1 --2- 172.21.15.16:0/3580105443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488003ef0 0x7f549c1470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.550+0000 7f5493fff640 1 -- 172.21.15.16:0/3580105443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c1568a0 con 0x7f549c156f20 2023-12-08T08:39:26.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.551+0000 7f5493fff640 1 --2- 172.21.15.16:0/3580105443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c156f20 0x7f549c0b6e30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f548402c250 tx=0x7f5484004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.551+0000 7f5492ffd640 1 -- 172.21.15.16:0/3580105443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484003810 con 0x7f549c156f20 2023-12-08T08:39:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.551+0000 7f5492ffd640 1 -- 172.21.15.16:0/3580105443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5484003970 con 0x7f549c156f20 2023-12-08T08:39:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.551+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3580105443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c147620 con 0x7f549c156f20 2023-12-08T08:39:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.551+0000 7f5492ffd640 1 -- 172.21.15.16:0/3580105443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5484003ce0 con 0x7f549c156f20 2023-12-08T08:39:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.551+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3580105443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c0b8210 con 0x7f549c156f20 2023-12-08T08:39:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.552+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3580105443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c067430 con 0x7f549c156f20 2023-12-08T08:39:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.552+0000 7f5492ffd640 1 -- 172.21.15.16:0/3580105443 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5484040450 con 0x7f549c156f20 2023-12-08T08:39:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.553+0000 7f5492ffd640 1 --2- 172.21.15.16:0/3580105443 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f547c06ae40 0x7f547c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.553+0000 7f5492ffd640 1 -- 172.21.15.16:0/3580105443 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f54840ab840 con 0x7f549c156f20 2023-12-08T08:39:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.553+0000 7f549a59c640 1 --2- 172.21.15.16:0/3580105443 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f547c06ae40 0x7f547c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.554+0000 7f549a59c640 1 --2- 172.21.15.16:0/3580105443 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f547c06ae40 0x7f547c06d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f548c009ae0 tx=0x7f548c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:26.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.557+0000 7f5492ffd640 1 -- 172.21.15.16:0/3580105443 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5484076650 con 0x7f549c156f20 2023-12-08T08:39:26.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.769+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3580105443 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f549c0d13d0 con 0x7f549c156f20 2023-12-08T08:39:26.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.770+0000 7f5492ffd640 1 -- 172.21.15.16:0/3580105443 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f54840044e0 con 0x7f549c156f20 2023-12-08T08:39:26.772 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:26.773 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/307039451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":307039451},{"type":"v1","addr":"172.21.15.112:6836","nonce":307039451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1424152341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1424152341},{"type":"v1","addr":"172.21.15.16:6833","nonce":1424152341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3597811670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3597811670},{"type":"v1","addr":"172.21.15.16:6835","nonce":3597811670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-08T08:38:18.991090+0000","modified":"2023-12-08T08:38:41.075742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1082241707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1082241707},{"type":"v1","addr":"172.21.15.112:6837","nonce":1082241707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-08T08:39:26.773 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-12-08T08:39:26.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.772+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3580105443 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f547c06ae40 msgr2=0x7f547c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.772+0000 7f54a0c4f640 1 --2- 172.21.15.16:0/3580105443 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f547c06ae40 0x7f547c06d300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f548c009ae0 tx=0x7f548c009190 comp rx=0 tx=0).stop 2023-12-08T08:39:26.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.772+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3580105443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c156f20 msgr2=0x7f549c0b6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.772+0000 7f54a0c4f640 1 --2- 172.21.15.16:0/3580105443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c156f20 0x7f549c0b6e30 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f548402c250 tx=0x7f5484004730 comp rx=0 tx=0).stop 2023-12-08T08:39:26.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.772+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3580105443 shutdown_connections 2023-12-08T08:39:26.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.772+0000 7f54a0c4f640 1 --2- 172.21.15.16:0/3580105443 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f547c06ae40 0x7f547c06d300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.772+0000 7f54a0c4f640 1 --2- 172.21.15.16:0/3580105443 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f549c0b7370 0x7f549c0b7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.772+0000 7f54a0c4f640 1 --2- 172.21.15.16:0/3580105443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f549c156f20 0x7f549c0b6e30 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.772+0000 7f54a0c4f640 1 --2- 172.21.15.16:0/3580105443 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5488003ef0 0x7f549c1470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.772+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3580105443 >> 172.21.15.16:0/3580105443 conn(0x7f549c09c2f0 msgr2=0x7f549c0cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:26.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.772+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3580105443 shutdown_connections 2023-12-08T08:39:26.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.772+0000 7f54a0c4f640 1 -- 172.21.15.16:0/3580105443 wait complete. 2023-12-08T08:39:26.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:26.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.890+0000 7f55f5f1d640 1 Processor -- start 2023-12-08T08:39:26.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.890+0000 7f55f5f1d640 1 -- start start 2023-12-08T08:39:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.890+0000 7f55f5f1d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f55f01608a0 0x7f55f00d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.890+0000 7f55f5f1d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55f00b1d70 0x7f55f00d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.890+0000 7f55f5f1d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f00d20b0 con 0x7f55f00afce0 2023-12-08T08:39:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.890+0000 7f55f5f1d640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f00d0620 con 0x7f55f00b1d70 2023-12-08T08:39:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.890+0000 7f55f5f1d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f00d1dd0 con 0x7f55f01608a0 2023-12-08T08:39:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.891+0000 7f55eeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55f00b1d70 0x7f55f00d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.891+0000 7f55eeffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55f00b1d70 0x7f55f00d1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47532/0 (socket says 172.21.15.16:47532) 2023-12-08T08:39:26.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.891+0000 7f55eeffd640 1 -- 172.21.15.16:0/2086664992 learned_addr learned my addr 172.21.15.16:0/2086664992 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:26.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.891+0000 7f55ef7fe640 1 --2- 172.21.15.16:0/2086664992 >> v2:172.21.15.112:3300/0 conn(0x7f55f01608a0 0x7f55f00d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.891+0000 7f55eeffd640 1 -- 172.21.15.16:0/2086664992 >> v2:172.21.15.112:3300/0 conn(0x7f55f01608a0 msgr2=0x7f55f00d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:26.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.891+0000 7f55eeffd640 1 --2- 172.21.15.16:0/2086664992 >> v2:172.21.15.112:3300/0 conn(0x7f55f01608a0 0x7f55f00d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.891+0000 7f55eeffd640 1 -- 172.21.15.16:0/2086664992 >> v1:172.21.15.112:6789/0 conn(0x7f55f00afce0 legacy=0x7f55f00b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.891+0000 7f55eeffd640 1 -- 172.21.15.16:0/2086664992 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f00cff60 con 0x7f55f00b1d70 2023-12-08T08:39:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.892+0000 7f55eeffd640 1 --2- 172.21.15.16:0/2086664992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00b1d70 0x7f55f00d1470 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f55f008b770 tx=0x7f55e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cafea80501c14 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.892+0000 7f55ee7fc640 1 -- 172.21.15.16:0/2086664992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0031870 con 0x7f55f00b1d70 2023-12-08T08:39:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.892+0000 7f55ee7fc640 1 -- 172.21.15.16:0/2086664992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e00319d0 con 0x7f55f00b1d70 2023-12-08T08:39:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.892+0000 7f55ee7fc640 1 -- 172.21.15.16:0/2086664992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0031d60 con 0x7f55f00b1d70 2023-12-08T08:39:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.892+0000 7f55f5f1d640 1 -- 172.21.15.16:0/2086664992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00b1d70 msgr2=0x7f55f00d1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.892+0000 7f55f5f1d640 1 --2- 172.21.15.16:0/2086664992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00b1d70 0x7f55f00d1470 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f55f008b770 tx=0x7f55e002b910 comp rx=0 tx=0).stop 2023-12-08T08:39:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.892+0000 7f55f5f1d640 1 -- 172.21.15.16:0/2086664992 shutdown_connections 2023-12-08T08:39:26.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.892+0000 7f55f5f1d640 1 --2- 172.21.15.16:0/2086664992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f00b1d70 0x7f55f00d1470 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.892+0000 7f55f5f1d640 1 --2- 172.21.15.16:0/2086664992 >> v2:172.21.15.112:3300/0 conn(0x7f55f01608a0 0x7f55f00d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.892+0000 7f55f5f1d640 1 -- 172.21.15.16:0/2086664992 >> 172.21.15.16:0/2086664992 conn(0x7f55f00d3030 msgr2=0x7f55f00d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.892+0000 7f55f5f1d640 1 -- 172.21.15.16:0/2086664992 shutdown_connections 2023-12-08T08:39:26.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.892+0000 7f55f5f1d640 1 -- 172.21.15.16:0/2086664992 wait complete. 2023-12-08T08:39:26.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.894+0000 7f55f5f1d640 1 Processor -- start 2023-12-08T08:39:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.894+0000 7f55f5f1d640 1 -- start start 2023-12-08T08:39:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.894+0000 7f55f5f1d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55f00afce0 0x7f55f015f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55f5f1d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f00b1d70 0x7f55f00ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55f5f1d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f01608a0 0x7f55f00ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55f5f1d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00cb880 con 0x7f55f01608a0 2023-12-08T08:39:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55f5f1d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00b86c0 con 0x7f55f00afce0 2023-12-08T08:39:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55f5f1d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00cac70 con 0x7f55f00b1d70 2023-12-08T08:39:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f01608a0 0x7f55f00ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55effff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f01608a0 0x7f55f00ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47540/0 (socket says 172.21.15.16:47540) 2023-12-08T08:39:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55effff640 1 -- 172.21.15.16:0/556577522 learned_addr learned my addr 172.21.15.16:0/556577522 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55eeffd640 1 --2- 172.21.15.16:0/556577522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f00b1d70 0x7f55f00ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55ef7fe640 1 --2- 172.21.15.16:0/556577522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55f00afce0 0x7f55f015f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55effff640 1 -- 172.21.15.16:0/556577522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f00b1d70 msgr2=0x7f55f00ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55effff640 1 --2- 172.21.15.16:0/556577522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f00b1d70 0x7f55f00ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55effff640 1 -- 172.21.15.16:0/556577522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55f00afce0 msgr2=0x7f55f015f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55effff640 1 --2- 172.21.15.16:0/556577522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55f00afce0 0x7f55f015f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55effff640 1 -- 172.21.15.16:0/556577522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e00094d0 con 0x7f55f01608a0 2023-12-08T08:39:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55effff640 1 --2- 172.21.15.16:0/556577522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f01608a0 0x7f55f00ca640 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f55e800c6b0 tx=0x7f55e800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55e7fff640 1 -- 172.21.15.16:0/556577522 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8003b70 con 0x7f55f01608a0 2023-12-08T08:39:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55e7fff640 1 -- 172.21.15.16:0/556577522 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e8003cf0 con 0x7f55f01608a0 2023-12-08T08:39:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55e7fff640 1 -- 172.21.15.16:0/556577522 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8003ea0 con 0x7f55f01608a0 2023-12-08T08:39:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55f5f1d640 1 -- 172.21.15.16:0/556577522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f00caf50 con 0x7f55f01608a0 2023-12-08T08:39:26.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.895+0000 7f55f5f1d640 1 -- 172.21.15.16:0/556577522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f00cb510 con 0x7f55f01608a0 2023-12-08T08:39:26.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.896+0000 7f55f5f1d640 1 -- 172.21.15.16:0/556577522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f00cb6a0 con 0x7f55f01608a0 2023-12-08T08:39:26.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.897+0000 7f55e7fff640 1 -- 172.21.15.16:0/556577522 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f55e8005310 con 0x7f55f01608a0 2023-12-08T08:39:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.897+0000 7f55e7fff640 1 --2- 172.21.15.16:0/556577522 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f55d006ae70 0x7f55d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.897+0000 7f55e7fff640 1 -- 172.21.15.16:0/556577522 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f55e8089790 con 0x7f55f01608a0 2023-12-08T08:39:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.898+0000 7f55ef7fe640 1 --2- 172.21.15.16:0/556577522 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f55d006ae70 0x7f55d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.898+0000 7f55ef7fe640 1 --2- 172.21.15.16:0/556577522 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f55d006ae70 0x7f55d006d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f55d8006fd0 tx=0x7f55d8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:26.902+0000 7f55e7fff640 1 -- 172.21.15.16:0/556577522 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f55e8054460 con 0x7f55f01608a0 2023-12-08T08:39:27.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.080+0000 7f55f5f1d640 1 -- 172.21.15.16:0/556577522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55f015f9b0 con 0x7f55f01608a0 2023-12-08T08:39:27.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.081+0000 7f55e7fff640 1 -- 172.21.15.16:0/556577522 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7f55e8057a80 con 0x7f55f01608a0 2023-12-08T08:39:27.083 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:27.083 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.083+0000 7f55f5f1d640 1 -- 172.21.15.16:0/556577522 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f55d006ae70 msgr2=0x7f55d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.083+0000 7f55f5f1d640 1 --2- 172.21.15.16:0/556577522 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f55d006ae70 0x7f55d006d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f55d8006fd0 tx=0x7f55d8008040 comp rx=0 tx=0).stop 2023-12-08T08:39:27.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.083+0000 7f55f5f1d640 1 -- 172.21.15.16:0/556577522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f01608a0 msgr2=0x7f55f00ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.083+0000 7f55f5f1d640 1 --2- 172.21.15.16:0/556577522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f01608a0 0x7f55f00ca640 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f55e800c6b0 tx=0x7f55e800cb70 comp rx=0 tx=0).stop 2023-12-08T08:39:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.083+0000 7f55f5f1d640 1 -- 172.21.15.16:0/556577522 shutdown_connections 2023-12-08T08:39:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.083+0000 7f55f5f1d640 1 --2- 172.21.15.16:0/556577522 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f55d006ae70 0x7f55d006d330 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.083+0000 7f55f5f1d640 1 --2- 172.21.15.16:0/556577522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55f01608a0 0x7f55f00ca640 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.083+0000 7f55f5f1d640 1 --2- 172.21.15.16:0/556577522 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f55f00b1d70 0x7f55f00ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.083+0000 7f55f5f1d640 1 --2- 172.21.15.16:0/556577522 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55f00afce0 0x7f55f015f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.083+0000 7f55f5f1d640 1 -- 172.21.15.16:0/556577522 >> 172.21.15.16:0/556577522 conn(0x7f55f00d3030 msgr2=0x7f55f006c600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.083+0000 7f55f5f1d640 1 -- 172.21.15.16:0/556577522 shutdown_connections 2023-12-08T08:39:27.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.083+0000 7f55f5f1d640 1 -- 172.21.15.16:0/556577522 wait complete. 2023-12-08T08:39:27.096 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-08T08:38:17.121453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-08T08:38:17.965614+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-08T08:38:21.683956+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-08T08:39:27.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:27.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.204+0000 7feabe5c9640 1 Processor -- start 2023-12-08T08:39:27.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.204+0000 7feabe5c9640 1 -- start start 2023-12-08T08:39:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.205+0000 7feabe5c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab806bbf0 0x7feab80ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.205+0000 7feabe5c9640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab815d500 con 0x7feab815d120 2023-12-08T08:39:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.205+0000 7feabe5c9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab80ad7c0 con 0x7feab80ad290 2023-12-08T08:39:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.205+0000 7feabe5c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab80b2f80 con 0x7feab806bbf0 2023-12-08T08:39:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.205+0000 7feab7fff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7feab815d120 0x7feab806b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33294/0 (socket says 172.21.15.16:33294) 2023-12-08T08:39:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.205+0000 7feab7fff640 1 -- 172.21.15.16:0/487950468 learned_addr learned my addr 172.21.15.16:0/487950468 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab77fe640 1 --2- 172.21.15.16:0/487950468 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab806bbf0 0x7feab80ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 -- 172.21.15.16:0/487950468 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879021058 0 0) 0x7feab815d500 con 0x7feab815d120 2023-12-08T08:39:27.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 -- 172.21.15.16:0/487950468 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaa0003520 con 0x7feab815d120 2023-12-08T08:39:27.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 -- 172.21.15.16:0/487950468 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3216539987 0 0) 0x7feaa0003520 con 0x7feab815d120 2023-12-08T08:39:27.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 -- 172.21.15.16:0/487950468 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab806bbf0 msgr2=0x7feab80ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:27.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 --2- 172.21.15.16:0/487950468 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab806bbf0 0x7feab80ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 -- 172.21.15.16:0/487950468 >> v1:172.21.15.112:6789/0 conn(0x7feab80ad290 legacy=0x7feab80b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 -- 172.21.15.16:0/487950468 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab80b3260 con 0x7feab815d120 2023-12-08T08:39:27.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 -- 172.21.15.16:0/487950468 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7feaac001670 con 0x7feab815d120 2023-12-08T08:39:27.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 -- 172.21.15.16:0/487950468 >> v1:172.21.15.16:6789/0 conn(0x7feab815d120 legacy=0x7feab806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 --2- 172.21.15.16:0/487950468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa00039a0 0x7feaa0003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 --2- 172.21.15.16:0/487950468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaa00047c0 0x7feaa0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 --2- 172.21.15.16:0/487950468 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab806bbf0 0x7feaa0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 -- 172.21.15.16:0/487950468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab80b2f80 con 0x7feaa00039a0 2023-12-08T08:39:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 -- 172.21.15.16:0/487950468 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa000ba30 con 0x7feaa00047c0 2023-12-08T08:39:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab6ffd640 1 -- 172.21.15.16:0/487950468 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa000bbc0 con 0x7feab806bbf0 2023-12-08T08:39:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab7fff640 1 --2- 172.21.15.16:0/487950468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa00039a0 0x7feaa0003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab77fe640 1 --2- 172.21.15.16:0/487950468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaa00047c0 0x7feaa0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feabcb66640 1 --2- 172.21.15.16:0/487950468 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab806bbf0 0x7feaa0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab7fff640 1 -- 172.21.15.16:0/487950468 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab806bbf0 msgr2=0x7feaa0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab7fff640 1 --2- 172.21.15.16:0/487950468 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feab806bbf0 0x7feaa0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab7fff640 1 -- 172.21.15.16:0/487950468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaa00047c0 msgr2=0x7feaa0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab7fff640 1 --2- 172.21.15.16:0/487950468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaa00047c0 0x7feaa0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab7fff640 1 -- 172.21.15.16:0/487950468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa0003520 con 0x7feaa00039a0 2023-12-08T08:39:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.206+0000 7feab77fe640 1 --2- 172.21.15.16:0/487950468 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feaa00047c0 0x7feaa0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.207+0000 7feab7fff640 1 --2- 172.21.15.16:0/487950468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa00039a0 0x7feaa0003d80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7feaac00c160 tx=0x7feaa00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dcf34f8a60221cd server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.207+0000 7feab6ffd640 1 -- 172.21.15.16:0/487950468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac005ad0 con 0x7feaa00039a0 2023-12-08T08:39:27.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.207+0000 7feab6ffd640 1 -- 172.21.15.16:0/487950468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaac005c30 con 0x7feaa00039a0 2023-12-08T08:39:27.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.207+0000 7feab6ffd640 1 -- 172.21.15.16:0/487950468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac02c7c0 con 0x7feaa00039a0 2023-12-08T08:39:27.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.208+0000 7feabe5c9640 1 -- 172.21.15.16:0/487950468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa00039a0 msgr2=0x7feaa0003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.208+0000 7feabe5c9640 1 --2- 172.21.15.16:0/487950468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaa00039a0 0x7feaa0003d80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7feaac00c160 tx=0x7feaa00057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.208+0000 7feabcb66640 1 -- 172.21.15.16:0/487950468 reap_dead start 2023-12-08T08:39:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.208+0000 7feabe5c9640 1 -- 172.21.15.16:0/487950468 shutdown_connections 2023-12-08T08:39:27.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.208+0000 7feabe5c9640 1 -- 172.21.15.16:0/487950468 >> 172.21.15.16:0/487950468 conn(0x7feab8070960 msgr2=0x7feab8070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.208+0000 7feabe5c9640 1 -- 172.21.15.16:0/487950468 shutdown_connections 2023-12-08T08:39:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.208+0000 7feabe5c9640 1 -- 172.21.15.16:0/487950468 wait complete. 2023-12-08T08:39:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.209+0000 7feabe5c9640 1 Processor -- start 2023-12-08T08:39:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.209+0000 7feabe5c9640 1 -- start start 2023-12-08T08:39:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feabe5c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaa00039a0 0x7feab814eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feabe5c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab8153270 0x7feab8151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feabe5c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab80b0330 0x7feab80b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feabe5c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab814f4f0 con 0x7feab8153270 2023-12-08T08:39:27.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feabe5c9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab81593c0 con 0x7feab80b0330 2023-12-08T08:39:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feabe5c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab80b0cf0 con 0x7feaa00039a0 2023-12-08T08:39:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feab77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab8153270 0x7feab8151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feab77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab8153270 0x7feab8151790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47564/0 (socket says 172.21.15.16:47564) 2023-12-08T08:39:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feab77fe640 1 -- 172.21.15.16:0/1788321150 learned_addr learned my addr 172.21.15.16:0/1788321150 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feab7fff640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaa00039a0 0x7feab814eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feabcb66640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab80b0330 0x7feab80b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feab77fe640 1 -- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaa00039a0 msgr2=0x7feab814eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feab77fe640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaa00039a0 0x7feab814eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feab77fe640 1 -- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab80b0330 msgr2=0x7feab80b07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feab77fe640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab80b0330 0x7feab80b07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feab77fe640 1 -- 172.21.15.16:0/1788321150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab81f7390 con 0x7feab8153270 2023-12-08T08:39:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.210+0000 7feabcb66640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab80b0330 0x7feab80b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.211+0000 7feab77fe640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab8153270 0x7feab8151790 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7feaa80115e0 tx=0x7feaa8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:27.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.211+0000 7feab4ff9640 1 -- 172.21.15.16:0/1788321150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8004220 con 0x7feab8153270 2023-12-08T08:39:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.211+0000 7feab4ff9640 1 -- 172.21.15.16:0/1788321150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa8004380 con 0x7feab8153270 2023-12-08T08:39:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.211+0000 7feabe5c9640 1 -- 172.21.15.16:0/1788321150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab815c800 con 0x7feab8153270 2023-12-08T08:39:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.211+0000 7feab4ff9640 1 -- 172.21.15.16:0/1788321150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa800f910 con 0x7feab8153270 2023-12-08T08:39:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.211+0000 7feabe5c9640 1 -- 172.21.15.16:0/1788321150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab81f6af0 con 0x7feab8153270 2023-12-08T08:39:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.212+0000 7feabe5c9640 1 -- 172.21.15.16:0/1788321150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab8067430 con 0x7feab8153270 2023-12-08T08:39:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.212+0000 7feab4ff9640 1 -- 172.21.15.16:0/1788321150 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feaa800fa70 con 0x7feab8153270 2023-12-08T08:39:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.212+0000 7feab4ff9640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7feaa406ae40 0x7feaa406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.213+0000 7feab4ff9640 1 -- 172.21.15.16:0/1788321150 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7feaa8089a90 con 0x7feab8153270 2023-12-08T08:39:27.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.213+0000 7feab7fff640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7feaa406ae40 0x7feaa406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.213+0000 7feab7fff640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7feaa406ae40 0x7feaa406d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feaac034000 tx=0x7feaac009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:27.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.217+0000 7feab4ff9640 1 -- 172.21.15.16:0/1788321150 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feaa80546d0 con 0x7feab8153270 2023-12-08T08:39:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.393+0000 7feabe5c9640 1 -- 172.21.15.16:0/1788321150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feab8073be0 con 0x7feab8153270 2023-12-08T08:39:27.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.394+0000 7feab4ff9640 1 -- 172.21.15.16:0/1788321150 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7feaa80220a0 con 0x7feab8153270 2023-12-08T08:39:27.396 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.395+0000 7feabe5c9640 1 -- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7feaa406ae40 msgr2=0x7feaa406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.395+0000 7feabe5c9640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7feaa406ae40 0x7feaa406d300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feaac034000 tx=0x7feaac009030 comp rx=0 tx=0).stop 2023-12-08T08:39:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.395+0000 7feabe5c9640 1 -- 172.21.15.16:0/1788321150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab8153270 msgr2=0x7feab8151790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.395+0000 7feabe5c9640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab8153270 0x7feab8151790 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7feaa80115e0 tx=0x7feaa8011aa0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.395+0000 7feabe5c9640 1 -- 172.21.15.16:0/1788321150 shutdown_connections 2023-12-08T08:39:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.395+0000 7feabe5c9640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7feaa406ae40 0x7feaa406d300 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.395+0000 7feabe5c9640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7feab80b0330 0x7feab80b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.395+0000 7feabe5c9640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feab8153270 0x7feab8151790 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.395+0000 7feabe5c9640 1 --2- 172.21.15.16:0/1788321150 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7feaa00039a0 0x7feab814eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.396+0000 7feabe5c9640 1 -- 172.21.15.16:0/1788321150 >> 172.21.15.16:0/1788321150 conn(0x7feab8070960 msgr2=0x7feab815f700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.396+0000 7feabe5c9640 1 -- 172.21.15.16:0/1788321150 shutdown_connections 2023-12-08T08:39:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.396+0000 7feabe5c9640 1 -- 172.21.15.16:0/1788321150 wait complete. 2023-12-08T08:39:27.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-08T08:39:27.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.516+0000 7f4a5c7b6640 1 Processor -- start 2023-12-08T08:39:27.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.516+0000 7f4a5c7b6640 1 -- start start 2023-12-08T08:39:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.517+0000 7f4a5c7b6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f4a540cd3d0 0x7f4a540712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.517+0000 7f4a5c7b6640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a540cdec0 0x7f4a5415b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.517+0000 7f4a5c7b6640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a540ce370 con 0x7f4a5406d6a0 2023-12-08T08:39:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.517+0000 7f4a5c7b6640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a5406d280 con 0x7f4a540cdec0 2023-12-08T08:39:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.517+0000 7f4a5c7b6640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a54071800 con 0x7f4a540cd3d0 2023-12-08T08:39:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.517+0000 7f4a5ad53640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a540cdec0 0x7f4a5415b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.517+0000 7f4a5ad53640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a540cdec0 0x7f4a5415b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47580/0 (socket says 172.21.15.16:47580) 2023-12-08T08:39:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.517+0000 7f4a5ad53640 1 -- 172.21.15.16:0/2508961435 learned_addr learned my addr 172.21.15.16:0/2508961435 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.518+0000 7f4a5ad53640 1 -- 172.21.15.16:0/2508961435 >> v2:172.21.15.112:3300/0 conn(0x7f4a540cd3d0 msgr2=0x7f4a540712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.518+0000 7f4a5a552640 1 --2- 172.21.15.16:0/2508961435 >> v2:172.21.15.112:3300/0 conn(0x7f4a540cd3d0 0x7f4a540712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.518+0000 7f4a5ad53640 1 --2- 172.21.15.16:0/2508961435 >> v2:172.21.15.112:3300/0 conn(0x7f4a540cd3d0 0x7f4a540712c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.518+0000 7f4a5ad53640 1 -- 172.21.15.16:0/2508961435 >> v1:172.21.15.112:6789/0 conn(0x7f4a5406d6a0 legacy=0x7f4a540cd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.518+0000 7f4a5ad53640 1 -- 172.21.15.16:0/2508961435 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5406c8e0 con 0x7f4a540cdec0 2023-12-08T08:39:27.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.518+0000 7f4a5ad53640 1 --2- 172.21.15.16:0/2508961435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a540cdec0 0x7f4a5415b4b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4a5406fd80 tx=0x7f4a4402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73081e0067b4abe2 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:27.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.519+0000 7f4a59550640 1 -- 172.21.15.16:0/2508961435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44031870 con 0x7f4a540cdec0 2023-12-08T08:39:27.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.519+0000 7f4a59550640 1 -- 172.21.15.16:0/2508961435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a440319d0 con 0x7f4a540cdec0 2023-12-08T08:39:27.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.519+0000 7f4a59550640 1 -- 172.21.15.16:0/2508961435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44031d60 con 0x7f4a540cdec0 2023-12-08T08:39:27.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.519+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2508961435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a540cdec0 msgr2=0x7f4a5415b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.519+0000 7f4a5c7b6640 1 --2- 172.21.15.16:0/2508961435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a540cdec0 0x7f4a5415b4b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4a5406fd80 tx=0x7f4a4402b910 comp rx=0 tx=0).stop 2023-12-08T08:39:27.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.519+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2508961435 shutdown_connections 2023-12-08T08:39:27.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.519+0000 7f4a5c7b6640 1 --2- 172.21.15.16:0/2508961435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a540cdec0 0x7f4a5415b4b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.519+0000 7f4a5c7b6640 1 --2- 172.21.15.16:0/2508961435 >> v2:172.21.15.112:3300/0 conn(0x7f4a540cd3d0 0x7f4a540712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.519+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2508961435 >> 172.21.15.16:0/2508961435 conn(0x7f4a540b0850 msgr2=0x7f4a540b0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.519+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2508961435 shutdown_connections 2023-12-08T08:39:27.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.519+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2508961435 wait complete. 2023-12-08T08:39:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.520+0000 7f4a5c7b6640 1 Processor -- start 2023-12-08T08:39:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.520+0000 7f4a5c7b6640 1 -- start start 2023-12-08T08:39:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.521+0000 7f4a5c7b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a5406d6a0 0x7f4a541345d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.521+0000 7f4a5c7b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a540cd3d0 0x7f4a5413dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.521+0000 7f4a5c7b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a540cdec0 0x7f4a541401b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.521+0000 7f4a5c7b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5413e4e0 con 0x7f4a540cdec0 2023-12-08T08:39:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.521+0000 7f4a5c7b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5413e780 con 0x7f4a540cd3d0 2023-12-08T08:39:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.521+0000 7f4a5c7b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5413ea60 con 0x7f4a5406d6a0 2023-12-08T08:39:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.521+0000 7f4a5ad53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a540cdec0 0x7f4a541401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.521+0000 7f4a5ad53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a540cdec0 0x7f4a541401b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47586/0 (socket says 172.21.15.16:47586) 2023-12-08T08:39:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.521+0000 7f4a5ad53640 1 -- 172.21.15.16:0/2531562526 learned_addr learned my addr 172.21.15.16:0/2531562526 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.521+0000 7f4a5a552640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a5406d6a0 0x7f4a541345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.521+0000 7f4a59d51640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a540cd3d0 0x7f4a5413dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.522+0000 7f4a5ad53640 1 -- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a5406d6a0 msgr2=0x7f4a541345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.522+0000 7f4a5ad53640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a5406d6a0 0x7f4a541345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.522+0000 7f4a5ad53640 1 -- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a540cd3d0 msgr2=0x7f4a5413dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.522+0000 7f4a5ad53640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a540cd3d0 0x7f4a5413dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.522+0000 7f4a5ad53640 1 -- 172.21.15.16:0/2531562526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a440094d0 con 0x7f4a540cdec0 2023-12-08T08:39:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.522+0000 7f4a5a552640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a5406d6a0 0x7f4a541345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.522+0000 7f4a59d51640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a540cd3d0 0x7f4a5413dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.522+0000 7f4a5ad53640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a540cdec0 0x7f4a541401b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4a440090b0 tx=0x7f4a440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.522+0000 7f4a52ffd640 1 -- 172.21.15.16:0/2531562526 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4403adf0 con 0x7f4a540cdec0 2023-12-08T08:39:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.522+0000 7f4a52ffd640 1 -- 172.21.15.16:0/2531562526 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a440443f0 con 0x7f4a540cdec0 2023-12-08T08:39:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.522+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2531562526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5413ed40 con 0x7f4a540cdec0 2023-12-08T08:39:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.522+0000 7f4a52ffd640 1 -- 172.21.15.16:0/2531562526 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a440447a0 con 0x7f4a540cdec0 2023-12-08T08:39:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.522+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2531562526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5413f300 con 0x7f4a540cdec0 2023-12-08T08:39:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.523+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2531562526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a540719d0 con 0x7f4a540cdec0 2023-12-08T08:39:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.524+0000 7f4a52ffd640 1 -- 172.21.15.16:0/2531562526 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a4404d450 con 0x7f4a540cdec0 2023-12-08T08:39:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.524+0000 7f4a52ffd640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4a4006ae40 0x7f4a4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.524+0000 7f4a52ffd640 1 -- 172.21.15.16:0/2531562526 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f4a4402d040 con 0x7f4a540cdec0 2023-12-08T08:39:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.524+0000 7f4a5a552640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4a4006ae40 0x7f4a4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.525+0000 7f4a5a552640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4a4006ae40 0x7f4a4006d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c007f50 tx=0x7f4a4c0074a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:27.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.528+0000 7f4a52ffd640 1 -- 172.21.15.16:0/2531562526 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a44076d20 con 0x7f4a540cdec0 2023-12-08T08:39:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.706+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2531562526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a54067430 con 0x7f4a540cdec0 2023-12-08T08:39:27.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.706+0000 7f4a52ffd640 1 -- 172.21.15.16:0/2531562526 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4a4407a340 con 0x7f4a540cdec0 2023-12-08T08:39:27.709 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.709+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4a4006ae40 msgr2=0x7f4a4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.709+0000 7f4a5c7b6640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4a4006ae40 0x7f4a4006d300 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c007f50 tx=0x7f4a4c0074a0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.709+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2531562526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a540cdec0 msgr2=0x7f4a541401b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.709+0000 7f4a5c7b6640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a540cdec0 0x7f4a541401b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4a440090b0 tx=0x7f4a440046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.709+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2531562526 shutdown_connections 2023-12-08T08:39:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.709+0000 7f4a5c7b6640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f4a4006ae40 0x7f4a4006d300 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.709+0000 7f4a5c7b6640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a540cdec0 0x7f4a541401b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.709+0000 7f4a5c7b6640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f4a540cd3d0 0x7f4a5413dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.709+0000 7f4a5c7b6640 1 --2- 172.21.15.16:0/2531562526 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f4a5406d6a0 0x7f4a541345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.709+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2531562526 >> 172.21.15.16:0/2531562526 conn(0x7f4a540b0850 msgr2=0x7f4a5415b520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.709+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2531562526 shutdown_connections 2023-12-08T08:39:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.709+0000 7f4a5c7b6640 1 -- 172.21.15.16:0/2531562526 wait complete. 2023-12-08T08:39:27.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-08T08:39:27.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.829+0000 7f8fd29ce640 1 Processor -- start 2023-12-08T08:39:27.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.829+0000 7f8fd29ce640 1 -- start start 2023-12-08T08:39:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.829+0000 7f8fd29ce640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8fcc0afeb0 0x7f8fcc0cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.830+0000 7f8fd29ce640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8fcc0ae350 0x7f8fcc0ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.830+0000 7f8fd29ce640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc06d650 con 0x7f8fcc15d590 2023-12-08T08:39:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.830+0000 7f8fd29ce640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fcc0adbe0 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.830+0000 7f8fd29ce640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fcc15d970 con 0x7f8fcc0ae350 2023-12-08T08:39:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.830+0000 7f8fcbfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8fcc15d590 0x7f8fcc0ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33308/0 (socket says 172.21.15.16:33308) 2023-12-08T08:39:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.830+0000 7f8fcbfff640 1 -- 172.21.15.16:0/428167178 learned_addr learned my addr 172.21.15.16:0/428167178 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.830+0000 7f8fcb7fe640 1 --2- 172.21.15.16:0/428167178 >> v2:172.21.15.16:3300/0 conn(0x7f8fcc0afeb0 0x7f8fcc0cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.830+0000 7f8fd0f6b640 1 --2- 172.21.15.16:0/428167178 >> v2:172.21.15.112:3300/0 conn(0x7f8fcc0ae350 0x7f8fcc0ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.830+0000 7f8fcb7fe640 1 -- 172.21.15.16:0/428167178 >> v2:172.21.15.112:3300/0 conn(0x7f8fcc0ae350 msgr2=0x7f8fcc0ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.830+0000 7f8fcb7fe640 1 --2- 172.21.15.16:0/428167178 >> v2:172.21.15.112:3300/0 conn(0x7f8fcc0ae350 0x7f8fcc0ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.830+0000 7f8fcb7fe640 1 -- 172.21.15.16:0/428167178 >> v1:172.21.15.16:6789/0 conn(0x7f8fcc15d590 legacy=0x7f8fcc0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.830+0000 7f8fcb7fe640 1 -- 172.21.15.16:0/428167178 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc06c810 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.830+0000 7f8fcaffd640 1 -- 172.21.15.16:0/428167178 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249516799 0 0) 0x7f8fbc001110 con 0x7f8fcc15d590 2023-12-08T08:39:27.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.830+0000 7f8fd0f6b640 1 --2- 172.21.15.16:0/428167178 >> v2:172.21.15.112:3300/0 conn(0x7f8fcc0ae350 0x7f8fcc0ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.831+0000 7f8fcb7fe640 1 --2- 172.21.15.16:0/428167178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fcc0afeb0 0x7f8fcc0cf420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0714d0 tx=0x7f8fc002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a118aaabb079ca82 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.831+0000 7f8fcaffd640 1 -- 172.21.15.16:0/428167178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc002e070 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.831+0000 7f8fcaffd640 1 -- 172.21.15.16:0/428167178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc00315a0 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.831+0000 7f8fcaffd640 1 -- 172.21.15.16:0/428167178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0031930 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.831+0000 7f8fd29ce640 1 -- 172.21.15.16:0/428167178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fcc0afeb0 msgr2=0x7f8fcc0cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.831+0000 7f8fd29ce640 1 --2- 172.21.15.16:0/428167178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fcc0afeb0 0x7f8fcc0cf420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0714d0 tx=0x7f8fc002b840 comp rx=0 tx=0).stop 2023-12-08T08:39:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.831+0000 7f8fd29ce640 1 -- 172.21.15.16:0/428167178 shutdown_connections 2023-12-08T08:39:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.831+0000 7f8fd29ce640 1 --2- 172.21.15.16:0/428167178 >> v2:172.21.15.112:3300/0 conn(0x7f8fcc0ae350 0x7f8fcc0ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.831+0000 7f8fd29ce640 1 --2- 172.21.15.16:0/428167178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fcc0afeb0 0x7f8fcc0cf420 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.831+0000 7f8fd29ce640 1 -- 172.21.15.16:0/428167178 >> 172.21.15.16:0/428167178 conn(0x7f8fcc0d0e10 msgr2=0x7f8fcc0d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.832+0000 7f8fd29ce640 1 -- 172.21.15.16:0/428167178 shutdown_connections 2023-12-08T08:39:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.832+0000 7f8fd29ce640 1 -- 172.21.15.16:0/428167178 wait complete. 2023-12-08T08:39:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.833+0000 7f8fd29ce640 1 Processor -- start 2023-12-08T08:39:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.833+0000 7f8fd29ce640 1 -- start start 2023-12-08T08:39:27.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fd29ce640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fcc0ae350 0x7f8fcc15dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fd29ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fcc0afeb0 0x7f8fcc154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fd29ce640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8fcc15d590 0x7f8fcc15c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fd29ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc1f74d0 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fd29ce640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc1f68c0 con 0x7f8fcc0ae350 2023-12-08T08:39:27.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fd29ce640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc1f6ba0 con 0x7f8fcc15d590 2023-12-08T08:39:27.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fcb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fcc0afeb0 0x7f8fcc154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fcb7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fcc0afeb0 0x7f8fcc154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47610/0 (socket says 172.21.15.16:47610) 2023-12-08T08:39:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fcb7fe640 1 -- 172.21.15.16:0/4144589589 learned_addr learned my addr 172.21.15.16:0/4144589589 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fcb7fe640 1 -- 172.21.15.16:0/4144589589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8fcc15d590 msgr2=0x7f8fcc15c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fcb7fe640 1 --2- 172.21.15.16:0/4144589589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8fcc15d590 0x7f8fcc15c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fcb7fe640 1 -- 172.21.15.16:0/4144589589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fcc0ae350 msgr2=0x7f8fcc15dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fcb7fe640 1 --2- 172.21.15.16:0/4144589589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fcc0ae350 0x7f8fcc15dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fcb7fe640 1 -- 172.21.15.16:0/4144589589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc0009400 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fcb7fe640 1 --2- 172.21.15.16:0/4144589589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fcc0afeb0 0x7f8fcc154350 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0034040 tx=0x7f8fc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fc8ff9640 1 -- 172.21.15.16:0/4144589589 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0004230 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fc8ff9640 1 -- 172.21.15.16:0/4144589589 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc0004390 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fc8ff9640 1 -- 172.21.15.16:0/4144589589 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0032880 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fd29ce640 1 -- 172.21.15.16:0/4144589589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc1f6e80 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.834+0000 7f8fd29ce640 1 -- 172.21.15.16:0/4144589589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc1f6540 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.835+0000 7f8fd29ce640 1 -- 172.21.15.16:0/4144589589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fcc0d1a80 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.836+0000 7f8fc8ff9640 1 -- 172.21.15.16:0/4144589589 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8fc00329e0 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.836+0000 7f8fc8ff9640 1 --2- 172.21.15.16:0/4144589589 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8fb806ae40 0x7f8fb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.836+0000 7f8fc8ff9640 1 -- 172.21.15.16:0/4144589589 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f8fc00abac0 con 0x7f8fcc0afeb0 2023-12-08T08:39:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.836+0000 7f8fcbfff640 1 --2- 172.21.15.16:0/4144589589 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8fb806ae40 0x7f8fb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.837+0000 7f8fcbfff640 1 --2- 172.21.15.16:0/4144589589 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8fb806ae40 0x7f8fb806d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc007fd0 tx=0x7f8fbc007960 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:27.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:27.840+0000 7f8fc8ff9640 1 -- 172.21.15.16:0/4144589589 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8fc0076820 con 0x7f8fcc0afeb0 2023-12-08T08:39:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.017+0000 7f8fd29ce640 1 -- 172.21.15.16:0/4144589589 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8fcc1f2fa0 con 0x7f8fcc0afeb0 2023-12-08T08:39:28.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.018+0000 7f8fc8ff9640 1 -- 172.21.15.16:0/4144589589 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8fc00abe60 con 0x7f8fcc0afeb0 2023-12-08T08:39:28.020 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-12-08T08:39:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.020+0000 7f8fd29ce640 1 -- 172.21.15.16:0/4144589589 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8fb806ae40 msgr2=0x7f8fb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.020+0000 7f8fd29ce640 1 --2- 172.21.15.16:0/4144589589 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8fb806ae40 0x7f8fb806d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc007fd0 tx=0x7f8fbc007960 comp rx=0 tx=0).stop 2023-12-08T08:39:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.020+0000 7f8fd29ce640 1 -- 172.21.15.16:0/4144589589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fcc0afeb0 msgr2=0x7f8fcc154350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.020+0000 7f8fd29ce640 1 --2- 172.21.15.16:0/4144589589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fcc0afeb0 0x7f8fcc154350 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0034040 tx=0x7f8fc00046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.020+0000 7f8fd29ce640 1 -- 172.21.15.16:0/4144589589 shutdown_connections 2023-12-08T08:39:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.020+0000 7f8fd29ce640 1 --2- 172.21.15.16:0/4144589589 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8fb806ae40 0x7f8fb806d300 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.020+0000 7f8fd29ce640 1 --2- 172.21.15.16:0/4144589589 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8fcc15d590 0x7f8fcc15c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.020+0000 7f8fd29ce640 1 --2- 172.21.15.16:0/4144589589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fcc0afeb0 0x7f8fcc154350 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.020+0000 7f8fd29ce640 1 --2- 172.21.15.16:0/4144589589 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8fcc0ae350 0x7f8fcc15dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.020+0000 7f8fd29ce640 1 -- 172.21.15.16:0/4144589589 >> 172.21.15.16:0/4144589589 conn(0x7f8fcc0d0e10 msgr2=0x7f8fcc0cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.020+0000 7f8fd29ce640 1 -- 172.21.15.16:0/4144589589 shutdown_connections 2023-12-08T08:39:28.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.020+0000 7f8fd29ce640 1 -- 172.21.15.16:0/4144589589 wait complete. 2023-12-08T08:39:28.032 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-08T08:39:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.139+0000 7f14277c9640 1 Processor -- start 2023-12-08T08:39:28.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.139+0000 7f14277c9640 1 -- start start 2023-12-08T08:39:28.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.140+0000 7f14277c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14200b4150 0x7f14200b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.140+0000 7f14277c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14200cfe10 0x7f14200cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.140+0000 7f14277c9640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14200c6be0 con 0x7f14200cfa30 2023-12-08T08:39:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.140+0000 7f14277c9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14200d3ff0 con 0x7f14200cfe10 2023-12-08T08:39:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.140+0000 7f14277c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14200b4860 con 0x7f14200b4150 2023-12-08T08:39:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.140+0000 7f1425d66640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14200cfe10 0x7f14200cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.140+0000 7f1425d66640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14200cfe10 0x7f14200cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47624/0 (socket says 172.21.15.16:47624) 2023-12-08T08:39:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.140+0000 7f1425d66640 1 -- 172.21.15.16:0/3561156187 learned_addr learned my addr 172.21.15.16:0/3561156187 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.141+0000 7f1424d64640 1 --2- 172.21.15.16:0/3561156187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14200b4150 0x7f14200b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.141+0000 7f1425d66640 1 -- 172.21.15.16:0/3561156187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14200b4150 msgr2=0x7f14200b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.141+0000 7f1425d66640 1 --2- 172.21.15.16:0/3561156187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14200b4150 0x7f14200b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.141+0000 7f1425d66640 1 -- 172.21.15.16:0/3561156187 >> v1:172.21.15.112:6789/0 conn(0x7f14200cfa30 legacy=0x7f14200b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:28.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.141+0000 7f1425d66640 1 -- 172.21.15.16:0/3561156187 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14200d3d50 con 0x7f14200cfe10 2023-12-08T08:39:28.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.141+0000 7f1425d66640 1 --2- 172.21.15.16:0/3561156187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14200cfe10 0x7f14200cdf50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f141c009e50 tx=0x7f141c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f45c5225e460a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:28.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.142+0000 7f1417fff640 1 -- 172.21.15.16:0/3561156187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c02c9e0 con 0x7f14200cfe10 2023-12-08T08:39:28.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.142+0000 7f1417fff640 1 -- 172.21.15.16:0/3561156187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141c02cb40 con 0x7f14200cfe10 2023-12-08T08:39:28.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.142+0000 7f1417fff640 1 -- 172.21.15.16:0/3561156187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c0343f0 con 0x7f14200cfe10 2023-12-08T08:39:28.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.142+0000 7f14277c9640 1 -- 172.21.15.16:0/3561156187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14200cfe10 msgr2=0x7f14200cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.142+0000 7f14277c9640 1 --2- 172.21.15.16:0/3561156187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14200cfe10 0x7f14200cdf50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f141c009e50 tx=0x7f141c02be40 comp rx=0 tx=0).stop 2023-12-08T08:39:28.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.142+0000 7f14277c9640 1 -- 172.21.15.16:0/3561156187 shutdown_connections 2023-12-08T08:39:28.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.142+0000 7f14277c9640 1 --2- 172.21.15.16:0/3561156187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14200cfe10 0x7f14200cdf50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.142+0000 7f14277c9640 1 --2- 172.21.15.16:0/3561156187 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14200b4150 0x7f14200b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.142+0000 7f14277c9640 1 -- 172.21.15.16:0/3561156187 >> 172.21.15.16:0/3561156187 conn(0x7f14200d3090 msgr2=0x7f14200d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:28.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.142+0000 7f14277c9640 1 -- 172.21.15.16:0/3561156187 shutdown_connections 2023-12-08T08:39:28.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.142+0000 7f14277c9640 1 -- 172.21.15.16:0/3561156187 wait complete. 2023-12-08T08:39:28.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.143+0000 7f14277c9640 1 Processor -- start 2023-12-08T08:39:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.144+0000 7f14277c9640 1 -- start start 2023-12-08T08:39:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.144+0000 7f14277c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14200b4150 0x7f14200cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.144+0000 7f14277c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14200cfa30 0x7f1420152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.144+0000 7f14277c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14200cfe10 0x7f14201553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.144+0000 7f14277c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14200ccc50 con 0x7f14200cfe10 2023-12-08T08:39:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.144+0000 7f14277c9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14200b7220 con 0x7f14200cfa30 2023-12-08T08:39:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.144+0000 7f14277c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14200b7780 con 0x7f14200b4150 2023-12-08T08:39:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.144+0000 7f1425d66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14200cfe10 0x7f14201553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.144+0000 7f1425d66640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14200cfe10 0x7f14201553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47626/0 (socket says 172.21.15.16:47626) 2023-12-08T08:39:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.144+0000 7f1425d66640 1 -- 172.21.15.16:0/1714024890 learned_addr learned my addr 172.21.15.16:0/1714024890 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:28.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.144+0000 7f1424d64640 1 --2- 172.21.15.16:0/1714024890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14200cfa30 0x7f1420152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.144+0000 7f1425565640 1 --2- 172.21.15.16:0/1714024890 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14200b4150 0x7f14200cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.145+0000 7f1425d66640 1 -- 172.21.15.16:0/1714024890 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14200b4150 msgr2=0x7f14200cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.145+0000 7f1425d66640 1 --2- 172.21.15.16:0/1714024890 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14200b4150 0x7f14200cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.145+0000 7f1425d66640 1 -- 172.21.15.16:0/1714024890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14200cfa30 msgr2=0x7f1420152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.145+0000 7f1425d66640 1 --2- 172.21.15.16:0/1714024890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14200cfa30 0x7f1420152490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.145+0000 7f1425d66640 1 -- 172.21.15.16:0/1714024890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c009b30 con 0x7f14200cfe10 2023-12-08T08:39:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.145+0000 7f1425d66640 1 --2- 172.21.15.16:0/1714024890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14200cfe10 0x7f14201553c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f141c036040 tx=0x7f141c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.145+0000 7f1415ffb640 1 -- 172.21.15.16:0/1714024890 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c009510 con 0x7f14200cfe10 2023-12-08T08:39:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.145+0000 7f1415ffb640 1 -- 172.21.15.16:0/1714024890 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141c03e450 con 0x7f14200cfe10 2023-12-08T08:39:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.145+0000 7f14277c9640 1 -- 172.21.15.16:0/1714024890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14200cc690 con 0x7f14200cfe10 2023-12-08T08:39:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.145+0000 7f1415ffb640 1 -- 172.21.15.16:0/1714024890 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c03e7e0 con 0x7f14200cfe10 2023-12-08T08:39:28.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.145+0000 7f14277c9640 1 -- 172.21.15.16:0/1714024890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14200b7410 con 0x7f14200cfe10 2023-12-08T08:39:28.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.146+0000 7f14277c9640 1 -- 172.21.15.16:0/1714024890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14200b7930 con 0x7f14200cfe10 2023-12-08T08:39:28.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.147+0000 7f1415ffb640 1 -- 172.21.15.16:0/1714024890 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f141c03e940 con 0x7f14200cfe10 2023-12-08T08:39:28.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.147+0000 7f1415ffb640 1 --2- 172.21.15.16:0/1714024890 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f140406ae40 0x7f140406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.147+0000 7f1415ffb640 1 -- 172.21.15.16:0/1714024890 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f141c0ac700 con 0x7f14200cfe10 2023-12-08T08:39:28.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.147+0000 7f1425565640 1 --2- 172.21.15.16:0/1714024890 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f140406ae40 0x7f140406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.148+0000 7f1425565640 1 --2- 172.21.15.16:0/1714024890 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f140406ae40 0x7f140406d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1418006fd0 tx=0x7f1418009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:28.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.151+0000 7f1415ffb640 1 -- 172.21.15.16:0/1714024890 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14200b7930 con 0x7f14200cfe10 2023-12-08T08:39:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.326+0000 7f14277c9640 1 -- 172.21.15.16:0/1714024890 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f14200b78f0 con 0x7f14200cfe10 2023-12-08T08:39:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.326+0000 7f1415ffb640 1 -- 172.21.15.16:0/1714024890 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f141c043020 con 0x7f14200cfe10 2023-12-08T08:39:28.329 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-12-08T08:39:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.329+0000 7f14277c9640 1 -- 172.21.15.16:0/1714024890 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f140406ae40 msgr2=0x7f140406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.329+0000 7f14277c9640 1 --2- 172.21.15.16:0/1714024890 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f140406ae40 0x7f140406d300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1418006fd0 tx=0x7f1418009060 comp rx=0 tx=0).stop 2023-12-08T08:39:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.329+0000 7f14277c9640 1 -- 172.21.15.16:0/1714024890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14200cfe10 msgr2=0x7f14201553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.329+0000 7f14277c9640 1 --2- 172.21.15.16:0/1714024890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14200cfe10 0x7f14201553c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f141c036040 tx=0x7f141c004a40 comp rx=0 tx=0).stop 2023-12-08T08:39:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.329+0000 7f14277c9640 1 -- 172.21.15.16:0/1714024890 shutdown_connections 2023-12-08T08:39:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.329+0000 7f14277c9640 1 --2- 172.21.15.16:0/1714024890 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f140406ae40 0x7f140406d300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.329+0000 7f14277c9640 1 --2- 172.21.15.16:0/1714024890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14200cfe10 0x7f14201553c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.329+0000 7f14277c9640 1 --2- 172.21.15.16:0/1714024890 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f14200cfa30 0x7f1420152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.329+0000 7f14277c9640 1 --2- 172.21.15.16:0/1714024890 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f14200b4150 0x7f14200cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.329+0000 7f14277c9640 1 -- 172.21.15.16:0/1714024890 >> 172.21.15.16:0/1714024890 conn(0x7f14200d3090 msgr2=0x7f14200c7070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:28.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.329+0000 7f14277c9640 1 -- 172.21.15.16:0/1714024890 shutdown_connections 2023-12-08T08:39:28.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.329+0000 7f14277c9640 1 -- 172.21.15.16:0/1714024890 wait complete. 2023-12-08T08:39:28.341 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-08T08:39:28.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.445+0000 7ff207773640 1 Processor -- start 2023-12-08T08:39:28.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.445+0000 7ff207773640 1 -- start start 2023-12-08T08:39:28.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.446+0000 7ff207773640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2000b6ac0 0x7ff2000cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.446+0000 7ff207773640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2000cd660 0x7ff2000cda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.446+0000 7ff207773640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff2000cdfc0 0x7ff20006b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.446+0000 7ff207773640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff200160cb0 con 0x7ff2000b6ac0 2023-12-08T08:39:28.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.446+0000 7ff207773640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2000d0070 con 0x7ff2000cdfc0 2023-12-08T08:39:28.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.446+0000 7ff207773640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2000d02b0 con 0x7ff2000cd660 2023-12-08T08:39:28.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.446+0000 7ff20550f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2000b6ac0 0x7ff2000cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.446+0000 7ff20550f640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2000b6ac0 0x7ff2000cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47634/0 (socket says 172.21.15.16:47634) 2023-12-08T08:39:28.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.446+0000 7ff20550f640 1 -- 172.21.15.16:0/2482097495 learned_addr learned my addr 172.21.15.16:0/2482097495 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:28.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.447+0000 7ff204d0e640 1 --2- 172.21.15.16:0/2482097495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2000cd660 0x7ff2000cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.447+0000 7ff205d10640 1 --2- 172.21.15.16:0/2482097495 >> v2:172.21.15.112:3300/0 conn(0x7ff2000cdfc0 0x7ff20006b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.447+0000 7ff20550f640 1 -- 172.21.15.16:0/2482097495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2000cd660 msgr2=0x7ff2000cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:28.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.447+0000 7ff20550f640 1 --2- 172.21.15.16:0/2482097495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2000cd660 0x7ff2000cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.447+0000 7ff20550f640 1 -- 172.21.15.16:0/2482097495 >> v2:172.21.15.112:3300/0 conn(0x7ff2000cdfc0 msgr2=0x7ff20006b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.447+0000 7ff20550f640 1 --2- 172.21.15.16:0/2482097495 >> v2:172.21.15.112:3300/0 conn(0x7ff2000cdfc0 0x7ff20006b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.447+0000 7ff20550f640 1 -- 172.21.15.16:0/2482097495 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2000cb430 con 0x7ff2000b6ac0 2023-12-08T08:39:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.447+0000 7ff205d10640 1 --2- 172.21.15.16:0/2482097495 >> v2:172.21.15.112:3300/0 conn(0x7ff2000cdfc0 0x7ff20006b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.447+0000 7ff20550f640 1 --2- 172.21.15.16:0/2482097495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000b6ac0 0x7ff2000cfb30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff20006e0d0 tx=0x7ff1fc02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96fc1a9ca6bcb3 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.448+0000 7ff1f7fff640 1 -- 172.21.15.16:0/2482097495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc031750 con 0x7ff2000b6ac0 2023-12-08T08:39:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.448+0000 7ff1f7fff640 1 -- 172.21.15.16:0/2482097495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1fc0318b0 con 0x7ff2000b6ac0 2023-12-08T08:39:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.448+0000 7ff1f7fff640 1 -- 172.21.15.16:0/2482097495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc031c40 con 0x7ff2000b6ac0 2023-12-08T08:39:28.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.448+0000 7ff207773640 1 -- 172.21.15.16:0/2482097495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000b6ac0 msgr2=0x7ff2000cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.448+0000 7ff207773640 1 --2- 172.21.15.16:0/2482097495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000b6ac0 0x7ff2000cfb30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff20006e0d0 tx=0x7ff1fc02b7f0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.448+0000 7ff207773640 1 -- 172.21.15.16:0/2482097495 shutdown_connections 2023-12-08T08:39:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.448+0000 7ff207773640 1 --2- 172.21.15.16:0/2482097495 >> v2:172.21.15.112:3300/0 conn(0x7ff2000cdfc0 0x7ff20006b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.448+0000 7ff207773640 1 --2- 172.21.15.16:0/2482097495 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2000cd660 0x7ff2000cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.448+0000 7ff207773640 1 --2- 172.21.15.16:0/2482097495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000b6ac0 0x7ff2000cfb30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.448+0000 7ff207773640 1 -- 172.21.15.16:0/2482097495 >> 172.21.15.16:0/2482097495 conn(0x7ff2000d0de0 msgr2=0x7ff2000d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.448+0000 7ff207773640 1 -- 172.21.15.16:0/2482097495 shutdown_connections 2023-12-08T08:39:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.448+0000 7ff207773640 1 -- 172.21.15.16:0/2482097495 wait complete. 2023-12-08T08:39:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.449+0000 7ff207773640 1 Processor -- start 2023-12-08T08:39:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.449+0000 7ff207773640 1 -- start start 2023-12-08T08:39:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.450+0000 7ff207773640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2000b6ac0 0x7ff2000b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.450+0000 7ff207773640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000cd660 0x7ff20015d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.450+0000 7ff207773640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2000cdfc0 0x7ff20009c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.450+0000 7ff207773640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2000b11a0 con 0x7ff2000cd660 2023-12-08T08:39:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.450+0000 7ff207773640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20009c2b0 con 0x7ff2000cdfc0 2023-12-08T08:39:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.450+0000 7ff207773640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20015d980 con 0x7ff2000b6ac0 2023-12-08T08:39:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.450+0000 7ff204d0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000cd660 0x7ff20015d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.450+0000 7ff204d0e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000cd660 0x7ff20015d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47638/0 (socket says 172.21.15.16:47638) 2023-12-08T08:39:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.450+0000 7ff204d0e640 1 -- 172.21.15.16:0/203753343 learned_addr learned my addr 172.21.15.16:0/203753343 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.450+0000 7ff20550f640 1 --2- 172.21.15.16:0/203753343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2000b6ac0 0x7ff2000b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.450+0000 7ff205d10640 1 --2- 172.21.15.16:0/203753343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2000cdfc0 0x7ff20009c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.451+0000 7ff204d0e640 1 -- 172.21.15.16:0/203753343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2000b6ac0 msgr2=0x7ff2000b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.451+0000 7ff204d0e640 1 --2- 172.21.15.16:0/203753343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2000b6ac0 0x7ff2000b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.451+0000 7ff204d0e640 1 -- 172.21.15.16:0/203753343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2000cdfc0 msgr2=0x7ff20009c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.451+0000 7ff204d0e640 1 --2- 172.21.15.16:0/203753343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2000cdfc0 0x7ff20009c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.451+0000 7ff204d0e640 1 -- 172.21.15.16:0/203753343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1fc0094d0 con 0x7ff2000cd660 2023-12-08T08:39:28.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.451+0000 7ff204d0e640 1 --2- 172.21.15.16:0/203753343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000cd660 0x7ff20015d440 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000d830 tx=0x7ff1f000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:28.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.451+0000 7ff1f5ffb640 1 -- 172.21.15.16:0/203753343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f000be10 con 0x7ff2000cd660 2023-12-08T08:39:28.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.451+0000 7ff1f5ffb640 1 -- 172.21.15.16:0/203753343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f00098b0 con 0x7ff2000cd660 2023-12-08T08:39:28.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.451+0000 7ff207773640 1 -- 172.21.15.16:0/203753343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff20009b6a0 con 0x7ff2000cd660 2023-12-08T08:39:28.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.451+0000 7ff1f5ffb640 1 -- 172.21.15.16:0/203753343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0009a40 con 0x7ff2000cd660 2023-12-08T08:39:28.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.451+0000 7ff207773640 1 -- 172.21.15.16:0/203753343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff20009bc60 con 0x7ff2000cd660 2023-12-08T08:39:28.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.452+0000 7ff207773640 1 -- 172.21.15.16:0/203753343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff200067430 con 0x7ff2000cd660 2023-12-08T08:39:28.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.453+0000 7ff1f5ffb640 1 -- 172.21.15.16:0/203753343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff1f0009ba0 con 0x7ff2000cd660 2023-12-08T08:39:28.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.453+0000 7ff1f5ffb640 1 --2- 172.21.15.16:0/203753343 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff1e406ae40 0x7ff1e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.453+0000 7ff1f5ffb640 1 -- 172.21.15.16:0/203753343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7ff1f0089be0 con 0x7ff2000cd660 2023-12-08T08:39:28.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.453+0000 7ff20550f640 1 --2- 172.21.15.16:0/203753343 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff1e406ae40 0x7ff1e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.454+0000 7ff20550f640 1 --2- 172.21.15.16:0/203753343 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff1e406ae40 0x7ff1e406d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc035000 tx=0x7ff1fc008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.458+0000 7ff1f5ffb640 1 -- 172.21.15.16:0/203753343 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff1f00549f0 con 0x7ff2000cd660 2023-12-08T08:39:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.669+0000 7ff207773640 1 -- 172.21.15.16:0/203753343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff20009c3f0 con 0x7ff2000cd660 2023-12-08T08:39:28.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.669+0000 7ff1f5ffb640 1 -- 172.21.15.16:0/203753343 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7ff1f0058010 con 0x7ff2000cd660 2023-12-08T08:39:28.672 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:28.672 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/307039451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":307039451},{"type":"v1","addr":"172.21.15.112:6836","nonce":307039451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1424152341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1424152341},{"type":"v1","addr":"172.21.15.16:6833","nonce":1424152341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3597811670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3597811670},{"type":"v1","addr":"172.21.15.16:6835","nonce":3597811670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-08T08:38:18.991090+0000","modified":"2023-12-08T08:38:41.075742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1082241707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1082241707},{"type":"v1","addr":"172.21.15.112:6837","nonce":1082241707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-08T08:39:28.672 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-12-08T08:39:28.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.671+0000 7ff207773640 1 -- 172.21.15.16:0/203753343 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff1e406ae40 msgr2=0x7ff1e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.671+0000 7ff207773640 1 --2- 172.21.15.16:0/203753343 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff1e406ae40 0x7ff1e406d300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc035000 tx=0x7ff1fc008030 comp rx=0 tx=0).stop 2023-12-08T08:39:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.671+0000 7ff207773640 1 -- 172.21.15.16:0/203753343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000cd660 msgr2=0x7ff20015d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.671+0000 7ff207773640 1 --2- 172.21.15.16:0/203753343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000cd660 0x7ff20015d440 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000d830 tx=0x7ff1f000dcf0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.671+0000 7ff207773640 1 -- 172.21.15.16:0/203753343 shutdown_connections 2023-12-08T08:39:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.671+0000 7ff207773640 1 --2- 172.21.15.16:0/203753343 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff1e406ae40 0x7ff1e406d300 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.671+0000 7ff207773640 1 --2- 172.21.15.16:0/203753343 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff2000cdfc0 0x7ff20009c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.671+0000 7ff207773640 1 --2- 172.21.15.16:0/203753343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2000cd660 0x7ff20015d440 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.671+0000 7ff207773640 1 --2- 172.21.15.16:0/203753343 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff2000b6ac0 0x7ff2000b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.671+0000 7ff207773640 1 -- 172.21.15.16:0/203753343 >> 172.21.15.16:0/203753343 conn(0x7ff2000d0de0 msgr2=0x7ff20006e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:28.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.671+0000 7ff207773640 1 -- 172.21.15.16:0/203753343 shutdown_connections 2023-12-08T08:39:28.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.671+0000 7ff207773640 1 -- 172.21.15.16:0/203753343 wait complete. 2023-12-08T08:39:28.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:28.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.789+0000 7f2b1e3c5640 1 Processor -- start 2023-12-08T08:39:28.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.789+0000 7f2b1e3c5640 1 -- start start 2023-12-08T08:39:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.790+0000 7f2b1e3c5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2b18158ee0 0x7f2b18070e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.790+0000 7f2b1e3c5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b1815b750 0x7f2b180cb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.790+0000 7f2b1e3c5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b180720d0 con 0x7f2b180cbb40 2023-12-08T08:39:28.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.790+0000 7f2b1e3c5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b180cc890 con 0x7f2b18158ee0 2023-12-08T08:39:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.790+0000 7f2b1e3c5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b180cbfd0 con 0x7f2b1815b750 2023-12-08T08:39:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.790+0000 7f2b177fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b1815b750 0x7f2b180cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.790+0000 7f2b17fff640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2b18158ee0 0x7f2b18070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.790+0000 7f2b177fe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b1815b750 0x7f2b180cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:55070/0 (socket says 172.21.15.16:55070) 2023-12-08T08:39:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.790+0000 7f2b177fe640 1 -- 172.21.15.16:0/175255793 learned_addr learned my addr 172.21.15.16:0/175255793 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.790+0000 7f2b1c962640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f2b180cbb40 0x7f2b1815ed20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:51706/0 (socket says 172.21.15.16:51706) 2023-12-08T08:39:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.790+0000 7f2b17fff640 1 -- 172.21.15.16:0/175255793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b1815b750 msgr2=0x7f2b180cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.790+0000 7f2b17fff640 1 --2- 172.21.15.16:0/175255793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b1815b750 0x7f2b180cb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.791+0000 7f2b17fff640 1 -- 172.21.15.16:0/175255793 >> v1:172.21.15.112:6789/0 conn(0x7f2b180cbb40 legacy=0x7f2b1815ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.791+0000 7f2b17fff640 1 -- 172.21.15.16:0/175255793 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b180714c0 con 0x7f2b18158ee0 2023-12-08T08:39:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.791+0000 7f2b17fff640 1 --2- 172.21.15.16:0/175255793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b18158ee0 0x7f2b18070e30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0097a0 tx=0x7f2b0c02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b5fa42e3aab59cd server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.791+0000 7f2b16ffd640 1 -- 172.21.15.16:0/175255793 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c031780 con 0x7f2b18158ee0 2023-12-08T08:39:28.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.792+0000 7f2b16ffd640 1 -- 172.21.15.16:0/175255793 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c0318e0 con 0x7f2b18158ee0 2023-12-08T08:39:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.792+0000 7f2b16ffd640 1 -- 172.21.15.16:0/175255793 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c031c70 con 0x7f2b18158ee0 2023-12-08T08:39:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.792+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/175255793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b18158ee0 msgr2=0x7f2b18070e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.792+0000 7f2b1e3c5640 1 --2- 172.21.15.16:0/175255793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b18158ee0 0x7f2b18070e30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0097a0 tx=0x7f2b0c02b820 comp rx=0 tx=0).stop 2023-12-08T08:39:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.792+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/175255793 shutdown_connections 2023-12-08T08:39:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.792+0000 7f2b1e3c5640 1 --2- 172.21.15.16:0/175255793 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b1815b750 0x7f2b180cb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.792+0000 7f2b1e3c5640 1 --2- 172.21.15.16:0/175255793 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b18158ee0 0x7f2b18070e30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.792+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/175255793 >> 172.21.15.16:0/175255793 conn(0x7f2b180b09f0 msgr2=0x7f2b180b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.792+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/175255793 shutdown_connections 2023-12-08T08:39:28.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.792+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/175255793 wait complete. 2023-12-08T08:39:28.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.793+0000 7f2b1e3c5640 1 Processor -- start 2023-12-08T08:39:28.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.793+0000 7f2b1e3c5640 1 -- start start 2023-12-08T08:39:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.794+0000 7f2b1e3c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b180cbb40 0x7f2b1809b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.794+0000 7f2b1e3c5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b18158ee0 0x7f2b18150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.794+0000 7f2b1e3c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b1815b750 0x7f2b180ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.794+0000 7f2b1e3c5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b18099f90 con 0x7f2b180cbb40 2023-12-08T08:39:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.794+0000 7f2b1e3c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b180cebf0 con 0x7f2b1815b750 2023-12-08T08:39:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.794+0000 7f2b1e3c5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1809bef0 con 0x7f2b18158ee0 2023-12-08T08:39:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.794+0000 7f2b17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b180cbb40 0x7f2b1809b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.794+0000 7f2b17fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b180cbb40 0x7f2b1809b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47642/0 (socket says 172.21.15.16:47642) 2023-12-08T08:39:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.794+0000 7f2b17fff640 1 -- 172.21.15.16:0/2964081115 learned_addr learned my addr 172.21.15.16:0/2964081115 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.794+0000 7f2b177fe640 1 --2- 172.21.15.16:0/2964081115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b18158ee0 0x7f2b18150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.794+0000 7f2b1c962640 1 --2- 172.21.15.16:0/2964081115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b1815b750 0x7f2b180ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.795+0000 7f2b17fff640 1 -- 172.21.15.16:0/2964081115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b18158ee0 msgr2=0x7f2b18150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.795+0000 7f2b17fff640 1 --2- 172.21.15.16:0/2964081115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b18158ee0 0x7f2b18150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.795+0000 7f2b17fff640 1 -- 172.21.15.16:0/2964081115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b1815b750 msgr2=0x7f2b180ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.795+0000 7f2b17fff640 1 --2- 172.21.15.16:0/2964081115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b1815b750 0x7f2b180ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.795+0000 7f2b17fff640 1 -- 172.21.15.16:0/2964081115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0c009400 con 0x7f2b180cbb40 2023-12-08T08:39:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.795+0000 7f2b17fff640 1 --2- 172.21.15.16:0/2964081115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b180cbb40 0x7f2b1809b2a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c034040 tx=0x7f2b0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.795+0000 7f2b14ff9640 1 -- 172.21.15.16:0/2964081115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c032820 con 0x7f2b180cbb40 2023-12-08T08:39:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.795+0000 7f2b14ff9640 1 -- 172.21.15.16:0/2964081115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c0329a0 con 0x7f2b180cbb40 2023-12-08T08:39:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.795+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/2964081115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1809b930 con 0x7f2b180cbb40 2023-12-08T08:39:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.795+0000 7f2b14ff9640 1 -- 172.21.15.16:0/2964081115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c032d50 con 0x7f2b180cbb40 2023-12-08T08:39:28.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.795+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/2964081115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b18099bb0 con 0x7f2b180cbb40 2023-12-08T08:39:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.796+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/2964081115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b18099d40 con 0x7f2b180cbb40 2023-12-08T08:39:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.797+0000 7f2b14ff9640 1 -- 172.21.15.16:0/2964081115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b0c04d3f0 con 0x7f2b180cbb40 2023-12-08T08:39:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.797+0000 7f2b14ff9640 1 --2- 172.21.15.16:0/2964081115 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2b0406ae40 0x7f2b0406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.797+0000 7f2b14ff9640 1 -- 172.21.15.16:0/2964081115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f2b0c0ab4a0 con 0x7f2b180cbb40 2023-12-08T08:39:28.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.797+0000 7f2b177fe640 1 --2- 172.21.15.16:0/2964081115 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2b0406ae40 0x7f2b0406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:28.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.798+0000 7f2b177fe640 1 --2- 172.21.15.16:0/2964081115 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2b0406ae40 0x7f2b0406d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2b1815e8e0 tx=0x7f2b08009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.801+0000 7f2b14ff9640 1 -- 172.21.15.16:0/2964081115 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b0c076220 con 0x7f2b180cbb40 2023-12-08T08:39:28.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.981+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/2964081115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b18099a10 con 0x7f2b180cbb40 2023-12-08T08:39:28.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.982+0000 7f2b14ff9640 1 -- 172.21.15.16:0/2964081115 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7f2b18099a10 con 0x7f2b180cbb40 2023-12-08T08:39:28.985 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:28.985 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:28.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.984+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/2964081115 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2b0406ae40 msgr2=0x7f2b0406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.984+0000 7f2b1e3c5640 1 --2- 172.21.15.16:0/2964081115 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2b0406ae40 0x7f2b0406d300 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2b1815e8e0 tx=0x7f2b08009240 comp rx=0 tx=0).stop 2023-12-08T08:39:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.984+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/2964081115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b180cbb40 msgr2=0x7f2b1809b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.984+0000 7f2b1e3c5640 1 --2- 172.21.15.16:0/2964081115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b180cbb40 0x7f2b1809b2a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c034040 tx=0x7f2b0c0046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.984+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/2964081115 shutdown_connections 2023-12-08T08:39:28.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.984+0000 7f2b1e3c5640 1 --2- 172.21.15.16:0/2964081115 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2b0406ae40 0x7f2b0406d300 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.984+0000 7f2b1e3c5640 1 --2- 172.21.15.16:0/2964081115 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2b1815b750 0x7f2b180ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.984+0000 7f2b1e3c5640 1 --2- 172.21.15.16:0/2964081115 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2b18158ee0 0x7f2b18150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.984+0000 7f2b1e3c5640 1 --2- 172.21.15.16:0/2964081115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b180cbb40 0x7f2b1809b2a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.984+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/2964081115 >> 172.21.15.16:0/2964081115 conn(0x7f2b180b09f0 msgr2=0x7f2b1815a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.985+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/2964081115 shutdown_connections 2023-12-08T08:39:28.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:28.985+0000 7f2b1e3c5640 1 -- 172.21.15.16:0/2964081115 wait complete. 2023-12-08T08:39:28.999 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-08T08:39:28.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-08T08:39:29.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.104+0000 7f41351ec640 1 Processor -- start 2023-12-08T08:39:29.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.104+0000 7f41351ec640 1 -- start start 2023-12-08T08:39:29.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.105+0000 7f41351ec640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f41301608a0 0x7f41300d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.105+0000 7f41351ec640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41300b0cf0 0x7f41300cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.105+0000 7f41351ec640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41300d20b0 con 0x7f41300b1d70 2023-12-08T08:39:29.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.105+0000 7f41351ec640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41300d0620 con 0x7f41301608a0 2023-12-08T08:39:29.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.105+0000 7f41351ec640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41300d1b80 con 0x7f41300b0cf0 2023-12-08T08:39:29.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.106+0000 7f412f59e640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41300b0cf0 0x7f41300cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.106+0000 7f412e59c640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7f41300b1d70 0x7f41300d1470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:51720/0 (socket says 172.21.15.16:51720) 2023-12-08T08:39:29.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.106+0000 7f412e59c640 1 -- 172.21.15.16:0/1301302699 learned_addr learned my addr 172.21.15.16:0/1301302699 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:29.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.106+0000 7f412ed9d640 1 --2- 172.21.15.16:0/1301302699 >> v2:172.21.15.112:3300/0 conn(0x7f41301608a0 0x7f41300d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.106+0000 7f412dd9b640 1 -- 172.21.15.16:0/1301302699 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474259021 0 0) 0x7f41300d20b0 con 0x7f41300b1d70 2023-12-08T08:39:29.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.106+0000 7f412dd9b640 1 -- 172.21.15.16:0/1301302699 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4118003520 con 0x7f41300b1d70 2023-12-08T08:39:29.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.106+0000 7f412f59e640 1 -- 172.21.15.16:0/1301302699 >> v2:172.21.15.112:3300/0 conn(0x7f41301608a0 msgr2=0x7f41300d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:29.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.106+0000 7f412f59e640 1 --2- 172.21.15.16:0/1301302699 >> v2:172.21.15.112:3300/0 conn(0x7f41301608a0 0x7f41300d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.106+0000 7f412f59e640 1 -- 172.21.15.16:0/1301302699 >> v1:172.21.15.112:6789/0 conn(0x7f41300b1d70 legacy=0x7f41300d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.106+0000 7f412f59e640 1 -- 172.21.15.16:0/1301302699 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41300d1dd0 con 0x7f41300b0cf0 2023-12-08T08:39:29.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.107+0000 7f412f59e640 1 --2- 172.21.15.16:0/1301302699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41300b0cf0 0x7f41300cdc40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4128009d90 tx=0x7f412802be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ab05fa277ec2980 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.107+0000 7f412dd9b640 1 -- 172.21.15.16:0/1301302699 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412802cbc0 con 0x7f41300b0cf0 2023-12-08T08:39:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.107+0000 7f412dd9b640 1 -- 172.21.15.16:0/1301302699 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412802cd20 con 0x7f41300b0cf0 2023-12-08T08:39:29.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.107+0000 7f412dd9b640 1 -- 172.21.15.16:0/1301302699 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128034660 con 0x7f41300b0cf0 2023-12-08T08:39:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.108+0000 7f41351ec640 1 -- 172.21.15.16:0/1301302699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41300b0cf0 msgr2=0x7f41300cdc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.108+0000 7f41351ec640 1 --2- 172.21.15.16:0/1301302699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41300b0cf0 0x7f41300cdc40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4128009d90 tx=0x7f412802be10 comp rx=0 tx=0).stop 2023-12-08T08:39:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.108+0000 7f41351ec640 1 -- 172.21.15.16:0/1301302699 shutdown_connections 2023-12-08T08:39:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.108+0000 7f41351ec640 1 --2- 172.21.15.16:0/1301302699 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41300b0cf0 0x7f41300cdc40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.108+0000 7f41351ec640 1 --2- 172.21.15.16:0/1301302699 >> v2:172.21.15.112:3300/0 conn(0x7f41301608a0 0x7f41300d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.108+0000 7f41351ec640 1 -- 172.21.15.16:0/1301302699 >> 172.21.15.16:0/1301302699 conn(0x7f41300d3030 msgr2=0x7f41300d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.108+0000 7f41351ec640 1 -- 172.21.15.16:0/1301302699 shutdown_connections 2023-12-08T08:39:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.108+0000 7f41351ec640 1 -- 172.21.15.16:0/1301302699 wait complete. 2023-12-08T08:39:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.109+0000 7f41351ec640 1 Processor -- start 2023-12-08T08:39:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.109+0000 7f41351ec640 1 -- start start 2023-12-08T08:39:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f41351ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41300b0cf0 0x7f41300cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f41351ec640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41300b1d70 0x7f41300cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f41351ec640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41301608a0 0x7f41300c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f41351ec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41300cb2f0 con 0x7f41300b0cf0 2023-12-08T08:39:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f41351ec640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41300cb5d0 con 0x7f41301608a0 2023-12-08T08:39:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f41351ec640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41300cb8b0 con 0x7f41300b1d70 2023-12-08T08:39:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f412ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41300b0cf0 0x7f41300cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f412ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41300b0cf0 0x7f41300cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47652/0 (socket says 172.21.15.16:47652) 2023-12-08T08:39:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f412ed9d640 1 -- 172.21.15.16:0/3998451545 learned_addr learned my addr 172.21.15.16:0/3998451545 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f412e59c640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41300b1d70 0x7f41300cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f412f59e640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41301608a0 0x7f41300c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f412ed9d640 1 -- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41300b1d70 msgr2=0x7f41300cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f412ed9d640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41300b1d70 0x7f41300cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f412ed9d640 1 -- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41301608a0 msgr2=0x7f41300c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f412ed9d640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41301608a0 0x7f41300c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.110+0000 7f412ed9d640 1 -- 172.21.15.16:0/3998451545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4128009a70 con 0x7f41300b0cf0 2023-12-08T08:39:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.111+0000 7f412f59e640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41301608a0 0x7f41300c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.111+0000 7f412ed9d640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41300b0cf0 0x7f41300cc370 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f412000e870 tx=0x7f412000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.111+0000 7f41177fe640 1 -- 172.21.15.16:0/3998451545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41200094f0 con 0x7f41300b0cf0 2023-12-08T08:39:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.111+0000 7f41177fe640 1 -- 172.21.15.16:0/3998451545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412000c940 con 0x7f41300b0cf0 2023-12-08T08:39:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.111+0000 7f41351ec640 1 -- 172.21.15.16:0/3998451545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41300cbb90 con 0x7f41300b0cf0 2023-12-08T08:39:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.111+0000 7f41177fe640 1 -- 172.21.15.16:0/3998451545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412000ccf0 con 0x7f41300b0cf0 2023-12-08T08:39:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.111+0000 7f41351ec640 1 -- 172.21.15.16:0/3998451545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41300c90a0 con 0x7f41300b0cf0 2023-12-08T08:39:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.112+0000 7f41351ec640 1 -- 172.21.15.16:0/3998451545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4130067430 con 0x7f41300b0cf0 2023-12-08T08:39:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.112+0000 7f41177fe640 1 -- 172.21.15.16:0/3998451545 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4120013470 con 0x7f41300b0cf0 2023-12-08T08:39:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.113+0000 7f41177fe640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f411c06ae40 0x7f411c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.113+0000 7f41177fe640 1 -- 172.21.15.16:0/3998451545 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f4120089b10 con 0x7f41300b0cf0 2023-12-08T08:39:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.113+0000 7f412e59c640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f411c06ae40 0x7f411c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.114+0000 7f412e59c640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f411c06ae40 0x7f411c06d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f413006b800 tx=0x7f4124009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:29.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.117+0000 7f41177fe640 1 -- 172.21.15.16:0/3998451545 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f41200547e0 con 0x7f41300b0cf0 2023-12-08T08:39:29.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.328+0000 7f41351ec640 1 -- 172.21.15.16:0/3998451545 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f413006bd00 con 0x7f41300b0cf0 2023-12-08T08:39:29.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.329+0000 7f41177fe640 1 -- 172.21.15.16:0/3998451545 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f4120017090 con 0x7f41300b0cf0 2023-12-08T08:39:29.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:29.331 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/307039451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":307039451},{"type":"v1","addr":"172.21.15.112:6836","nonce":307039451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1424152341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1424152341},{"type":"v1","addr":"172.21.15.16:6833","nonce":1424152341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3597811670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3597811670},{"type":"v1","addr":"172.21.15.16:6835","nonce":3597811670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-08T08:38:18.991090+0000","modified":"2023-12-08T08:38:41.075742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1082241707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1082241707},{"type":"v1","addr":"172.21.15.112:6837","nonce":1082241707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-08T08:39:29.331 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-12-08T08:39:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.331+0000 7f41351ec640 1 -- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f411c06ae40 msgr2=0x7f411c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.331+0000 7f41351ec640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f411c06ae40 0x7f411c06d300 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f413006b800 tx=0x7f4124009040 comp rx=0 tx=0).stop 2023-12-08T08:39:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.331+0000 7f41351ec640 1 -- 172.21.15.16:0/3998451545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41300b0cf0 msgr2=0x7f41300cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.331+0000 7f41351ec640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41300b0cf0 0x7f41300cc370 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f412000e870 tx=0x7f412000ed30 comp rx=0 tx=0).stop 2023-12-08T08:39:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.331+0000 7f41351ec640 1 -- 172.21.15.16:0/3998451545 shutdown_connections 2023-12-08T08:39:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.331+0000 7f41351ec640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f411c06ae40 0x7f411c06d300 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.331+0000 7f41351ec640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f41301608a0 0x7f41300c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.331+0000 7f41351ec640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f41300b1d70 0x7f41300cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.331+0000 7f41351ec640 1 --2- 172.21.15.16:0/3998451545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f41300b0cf0 0x7f41300cc370 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.331+0000 7f41351ec640 1 -- 172.21.15.16:0/3998451545 >> 172.21.15.16:0/3998451545 conn(0x7f41300d3030 msgr2=0x7f41300d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:29.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.331+0000 7f41351ec640 1 -- 172.21.15.16:0/3998451545 shutdown_connections 2023-12-08T08:39:29.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.331+0000 7f41351ec640 1 -- 172.21.15.16:0/3998451545 wait complete. 2023-12-08T08:39:29.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-08T08:39:29.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.448+0000 7f98c86ad640 1 Processor -- start 2023-12-08T08:39:29.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.448+0000 7f98c86ad640 1 -- start start 2023-12-08T08:39:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.449+0000 7f98c86ad640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98c00ce3e0 0x7f98c00b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.449+0000 7f98c86ad640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f98c0071e50 0x7f98c00b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.449+0000 7f98c86ad640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c00b29b0 con 0x7f98c0140a10 2023-12-08T08:39:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.449+0000 7f98c86ad640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c0070ba0 con 0x7f98c00ce3e0 2023-12-08T08:39:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.449+0000 7f98c86ad640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c0070db0 con 0x7f98c0071e50 2023-12-08T08:39:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.449+0000 7f98c5c48640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98c00ce3e0 0x7f98c00b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.449+0000 7f98c5c48640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98c00ce3e0 0x7f98c00b1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47656/0 (socket says 172.21.15.16:47656) 2023-12-08T08:39:29.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.449+0000 7f98c5c48640 1 -- 172.21.15.16:0/599512744 learned_addr learned my addr 172.21.15.16:0/599512744 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:29.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.449+0000 7f98c6c4a640 1 --2- 172.21.15.16:0/599512744 >> v2:172.21.15.112:3300/0 conn(0x7f98c0071e50 0x7f98c00b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.449+0000 7f98c5c48640 1 -- 172.21.15.16:0/599512744 >> v2:172.21.15.112:3300/0 conn(0x7f98c0071e50 msgr2=0x7f98c00b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:29.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.449+0000 7f98c5c48640 1 --2- 172.21.15.16:0/599512744 >> v2:172.21.15.112:3300/0 conn(0x7f98c0071e50 0x7f98c00b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.449+0000 7f98c5c48640 1 -- 172.21.15.16:0/599512744 >> v1:172.21.15.112:6789/0 conn(0x7f98c0140a10 legacy=0x7f98c00cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.449+0000 7f98c5c48640 1 -- 172.21.15.16:0/599512744 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0071090 con 0x7f98c00ce3e0 2023-12-08T08:39:29.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.449+0000 7f98c6c4a640 1 --2- 172.21.15.16:0/599512744 >> v2:172.21.15.112:3300/0 conn(0x7f98c0071e50 0x7f98c00b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:29.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.450+0000 7f98c5c48640 1 --2- 172.21.15.16:0/599512744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c00ce3e0 0x7f98c00b1f30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f98c006bf60 tx=0x7f98b002e200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9c227836c92179 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:29.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.450+0000 7f98c5447640 1 -- 172.21.15.16:0/599512744 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0036440 con 0x7f98c00ce3e0 2023-12-08T08:39:29.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.450+0000 7f98c5447640 1 -- 172.21.15.16:0/599512744 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b00365a0 con 0x7f98c00ce3e0 2023-12-08T08:39:29.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.450+0000 7f98c5447640 1 -- 172.21.15.16:0/599512744 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0036930 con 0x7f98c00ce3e0 2023-12-08T08:39:29.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.451+0000 7f98c86ad640 1 -- 172.21.15.16:0/599512744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c00ce3e0 msgr2=0x7f98c00b1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.451+0000 7f98c86ad640 1 --2- 172.21.15.16:0/599512744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c00ce3e0 0x7f98c00b1f30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f98c006bf60 tx=0x7f98b002e200 comp rx=0 tx=0).stop 2023-12-08T08:39:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.451+0000 7f98c86ad640 1 -- 172.21.15.16:0/599512744 shutdown_connections 2023-12-08T08:39:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.451+0000 7f98c86ad640 1 --2- 172.21.15.16:0/599512744 >> v2:172.21.15.112:3300/0 conn(0x7f98c0071e50 0x7f98c00b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.451+0000 7f98c86ad640 1 --2- 172.21.15.16:0/599512744 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c00ce3e0 0x7f98c00b1f30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.451+0000 7f98c86ad640 1 -- 172.21.15.16:0/599512744 >> 172.21.15.16:0/599512744 conn(0x7f98c00b09f0 msgr2=0x7f98c00b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.451+0000 7f98c86ad640 1 -- 172.21.15.16:0/599512744 shutdown_connections 2023-12-08T08:39:29.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.451+0000 7f98c86ad640 1 -- 172.21.15.16:0/599512744 wait complete. 2023-12-08T08:39:29.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.452+0000 7f98c86ad640 1 Processor -- start 2023-12-08T08:39:29.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.452+0000 7f98c86ad640 1 -- start start 2023-12-08T08:39:29.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c86ad640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c0071e50 0x7f98c00ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c86ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c00ce3e0 0x7f98c0156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c86ad640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98c0140a10 0x7f98c013eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c86ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c009c9b0 con 0x7f98c00ce3e0 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c86ad640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c00ae340 con 0x7f98c0071e50 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c86ad640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c013dbd0 con 0x7f98c0140a10 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c5c48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c00ce3e0 0x7f98c0156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c5c48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c00ce3e0 0x7f98c0156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47666/0 (socket says 172.21.15.16:47666) 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c5c48640 1 -- 172.21.15.16:0/622127250 learned_addr learned my addr 172.21.15.16:0/622127250 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c6449640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c0071e50 0x7f98c00ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c5c48640 1 -- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98c0140a10 msgr2=0x7f98c013eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c6c4a640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98c0140a10 0x7f98c013eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c5c48640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98c0140a10 0x7f98c013eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c5c48640 1 -- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c0071e50 msgr2=0x7f98c00ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c5c48640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c0071e50 0x7f98c00ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c5c48640 1 -- 172.21.15.16:0/622127250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b000b770 con 0x7f98c00ce3e0 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c6449640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c0071e50 0x7f98c00ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c5c48640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c00ce3e0 0x7f98c0156c10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f98b002e240 tx=0x7f98b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98b6ffd640 1 -- 172.21.15.16:0/622127250 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0036440 con 0x7f98c00ce3e0 2023-12-08T08:39:29.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98b6ffd640 1 -- 172.21.15.16:0/622127250 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b0034b00 con 0x7f98c00ce3e0 2023-12-08T08:39:29.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.453+0000 7f98c86ad640 1 -- 172.21.15.16:0/622127250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c009c3f0 con 0x7f98c00ce3e0 2023-12-08T08:39:29.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.454+0000 7f98b6ffd640 1 -- 172.21.15.16:0/622127250 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0041460 con 0x7f98c00ce3e0 2023-12-08T08:39:29.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.454+0000 7f98c86ad640 1 -- 172.21.15.16:0/622127250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c00abee0 con 0x7f98c00ce3e0 2023-12-08T08:39:29.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.454+0000 7f98c86ad640 1 -- 172.21.15.16:0/622127250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c00abd40 con 0x7f98c00ce3e0 2023-12-08T08:39:29.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.455+0000 7f98b6ffd640 1 -- 172.21.15.16:0/622127250 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98b004a3b0 con 0x7f98c00ce3e0 2023-12-08T08:39:29.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.455+0000 7f98b6ffd640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f98a406ae70 0x7f98a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.455+0000 7f98b6ffd640 1 -- 172.21.15.16:0/622127250 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f98b00ac150 con 0x7f98c00ce3e0 2023-12-08T08:39:29.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.456+0000 7f98c6449640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f98a406ae70 0x7f98a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.456+0000 7f98c6449640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f98a406ae70 0x7f98a406d330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f98bc000f60 tx=0x7f98bc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:29.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.460+0000 7f98b6ffd640 1 -- 172.21.15.16:0/622127250 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98c00abd40 con 0x7f98c00ce3e0 2023-12-08T08:39:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.673+0000 7f98c86ad640 1 -- 172.21.15.16:0/622127250 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98c00abd40 con 0x7f98c00ce3e0 2023-12-08T08:39:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.674+0000 7f98b6ffd640 1 -- 172.21.15.16:0/622127250 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4912 (secure 0 0 0) 0x7f98c00abd40 con 0x7f98c00ce3e0 2023-12-08T08:39:29.676 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:29.676 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6836/307039451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":307039451},{"type":"v1","addr":"172.21.15.112:6836","nonce":307039451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4275,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6833/1424152341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":1424152341},{"type":"v1","addr":"172.21.15.16:6833","nonce":1424152341}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3597811670","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3597811670},{"type":"v1","addr":"172.21.15.16:6835","nonce":3597811670}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-08T08:38:18.991090+0000","modified":"2023-12-08T08:38:41.075742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4297},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4297":{"gid":4297,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6837/1082241707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1082241707},{"type":"v1","addr":"172.21.15.112:6837","nonce":1082241707}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-08T08:39:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2023-12-08T08:39:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.676+0000 7f98c86ad640 1 -- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f98a406ae70 msgr2=0x7f98a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.676+0000 7f98c86ad640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f98a406ae70 0x7f98a406d330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f98bc000f60 tx=0x7f98bc009040 comp rx=0 tx=0).stop 2023-12-08T08:39:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.676+0000 7f98c86ad640 1 -- 172.21.15.16:0/622127250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c00ce3e0 msgr2=0x7f98c0156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.676+0000 7f98c86ad640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c00ce3e0 0x7f98c0156c10 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f98b002e240 tx=0x7f98b00046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.676+0000 7f98c86ad640 1 -- 172.21.15.16:0/622127250 shutdown_connections 2023-12-08T08:39:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.676+0000 7f98c86ad640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f98a406ae70 0x7f98a406d330 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.676+0000 7f98c86ad640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f98c0140a10 0x7f98c013eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.676+0000 7f98c86ad640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c00ce3e0 0x7f98c0156c10 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.676+0000 7f98c86ad640 1 --2- 172.21.15.16:0/622127250 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f98c0071e50 0x7f98c00ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.676+0000 7f98c86ad640 1 -- 172.21.15.16:0/622127250 >> 172.21.15.16:0/622127250 conn(0x7f98c00b09f0 msgr2=0x7f98c00cbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:29.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.676+0000 7f98c86ad640 1 -- 172.21.15.16:0/622127250 shutdown_connections 2023-12-08T08:39:29.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.676+0000 7f98c86ad640 1 -- 172.21.15.16:0/622127250 wait complete. 2023-12-08T08:39:29.688 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:29.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.793+0000 7f1f84dbe640 1 Processor -- start 2023-12-08T08:39:29.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.793+0000 7f1f84dbe640 1 -- start start 2023-12-08T08:39:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.794+0000 7f1f84dbe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f800ce270 0x7f1f8006bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.794+0000 7f1f84dbe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f8006c480 0x7f1f8015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.794+0000 7f1f84dbe640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f800ce6f0 con 0x7f1f800ccd70 2023-12-08T08:39:29.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.794+0000 7f1f84dbe640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f800d0a60 con 0x7f1f800ce270 2023-12-08T08:39:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.794+0000 7f1f84dbe640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8006c910 con 0x7f1f8006c480 2023-12-08T08:39:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.794+0000 7f1f77fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f800ce270 0x7f1f8006bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.794+0000 7f1f77fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f800ce270 0x7f1f8006bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47668/0 (socket says 172.21.15.16:47668) 2023-12-08T08:39:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.794+0000 7f1f77fff640 1 -- 172.21.15.16:0/417827066 learned_addr learned my addr 172.21.15.16:0/417827066 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.794+0000 7f1f7ed9d640 1 --2- 172.21.15.16:0/417827066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f8006c480 0x7f1f8015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.794+0000 7f1f77fff640 1 -- 172.21.15.16:0/417827066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f8006c480 msgr2=0x7f1f8015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.794+0000 7f1f77fff640 1 --2- 172.21.15.16:0/417827066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f8006c480 0x7f1f8015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.794+0000 7f1f77fff640 1 -- 172.21.15.16:0/417827066 >> v1:172.21.15.112:6789/0 conn(0x7f1f800ccd70 legacy=0x7f1f800cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.794+0000 7f1f77fff640 1 -- 172.21.15.16:0/417827066 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f800cf9c0 con 0x7f1f800ce270 2023-12-08T08:39:29.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.794+0000 7f1f7ed9d640 1 --2- 172.21.15.16:0/417827066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f8006c480 0x7f1f8015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.795+0000 7f1f77fff640 1 --2- 172.21.15.16:0/417827066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f800ce270 0x7f1f8006bf40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1f680098a0 tx=0x7f1f6802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc547e3bad4484c server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.795+0000 7f1f7dd9b640 1 -- 172.21.15.16:0/417827066 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68031640 con 0x7f1f800ce270 2023-12-08T08:39:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.795+0000 7f1f7dd9b640 1 -- 172.21.15.16:0/417827066 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f680317a0 con 0x7f1f800ce270 2023-12-08T08:39:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.795+0000 7f1f7dd9b640 1 -- 172.21.15.16:0/417827066 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68031b30 con 0x7f1f800ce270 2023-12-08T08:39:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.796+0000 7f1f84dbe640 1 -- 172.21.15.16:0/417827066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f800ce270 msgr2=0x7f1f8006bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.796+0000 7f1f84dbe640 1 --2- 172.21.15.16:0/417827066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f800ce270 0x7f1f8006bf40 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1f680098a0 tx=0x7f1f6802b8f0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.796+0000 7f1f84dbe640 1 -- 172.21.15.16:0/417827066 shutdown_connections 2023-12-08T08:39:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.796+0000 7f1f84dbe640 1 --2- 172.21.15.16:0/417827066 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f8006c480 0x7f1f8015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.796+0000 7f1f84dbe640 1 --2- 172.21.15.16:0/417827066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f800ce270 0x7f1f8006bf40 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.796+0000 7f1f84dbe640 1 -- 172.21.15.16:0/417827066 >> 172.21.15.16:0/417827066 conn(0x7f1f800d0de0 msgr2=0x7f1f800d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.796+0000 7f1f84dbe640 1 -- 172.21.15.16:0/417827066 shutdown_connections 2023-12-08T08:39:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.796+0000 7f1f84dbe640 1 -- 172.21.15.16:0/417827066 wait complete. 2023-12-08T08:39:29.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.797+0000 7f1f84dbe640 1 Processor -- start 2023-12-08T08:39:29.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.797+0000 7f1f84dbe640 1 -- start start 2023-12-08T08:39:29.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f84dbe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f8006c480 0x7f1f8009b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f84dbe640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800ccd70 0x7f1f800c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f84dbe640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f800ce270 0x7f1f801f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f84dbe640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8009bf00 con 0x7f1f8006c480 2023-12-08T08:39:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f84dbe640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f800c5100 con 0x7f1f800ccd70 2023-12-08T08:39:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f84dbe640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f800c7aa0 con 0x7f1f800ce270 2023-12-08T08:39:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f7e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f8006c480 0x7f1f8009b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f7e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f8006c480 0x7f1f8009b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47678/0 (socket says 172.21.15.16:47678) 2023-12-08T08:39:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f7e59c640 1 -- 172.21.15.16:0/152750712 learned_addr learned my addr 172.21.15.16:0/152750712 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f77fff640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800ccd70 0x7f1f800c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f7ed9d640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f800ce270 0x7f1f801f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f7e59c640 1 -- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f800ce270 msgr2=0x7f1f801f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f7e59c640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f800ce270 0x7f1f801f2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f7e59c640 1 -- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800ccd70 msgr2=0x7f1f800c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f7e59c640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800ccd70 0x7f1f800c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f7e59c640 1 -- 172.21.15.16:0/152750712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f680094d0 con 0x7f1f8006c480 2023-12-08T08:39:29.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f7ed9d640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f800ce270 0x7f1f801f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:29.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f7e59c640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f8006c480 0x7f1f8009b590 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000c490 tx=0x7f1f7000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:29.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f76ffd640 1 -- 172.21.15.16:0/152750712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70012830 con 0x7f1f8006c480 2023-12-08T08:39:29.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f76ffd640 1 -- 172.21.15.16:0/152750712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f70012990 con 0x7f1f8006c480 2023-12-08T08:39:29.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f76ffd640 1 -- 172.21.15.16:0/152750712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70012d40 con 0x7f1f8006c480 2023-12-08T08:39:29.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.798+0000 7f1f84dbe640 1 -- 172.21.15.16:0/152750712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f801f3450 con 0x7f1f8006c480 2023-12-08T08:39:29.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.799+0000 7f1f84dbe640 1 -- 172.21.15.16:0/152750712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f800c4ab0 con 0x7f1f8006c480 2023-12-08T08:39:29.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.800+0000 7f1f84dbe640 1 -- 172.21.15.16:0/152750712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f800c4910 con 0x7f1f8006c480 2023-12-08T08:39:29.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.800+0000 7f1f76ffd640 1 -- 172.21.15.16:0/152750712 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f7001b400 con 0x7f1f8006c480 2023-12-08T08:39:29.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.800+0000 7f1f76ffd640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1f6006ae70 0x7f1f6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:29.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.801+0000 7f1f76ffd640 1 -- 172.21.15.16:0/152750712 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f1f70089970 con 0x7f1f8006c480 2023-12-08T08:39:29.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.801+0000 7f1f77fff640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1f6006ae70 0x7f1f6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:29.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.801+0000 7f1f77fff640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1f6006ae70 0x7f1f6006d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1f68036000 tx=0x7f1f6803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:29.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.805+0000 7f1f76ffd640 1 -- 172.21.15.16:0/152750712 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f800c4910 con 0x7f1f8006c480 2023-12-08T08:39:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.984+0000 7f1f84dbe640 1 -- 172.21.15.16:0/152750712 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f800c4910 con 0x7f1f8006c480 2023-12-08T08:39:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.985+0000 7f1f76ffd640 1 -- 172.21.15.16:0/152750712 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18720 (secure 0 0 0) 0x7f1f800c4910 con 0x7f1f8006c480 2023-12-08T08:39:29.987 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:29.987 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:00.787962+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:38:17.121453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-08T08:38:17.965614+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-08T08:38:21.683956+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.987+0000 7f1f84dbe640 1 -- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1f6006ae70 msgr2=0x7f1f6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.987+0000 7f1f84dbe640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1f6006ae70 0x7f1f6006d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1f68036000 tx=0x7f1f6803c040 comp rx=0 tx=0).stop 2023-12-08T08:39:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.987+0000 7f1f84dbe640 1 -- 172.21.15.16:0/152750712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f8006c480 msgr2=0x7f1f8009b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.987+0000 7f1f84dbe640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f8006c480 0x7f1f8009b590 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000c490 tx=0x7f1f7000c950 comp rx=0 tx=0).stop 2023-12-08T08:39:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.987+0000 7f1f84dbe640 1 -- 172.21.15.16:0/152750712 shutdown_connections 2023-12-08T08:39:29.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.987+0000 7f1f84dbe640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1f6006ae70 0x7f1f6006d330 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.987+0000 7f1f84dbe640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f800ce270 0x7f1f801f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.987+0000 7f1f84dbe640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f800ccd70 0x7f1f800c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.987+0000 7f1f84dbe640 1 --2- 172.21.15.16:0/152750712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f8006c480 0x7f1f8009b590 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.987+0000 7f1f84dbe640 1 -- 172.21.15.16:0/152750712 >> 172.21.15.16:0/152750712 conn(0x7f1f800d0de0 msgr2=0x7f1f800d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.987+0000 7f1f84dbe640 1 -- 172.21.15.16:0/152750712 shutdown_connections 2023-12-08T08:39:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:29.987+0000 7f1f84dbe640 1 -- 172.21.15.16:0/152750712 wait complete. 2023-12-08T08:39:30.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-08T08:39:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.106+0000 7f1d06ddc640 1 Processor -- start 2023-12-08T08:39:30.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.106+0000 7f1d06ddc640 1 -- start start 2023-12-08T08:39:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.107+0000 7f1d06ddc640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1d000cf380 0x7f1d0006a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.107+0000 7f1d06ddc640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d000cf760 0x7f1d00161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.107+0000 7f1d06ddc640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0006b070 con 0x7f1d00071b80 2023-12-08T08:39:30.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.107+0000 7f1d06ddc640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d000d0d00 con 0x7f1d000cf760 2023-12-08T08:39:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.107+0000 7f1d06ddc640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d00161bd0 con 0x7f1d000cf380 2023-12-08T08:39:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.107+0000 7f1cfffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d000cf760 0x7f1d00161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.107+0000 7f1cfffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d000cf760 0x7f1d00161690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47688/0 (socket says 172.21.15.16:47688) 2023-12-08T08:39:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.107+0000 7f1cfffff640 1 -- 172.21.15.16:0/3359436865 learned_addr learned my addr 172.21.15.16:0/3359436865 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.107+0000 7f1cfffff640 1 -- 172.21.15.16:0/3359436865 >> v2:172.21.15.112:3300/0 conn(0x7f1d000cf380 msgr2=0x7f1d0006a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.107+0000 7f1d04b78640 1 --2- 172.21.15.16:0/3359436865 >> v2:172.21.15.112:3300/0 conn(0x7f1d000cf380 0x7f1d0006a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.107+0000 7f1cfffff640 1 --2- 172.21.15.16:0/3359436865 >> v2:172.21.15.112:3300/0 conn(0x7f1d000cf380 0x7f1d0006a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.107+0000 7f1cfffff640 1 -- 172.21.15.16:0/3359436865 >> v1:172.21.15.112:6789/0 conn(0x7f1d00071b80 legacy=0x7f1d000acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.107+0000 7f1cfffff640 1 -- 172.21.15.16:0/3359436865 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0006aab0 con 0x7f1d000cf760 2023-12-08T08:39:30.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.107+0000 7f1d04b78640 1 --2- 172.21.15.16:0/3359436865 >> v2:172.21.15.112:3300/0 conn(0x7f1d000cf380 0x7f1d0006a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.108+0000 7f1cfffff640 1 --2- 172.21.15.16:0/3359436865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d000cf760 0x7f1d00161690 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1d000cfd00 tx=0x7f1cec02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba932e23bf2a8efa server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.108+0000 7f1cff7fe640 1 -- 172.21.15.16:0/3359436865 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0317e0 con 0x7f1d000cf760 2023-12-08T08:39:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.108+0000 7f1cff7fe640 1 -- 172.21.15.16:0/3359436865 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec031940 con 0x7f1d000cf760 2023-12-08T08:39:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.108+0000 7f1cff7fe640 1 -- 172.21.15.16:0/3359436865 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec031cd0 con 0x7f1d000cf760 2023-12-08T08:39:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.109+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3359436865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d000cf760 msgr2=0x7f1d00161690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.109+0000 7f1d06ddc640 1 --2- 172.21.15.16:0/3359436865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d000cf760 0x7f1d00161690 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1d000cfd00 tx=0x7f1cec02b880 comp rx=0 tx=0).stop 2023-12-08T08:39:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.109+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3359436865 shutdown_connections 2023-12-08T08:39:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.109+0000 7f1d06ddc640 1 --2- 172.21.15.16:0/3359436865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d000cf760 0x7f1d00161690 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.109+0000 7f1d06ddc640 1 --2- 172.21.15.16:0/3359436865 >> v2:172.21.15.112:3300/0 conn(0x7f1d000cf380 0x7f1d0006a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.109+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3359436865 >> 172.21.15.16:0/3359436865 conn(0x7f1d000cfda0 msgr2=0x7f1d000d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.109+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3359436865 shutdown_connections 2023-12-08T08:39:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.109+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3359436865 wait complete. 2023-12-08T08:39:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.110+0000 7f1d06ddc640 1 Processor -- start 2023-12-08T08:39:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.110+0000 7f1d06ddc640 1 -- start start 2023-12-08T08:39:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1d06ddc640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d00071b80 0x7f1d00146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1d06ddc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d000cf380 0x7f1d00149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1d06ddc640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d000cf760 0x7f1d000b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:30.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1d06ddc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d00147380 con 0x7f1d000cf380 2023-12-08T08:39:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1d06ddc640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d001f6a10 con 0x7f1d00071b80 2023-12-08T08:39:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1d06ddc640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0015ae90 con 0x7f1d000cf760 2023-12-08T08:39:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1cfffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d000cf380 0x7f1d00149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1cfffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d000cf380 0x7f1d00149410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47696/0 (socket says 172.21.15.16:47696) 2023-12-08T08:39:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1cfffff640 1 -- 172.21.15.16:0/3661316772 learned_addr learned my addr 172.21.15.16:0/3661316772 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1cfffff640 1 -- 172.21.15.16:0/3661316772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d000cf760 msgr2=0x7f1d000b2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1cfffff640 1 --2- 172.21.15.16:0/3661316772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d000cf760 0x7f1d000b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1d04b78640 1 --2- 172.21.15.16:0/3661316772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d00071b80 0x7f1d00146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1cfffff640 1 -- 172.21.15.16:0/3661316772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d00071b80 msgr2=0x7f1d00146b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1cfffff640 1 --2- 172.21.15.16:0/3661316772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d00071b80 0x7f1d00146b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1cfffff640 1 -- 172.21.15.16:0/3661316772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cec0094d0 con 0x7f1d000cf380 2023-12-08T08:39:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.111+0000 7f1cfffff640 1 --2- 172.21.15.16:0/3661316772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d000cf380 0x7f1d00149410 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1cec000c00 tx=0x7f1cec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.112+0000 7f1cfd7fa640 1 -- 172.21.15.16:0/3661316772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0329c0 con 0x7f1d000cf380 2023-12-08T08:39:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.112+0000 7f1cfd7fa640 1 -- 172.21.15.16:0/3661316772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec032b40 con 0x7f1d000cf380 2023-12-08T08:39:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.112+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3661316772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d001470a0 con 0x7f1d000cf380 2023-12-08T08:39:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.112+0000 7f1cfd7fa640 1 -- 172.21.15.16:0/3661316772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0453f0 con 0x7f1d000cf380 2023-12-08T08:39:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.112+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3661316772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0014a140 con 0x7f1d000cf380 2023-12-08T08:39:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.113+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3661316772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0014a330 con 0x7f1d000cf380 2023-12-08T08:39:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.113+0000 7f1cfd7fa640 1 -- 172.21.15.16:0/3661316772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1cec04d3f0 con 0x7f1d000cf380 2023-12-08T08:39:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.113+0000 7f1cfd7fa640 1 --2- 172.21.15.16:0/3661316772 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1ce406ae70 0x7f1ce406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.113+0000 7f1cfd7fa640 1 -- 172.21.15.16:0/3661316772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f1cec0ab860 con 0x7f1d000cf380 2023-12-08T08:39:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.114+0000 7f1d04b78640 1 --2- 172.21.15.16:0/3661316772 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1ce406ae70 0x7f1ce406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.114+0000 7f1d04b78640 1 --2- 172.21.15.16:0/3661316772 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1ce406ae70 0x7f1ce406d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4006fd0 tx=0x7f1cf4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:30.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.118+0000 7f1cfd7fa640 1 -- 172.21.15.16:0/3661316772 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1cec076670 con 0x7f1d000cf380 2023-12-08T08:39:30.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.330+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3661316772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1d00067430 con 0x7f1d000cf380 2023-12-08T08:39:30.332 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-12-08T08:39:30.331+0000 7f54207dc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-08T08:39:30.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.332+0000 7f1cfd7fa640 1 -- 172.21.15.16:0/3661316772 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1cec079c90 con 0x7f1d000cf380 2023-12-08T08:39:30.334 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-08T08:39:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.334+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3661316772 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1ce406ae70 msgr2=0x7f1ce406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.334+0000 7f1d06ddc640 1 --2- 172.21.15.16:0/3661316772 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1ce406ae70 0x7f1ce406d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4006fd0 tx=0x7f1cf4008040 comp rx=0 tx=0).stop 2023-12-08T08:39:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.334+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3661316772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d000cf380 msgr2=0x7f1d00149410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.334+0000 7f1d06ddc640 1 --2- 172.21.15.16:0/3661316772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d000cf380 0x7f1d00149410 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1cec000c00 tx=0x7f1cec0046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.335+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3661316772 shutdown_connections 2023-12-08T08:39:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.335+0000 7f1d06ddc640 1 --2- 172.21.15.16:0/3661316772 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1ce406ae70 0x7f1ce406d330 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.335+0000 7f1d06ddc640 1 --2- 172.21.15.16:0/3661316772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d000cf760 0x7f1d000b2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.335+0000 7f1d06ddc640 1 --2- 172.21.15.16:0/3661316772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d000cf380 0x7f1d00149410 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.335+0000 7f1d06ddc640 1 --2- 172.21.15.16:0/3661316772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d00071b80 0x7f1d00146b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.335+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3661316772 >> 172.21.15.16:0/3661316772 conn(0x7f1d000cfda0 msgr2=0x7f1d000d0180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.335+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3661316772 shutdown_connections 2023-12-08T08:39:30.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.335+0000 7f1d06ddc640 1 -- 172.21.15.16:0/3661316772 wait complete. 2023-12-08T08:39:30.340 INFO:tasks.ceph.mds.d.smithi112.stderr: -4316> 2023-12-08T08:38:23.635+0000 7f4eb8a35640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-08T08:39:30.348 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-08T08:39:30.371 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2023-12-08T08:39:30.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.454+0000 7fef2607c640 1 Processor -- start 2023-12-08T08:39:30.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.454+0000 7fef2607c640 1 -- start start 2023-12-08T08:39:30.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.455+0000 7fef2607c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef2015d590 0x7fef200ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:30.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.455+0000 7fef2607c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef200ae350 0x7fef200ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:30.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.455+0000 7fef2607c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef2006d650 con 0x7fef200afeb0 2023-12-08T08:39:30.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.455+0000 7fef2607c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef200adbe0 con 0x7fef200ae350 2023-12-08T08:39:30.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.455+0000 7fef2607c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2015d970 con 0x7fef2015d590 2023-12-08T08:39:30.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.455+0000 7fef1ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef200ae350 0x7fef200ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:30.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.455+0000 7fef1ffff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef200ae350 0x7fef200ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47700/0 (socket says 172.21.15.16:47700) 2023-12-08T08:39:30.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.455+0000 7fef1ffff640 1 -- 172.21.15.16:0/1404782951 learned_addr learned my addr 172.21.15.16:0/1404782951 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:30.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.455+0000 7fef1f7fe640 1 --2- 172.21.15.16:0/1404782951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef2015d590 0x7fef200ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:30.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.455+0000 7fef1ffff640 1 -- 172.21.15.16:0/1404782951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef2015d590 msgr2=0x7fef200ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:30.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.455+0000 7fef1ffff640 1 --2- 172.21.15.16:0/1404782951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef2015d590 0x7fef200ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.455+0000 7fef1ffff640 1 -- 172.21.15.16:0/1404782951 >> v1:172.21.15.112:6789/0 conn(0x7fef200afeb0 legacy=0x7fef200cf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:30.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.455+0000 7fef1ffff640 1 -- 172.21.15.16:0/1404782951 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2006c810 con 0x7fef200ae350 2023-12-08T08:39:30.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.456+0000 7fef1ffff640 1 --2- 172.21.15.16:0/1404782951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef200ae350 0x7fef200ae7e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fef18009e50 tx=0x7fef1802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc641e6450a6c7c server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:30.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.456+0000 7fef1e7fc640 1 -- 172.21.15.16:0/1404782951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1802c9e0 con 0x7fef200ae350 2023-12-08T08:39:30.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.456+0000 7fef1e7fc640 1 -- 172.21.15.16:0/1404782951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1802cb40 con 0x7fef200ae350 2023-12-08T08:39:30.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.456+0000 7fef1e7fc640 1 -- 172.21.15.16:0/1404782951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef180343f0 con 0x7fef200ae350 2023-12-08T08:39:30.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.456+0000 7fef2607c640 1 -- 172.21.15.16:0/1404782951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef200ae350 msgr2=0x7fef200ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:30.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.456+0000 7fef2607c640 1 --2- 172.21.15.16:0/1404782951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef200ae350 0x7fef200ae7e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fef18009e50 tx=0x7fef1802be40 comp rx=0 tx=0).stop 2023-12-08T08:39:30.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.456+0000 7fef2607c640 1 -- 172.21.15.16:0/1404782951 shutdown_connections 2023-12-08T08:39:30.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.456+0000 7fef2607c640 1 --2- 172.21.15.16:0/1404782951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef200ae350 0x7fef200ae7e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.456+0000 7fef2607c640 1 --2- 172.21.15.16:0/1404782951 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef2015d590 0x7fef200ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.456+0000 7fef2607c640 1 -- 172.21.15.16:0/1404782951 >> 172.21.15.16:0/1404782951 conn(0x7fef200d0e10 msgr2=0x7fef200d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:30.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.457+0000 7fef2607c640 1 -- 172.21.15.16:0/1404782951 shutdown_connections 2023-12-08T08:39:30.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.457+0000 7fef2607c640 1 -- 172.21.15.16:0/1404782951 wait complete. 2023-12-08T08:39:30.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.458+0000 7fef2607c640 1 Processor -- start 2023-12-08T08:39:30.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.458+0000 7fef2607c640 1 -- start start 2023-12-08T08:39:30.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef2607c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef200ae350 0x7fef201f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:30.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef2607c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef200afeb0 0x7fef20159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:30.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef2607c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015d590 0x7fef200b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:30.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef2607c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef200ce2f0 con 0x7fef200afeb0 2023-12-08T08:39:30.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef2607c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef201f40f0 con 0x7fef2015d590 2023-12-08T08:39:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef2607c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2015a1c0 con 0x7fef200ae350 2023-12-08T08:39:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef1effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef200afeb0 0x7fef20159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef1effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef200afeb0 0x7fef20159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47716/0 (socket says 172.21.15.16:47716) 2023-12-08T08:39:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef1effd640 1 -- 172.21.15.16:0/1703456907 learned_addr learned my addr 172.21.15.16:0/1703456907 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef1f7fe640 1 --2- 172.21.15.16:0/1703456907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef200ae350 0x7fef201f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef1ffff640 1 --2- 172.21.15.16:0/1703456907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015d590 0x7fef200b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef1effd640 1 -- 172.21.15.16:0/1703456907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef200ae350 msgr2=0x7fef201f3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef1effd640 1 --2- 172.21.15.16:0/1703456907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef200ae350 0x7fef201f3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef1effd640 1 -- 172.21.15.16:0/1703456907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015d590 msgr2=0x7fef200b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef1effd640 1 --2- 172.21.15.16:0/1703456907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015d590 0x7fef200b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:30.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef1effd640 1 -- 172.21.15.16:0/1703456907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef18009b30 con 0x7fef200afeb0 2023-12-08T08:39:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef1effd640 1 --2- 172.21.15.16:0/1703456907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef200afeb0 0x7fef20159c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fef1000c430 tx=0x7fef1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef07fff640 1 -- 172.21.15.16:0/1703456907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10011790 con 0x7fef200afeb0 2023-12-08T08:39:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef07fff640 1 -- 172.21.15.16:0/1703456907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef100118f0 con 0x7fef200afeb0 2023-12-08T08:39:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef2607c640 1 -- 172.21.15.16:0/1703456907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef200b2980 con 0x7fef200afeb0 2023-12-08T08:39:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef07fff640 1 -- 172.21.15.16:0/1703456907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef10011ca0 con 0x7fef200afeb0 2023-12-08T08:39:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.459+0000 7fef2607c640 1 -- 172.21.15.16:0/1703456907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef200b2f40 con 0x7fef200afeb0 2023-12-08T08:39:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.460+0000 7fef2607c640 1 -- 172.21.15.16:0/1703456907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef20067430 con 0x7fef200afeb0 2023-12-08T08:39:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.461+0000 7fef07fff640 1 -- 172.21.15.16:0/1703456907 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef1001c460 con 0x7fef200afeb0 2023-12-08T08:39:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.461+0000 7fef07fff640 1 --2- 172.21.15.16:0/1703456907 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fef0c06ae40 0x7fef0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.461+0000 7fef07fff640 1 -- 172.21.15.16:0/1703456907 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6778+0+0 (secure 0 0 0) 0x7fef10089800 con 0x7fef200afeb0 2023-12-08T08:39:30.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.462+0000 7fef1f7fe640 1 --2- 172.21.15.16:0/1703456907 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fef0c06ae40 0x7fef0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:30.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.462+0000 7fef1f7fe640 1 --2- 172.21.15.16:0/1703456907 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fef0c06ae40 0x7fef0c06d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fef1400aa20 tx=0x7fef14008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:30.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.466+0000 7fef07fff640 1 -- 172.21.15.16:0/1703456907 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef100545c0 con 0x7fef200afeb0 2023-12-08T08:39:30.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:30.684+0000 7fef2607c640 1 -- 172.21.15.16:0/1703456907 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fef200b6d90 con 0x7fef200afeb0 2023-12-08T08:39:31.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.334+0000 7fef07fff640 1 -- 172.21.15.16:0/1703456907 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fef10057be0 con 0x7fef200afeb0 2023-12-08T08:39:31.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.337+0000 7fef2607c640 1 -- 172.21.15.16:0/1703456907 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fef0c06ae40 msgr2=0x7fef0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:31.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.337+0000 7fef2607c640 1 --2- 172.21.15.16:0/1703456907 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fef0c06ae40 0x7fef0c06d300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fef1400aa20 tx=0x7fef14008040 comp rx=0 tx=0).stop 2023-12-08T08:39:31.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.337+0000 7fef2607c640 1 -- 172.21.15.16:0/1703456907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef200afeb0 msgr2=0x7fef20159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:31.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.337+0000 7fef2607c640 1 --2- 172.21.15.16:0/1703456907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef200afeb0 0x7fef20159c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fef1000c430 tx=0x7fef1000c8f0 comp rx=0 tx=0).stop 2023-12-08T08:39:31.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.337+0000 7fef2607c640 1 -- 172.21.15.16:0/1703456907 shutdown_connections 2023-12-08T08:39:31.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.337+0000 7fef2607c640 1 --2- 172.21.15.16:0/1703456907 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fef0c06ae40 0x7fef0c06d300 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:31.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.337+0000 7fef2607c640 1 --2- 172.21.15.16:0/1703456907 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fef2015d590 0x7fef200b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:31.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.337+0000 7fef2607c640 1 --2- 172.21.15.16:0/1703456907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef200afeb0 0x7fef20159c80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:31.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.337+0000 7fef2607c640 1 --2- 172.21.15.16:0/1703456907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fef200ae350 0x7fef201f3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:31.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.337+0000 7fef2607c640 1 -- 172.21.15.16:0/1703456907 >> 172.21.15.16:0/1703456907 conn(0x7fef200d0e10 msgr2=0x7fef200cfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:31.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.337+0000 7fef2607c640 1 -- 172.21.15.16:0/1703456907 shutdown_connections 2023-12-08T08:39:31.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.337+0000 7fef2607c640 1 -- 172.21.15.16:0/1703456907 wait complete. 2023-12-08T08:39:31.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-08T08:39:31.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.468+0000 7f8bc6dd7640 1 Processor -- start 2023-12-08T08:39:31.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.468+0000 7f8bc6dd7640 1 -- start start 2023-12-08T08:39:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.469+0000 7f8bc6dd7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8bc015aec0 0x7f8bc00cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.469+0000 7f8bc6dd7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f8bc0160d50 0x7f8bc01611c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.469+0000 7f8bc6dd7640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc00ce080 con 0x7f8bc01618f0 2023-12-08T08:39:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.469+0000 7f8bc6dd7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc0071ae0 con 0x7f8bc015aec0 2023-12-08T08:39:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.469+0000 7f8bc6dd7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc0161700 con 0x7f8bc0160d50 2023-12-08T08:39:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.469+0000 7f8bc4b73640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8bc015aec0 0x7f8bc00cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.469+0000 7f8bbffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8bc01618f0 0x7f8bc00713d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33312/0 (socket says 172.21.15.16:33312) 2023-12-08T08:39:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.469+0000 7f8bbffff640 1 -- 172.21.15.16:0/79379044 learned_addr learned my addr 172.21.15.16:0/79379044 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.469+0000 7f8bc5374640 1 --2- 172.21.15.16:0/79379044 >> v2:172.21.15.112:3300/0 conn(0x7f8bc0160d50 0x7f8bc01611c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.469+0000 7f8bc4b73640 1 -- 172.21.15.16:0/79379044 >> v2:172.21.15.112:3300/0 conn(0x7f8bc0160d50 msgr2=0x7f8bc01611c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.469+0000 7f8bc4b73640 1 --2- 172.21.15.16:0/79379044 >> v2:172.21.15.112:3300/0 conn(0x7f8bc0160d50 0x7f8bc01611c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.469+0000 7f8bc4b73640 1 -- 172.21.15.16:0/79379044 >> v1:172.21.15.16:6789/0 conn(0x7f8bc01618f0 legacy=0x7f8bc00713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.469+0000 7f8bc4b73640 1 -- 172.21.15.16:0/79379044 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc00ce360 con 0x7f8bc015aec0 2023-12-08T08:39:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.470+0000 7f8bc4b73640 1 --2- 172.21.15.16:0/79379044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc015aec0 0x7f8bc00cee40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0070980 tx=0x7f8bb802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a839d5423c754d server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.470+0000 7f8bbf7fe640 1 -- 172.21.15.16:0/79379044 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb802cbd0 con 0x7f8bc015aec0 2023-12-08T08:39:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.470+0000 7f8bbf7fe640 1 -- 172.21.15.16:0/79379044 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb802cd30 con 0x7f8bc015aec0 2023-12-08T08:39:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.470+0000 7f8bbf7fe640 1 -- 172.21.15.16:0/79379044 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb802cea0 con 0x7f8bc015aec0 2023-12-08T08:39:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.471+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/79379044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc015aec0 msgr2=0x7f8bc00cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.471+0000 7f8bc6dd7640 1 --2- 172.21.15.16:0/79379044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc015aec0 0x7f8bc00cee40 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0070980 tx=0x7f8bb802be20 comp rx=0 tx=0).stop 2023-12-08T08:39:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.471+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/79379044 shutdown_connections 2023-12-08T08:39:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.471+0000 7f8bc6dd7640 1 --2- 172.21.15.16:0/79379044 >> v2:172.21.15.112:3300/0 conn(0x7f8bc0160d50 0x7f8bc01611c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.471+0000 7f8bc6dd7640 1 --2- 172.21.15.16:0/79379044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc015aec0 0x7f8bc00cee40 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.471+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/79379044 >> 172.21.15.16:0/79379044 conn(0x7f8bc00cf410 msgr2=0x7f8bc00cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.471+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/79379044 shutdown_connections 2023-12-08T08:39:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.471+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/79379044 wait complete. 2023-12-08T08:39:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.472+0000 7f8bc6dd7640 1 Processor -- start 2023-12-08T08:39:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.472+0000 7f8bc6dd7640 1 -- start start 2023-12-08T08:39:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc6dd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc015aec0 0x7f8bc0153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc6dd7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8bc0160d50 0x7f8bc01f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc6dd7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bc01618f0 0x7f8bc01f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:31.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc6dd7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc0153ca0 con 0x7f8bc015aec0 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc6dd7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc00b38c0 con 0x7f8bc01618f0 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc6dd7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc01f3dd0 con 0x7f8bc0160d50 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc4b73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc015aec0 0x7f8bc0153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc4b73640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc015aec0 0x7f8bc0153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47726/0 (socket says 172.21.15.16:47726) 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc4b73640 1 -- 172.21.15.16:0/961067204 learned_addr learned my addr 172.21.15.16:0/961067204 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc5374640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bc01618f0 0x7f8bc01f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc4b73640 1 -- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8bc0160d50 msgr2=0x7f8bc01f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bbffff640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8bc0160d50 0x7f8bc01f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc4b73640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8bc0160d50 0x7f8bc01f3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc4b73640 1 -- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bc01618f0 msgr2=0x7f8bc01f3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc4b73640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bc01618f0 0x7f8bc01f3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc4b73640 1 -- 172.21.15.16:0/961067204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb8009a70 con 0x7f8bc015aec0 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc5374640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bc01618f0 0x7f8bc01f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.473+0000 7f8bc4b73640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc015aec0 0x7f8bc0153760 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009d90 tx=0x7f8bb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.474+0000 7f8bbd7fa640 1 -- 172.21.15.16:0/961067204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb802cea0 con 0x7f8bc015aec0 2023-12-08T08:39:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.474+0000 7f8bbd7fa640 1 -- 172.21.15.16:0/961067204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb8040650 con 0x7f8bc015aec0 2023-12-08T08:39:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.474+0000 7f8bbd7fa640 1 -- 172.21.15.16:0/961067204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8040a00 con 0x7f8bc015aec0 2023-12-08T08:39:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.474+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/961067204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc01f44f0 con 0x7f8bc015aec0 2023-12-08T08:39:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.474+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/961067204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc015bab0 con 0x7f8bc015aec0 2023-12-08T08:39:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.474+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/961067204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc00cd000 con 0x7f8bc015aec0 2023-12-08T08:39:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.475+0000 7f8bbd7fa640 1 -- 172.21.15.16:0/961067204 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8bb8040b60 con 0x7f8bc015aec0 2023-12-08T08:39:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.475+0000 7f8bbd7fa640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8bac06ae70 0x7f8bac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.476+0000 7f8bbd7fa640 1 -- 172.21.15.16:0/961067204 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6778+0+0 (secure 0 0 0) 0x7f8bb80ac680 con 0x7f8bc015aec0 2023-12-08T08:39:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.476+0000 7f8bbffff640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8bac06ae70 0x7f8bac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.476+0000 7f8bbffff640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8bac06ae70 0x7f8bac06d330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0150830 tx=0x7f8bb4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.480+0000 7f8bbd7fa640 1 -- 172.21.15.16:0/961067204 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8bb8077390 con 0x7f8bc015aec0 2023-12-08T08:39:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:31.659+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/961067204 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8bc0067540 con 0x7f8bc015aec0 2023-12-08T08:39:32.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.335+0000 7f8bbd7fa640 1 -- 172.21.15.16:0/961067204 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8bb807a9b0 con 0x7f8bc015aec0 2023-12-08T08:39:32.338 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2023-12-08T08:39:32.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.339+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8bac06ae70 msgr2=0x7f8bac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:32.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.339+0000 7f8bc6dd7640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8bac06ae70 0x7f8bac06d330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0150830 tx=0x7f8bb4009040 comp rx=0 tx=0).stop 2023-12-08T08:39:32.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.339+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/961067204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc015aec0 msgr2=0x7f8bc0153760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:32.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.339+0000 7f8bc6dd7640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc015aec0 0x7f8bc0153760 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009d90 tx=0x7f8bb8004a40 comp rx=0 tx=0).stop 2023-12-08T08:39:32.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.340+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/961067204 shutdown_connections 2023-12-08T08:39:32.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.340+0000 7f8bc6dd7640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f8bac06ae70 0x7f8bac06d330 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:32.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.340+0000 7f8bc6dd7640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f8bc01618f0 0x7f8bc01f3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:32.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.340+0000 7f8bc6dd7640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f8bc0160d50 0x7f8bc01f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:32.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.340+0000 7f8bc6dd7640 1 --2- 172.21.15.16:0/961067204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc015aec0 0x7f8bc0153760 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:32.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.340+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/961067204 >> 172.21.15.16:0/961067204 conn(0x7f8bc00cf410 msgr2=0x7f8bc0066400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:32.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.340+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/961067204 shutdown_connections 2023-12-08T08:39:32.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.340+0000 7f8bc6dd7640 1 -- 172.21.15.16:0/961067204 wait complete. 2023-12-08T08:39:32.356 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-08T08:39:32.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.471+0000 7fb4e1301640 1 Processor -- start 2023-12-08T08:39:32.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.471+0000 7fb4e1301640 1 -- start start 2023-12-08T08:39:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.472+0000 7fb4e1301640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4dc160920 0x7fb4dc0b42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.472+0000 7fb4e1301640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4dc0d36a0 0x7fb4dc0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.472+0000 7fb4e1301640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4dc0d02f0 con 0x7fb4dc0cfec0 2023-12-08T08:39:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.472+0000 7fb4e1301640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4dc0ce640 con 0x7fb4dc160920 2023-12-08T08:39:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.472+0000 7fb4e1301640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0ce850 con 0x7fb4dc0d36a0 2023-12-08T08:39:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.472+0000 7fb4daffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4dc160920 0x7fb4dc0b42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.472+0000 7fb4daffd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4dc160920 0x7fb4dc0b42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47736/0 (socket says 172.21.15.16:47736) 2023-12-08T08:39:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.472+0000 7fb4daffd640 1 -- 172.21.15.16:0/3370898300 learned_addr learned my addr 172.21.15.16:0/3370898300 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.472+0000 7fb4d3fff640 1 --2- 172.21.15.16:0/3370898300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4dc0d36a0 0x7fb4dc0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:32.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.472+0000 7fb4daffd640 1 -- 172.21.15.16:0/3370898300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4dc0d36a0 msgr2=0x7fb4dc0cf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:32.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.472+0000 7fb4daffd640 1 --2- 172.21.15.16:0/3370898300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4dc0d36a0 0x7fb4dc0cf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:32.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.472+0000 7fb4daffd640 1 -- 172.21.15.16:0/3370898300 >> v1:172.21.15.112:6789/0 conn(0x7fb4dc0cfec0 legacy=0x7fb4dc0d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:32.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.472+0000 7fb4daffd640 1 -- 172.21.15.16:0/3370898300 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc0ceb30 con 0x7fb4dc160920 2023-12-08T08:39:32.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.472+0000 7fb4d3fff640 1 --2- 172.21.15.16:0/3370898300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4dc0d36a0 0x7fb4dc0cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:32.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.473+0000 7fb4daffd640 1 --2- 172.21.15.16:0/3370898300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4dc160920 0x7fb4dc0b42f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009870 tx=0x7fb4cc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9370f5b96c8fdb6 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:32.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.473+0000 7fb4da7fc640 1 -- 172.21.15.16:0/3370898300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc0317c0 con 0x7fb4dc160920 2023-12-08T08:39:32.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.473+0000 7fb4da7fc640 1 -- 172.21.15.16:0/3370898300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc031920 con 0x7fb4dc160920 2023-12-08T08:39:32.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.473+0000 7fb4da7fc640 1 -- 172.21.15.16:0/3370898300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc031cb0 con 0x7fb4dc160920 2023-12-08T08:39:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.474+0000 7fb4e1301640 1 -- 172.21.15.16:0/3370898300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4dc160920 msgr2=0x7fb4dc0b42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.474+0000 7fb4e1301640 1 --2- 172.21.15.16:0/3370898300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4dc160920 0x7fb4dc0b42f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009870 tx=0x7fb4cc02b860 comp rx=0 tx=0).stop 2023-12-08T08:39:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.474+0000 7fb4e1301640 1 -- 172.21.15.16:0/3370898300 shutdown_connections 2023-12-08T08:39:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.474+0000 7fb4e1301640 1 --2- 172.21.15.16:0/3370898300 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4dc0d36a0 0x7fb4dc0cf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.474+0000 7fb4e1301640 1 --2- 172.21.15.16:0/3370898300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4dc160920 0x7fb4dc0b42f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.474+0000 7fb4e1301640 1 -- 172.21.15.16:0/3370898300 >> 172.21.15.16:0/3370898300 conn(0x7fb4dc09c340 msgr2=0x7fb4dc0d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.474+0000 7fb4e1301640 1 -- 172.21.15.16:0/3370898300 shutdown_connections 2023-12-08T08:39:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.474+0000 7fb4e1301640 1 -- 172.21.15.16:0/3370898300 wait complete. 2023-12-08T08:39:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.475+0000 7fb4e1301640 1 Processor -- start 2023-12-08T08:39:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.475+0000 7fb4e1301640 1 -- start start 2023-12-08T08:39:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4e1301640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4dc0cfec0 0x7fb4dc156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:32.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4e1301640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4dc0d36a0 0x7fb4dc0ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4e1301640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4dc160920 0x7fb4dc153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4e1301640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0cbc50 con 0x7fb4dc160920 2023-12-08T08:39:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4e1301640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc1549c0 con 0x7fb4dc0d36a0 2023-12-08T08:39:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4e1301640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0cb000 con 0x7fb4dc0cfec0 2023-12-08T08:39:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4dc160920 0x7fb4dc153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4dc160920 0x7fb4dc153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47738/0 (socket says 172.21.15.16:47738) 2023-12-08T08:39:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4db7fe640 1 -- 172.21.15.16:0/2929386372 learned_addr learned my addr 172.21.15.16:0/2929386372 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4d3fff640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4dc0d36a0 0x7fb4dc0ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4daffd640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4dc0cfec0 0x7fb4dc156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4db7fe640 1 -- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4dc0cfec0 msgr2=0x7fb4dc156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:32.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4db7fe640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4dc0cfec0 0x7fb4dc156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:32.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4db7fe640 1 -- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4dc0d36a0 msgr2=0x7fb4dc0ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:32.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4db7fe640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4dc0d36a0 0x7fb4dc0ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:32.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4db7fe640 1 -- 172.21.15.16:0/2929386372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4cc0094d0 con 0x7fb4dc160920 2023-12-08T08:39:32.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4daffd640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4dc0cfec0 0x7fb4dc156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:32.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4d3fff640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4dc0d36a0 0x7fb4dc0ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:32.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4db7fe640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4dc160920 0x7fb4dc153410 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c7b0 tx=0x7fb4d400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:32.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4d37fe640 1 -- 172.21.15.16:0/2929386372 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4004b60 con 0x7fb4dc160920 2023-12-08T08:39:32.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4d37fe640 1 -- 172.21.15.16:0/2929386372 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d4004cc0 con 0x7fb4dc160920 2023-12-08T08:39:32.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4e1301640 1 -- 172.21.15.16:0/2929386372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4dc0ca760 con 0x7fb4dc160920 2023-12-08T08:39:32.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.476+0000 7fb4d37fe640 1 -- 172.21.15.16:0/2929386372 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400aa90 con 0x7fb4dc160920 2023-12-08T08:39:32.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.477+0000 7fb4e1301640 1 -- 172.21.15.16:0/2929386372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4dc0cad20 con 0x7fb4dc160920 2023-12-08T08:39:32.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.477+0000 7fb4e1301640 1 -- 172.21.15.16:0/2929386372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4dc0cbe00 con 0x7fb4dc160920 2023-12-08T08:39:32.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.478+0000 7fb4d37fe640 1 -- 172.21.15.16:0/2929386372 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb4d400abf0 con 0x7fb4dc160920 2023-12-08T08:39:32.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.478+0000 7fb4d37fe640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb4bc06ae40 0x7fb4bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:32.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.478+0000 7fb4d37fe640 1 -- 172.21.15.16:0/2929386372 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6340+0+0 (secure 0 0 0) 0x7fb4d40899d0 con 0x7fb4dc160920 2023-12-08T08:39:32.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.479+0000 7fb4daffd640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb4bc06ae40 0x7fb4bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:32.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.479+0000 7fb4daffd640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb4bc06ae40 0x7fb4bc06d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc036000 tx=0x7fb4cc03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:32.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.483+0000 7fb4d37fe640 1 -- 172.21.15.16:0/2929386372 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4dc0cbe00 con 0x7fb4dc160920 2023-12-08T08:39:32.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:32.658+0000 7fb4e1301640 1 -- 172.21.15.16:0/2929386372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb4dc0cbe00 con 0x7fb4dc160920 2023-12-08T08:39:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.337+0000 7fb4d37fe640 1 -- 172.21.15.16:0/2929386372 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb4dc0cbe00 con 0x7fb4dc160920 2023-12-08T08:39:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2023-12-08T08:39:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.342+0000 7fb4e1301640 1 -- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb4bc06ae40 msgr2=0x7fb4bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.342+0000 7fb4e1301640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb4bc06ae40 0x7fb4bc06d300 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc036000 tx=0x7fb4cc03d040 comp rx=0 tx=0).stop 2023-12-08T08:39:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.342+0000 7fb4e1301640 1 -- 172.21.15.16:0/2929386372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4dc160920 msgr2=0x7fb4dc153410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.342+0000 7fb4e1301640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4dc160920 0x7fb4dc153410 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400c7b0 tx=0x7fb4d400cc70 comp rx=0 tx=0).stop 2023-12-08T08:39:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.343+0000 7fb4e1301640 1 -- 172.21.15.16:0/2929386372 shutdown_connections 2023-12-08T08:39:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.343+0000 7fb4e1301640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb4bc06ae40 0x7fb4bc06d300 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.343+0000 7fb4e1301640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4dc160920 0x7fb4dc153410 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:33.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.343+0000 7fb4e1301640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb4dc0d36a0 0x7fb4dc0ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:33.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.343+0000 7fb4e1301640 1 --2- 172.21.15.16:0/2929386372 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb4dc0cfec0 0x7fb4dc156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:33.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.343+0000 7fb4e1301640 1 -- 172.21.15.16:0/2929386372 >> 172.21.15.16:0/2929386372 conn(0x7fb4dc09c340 msgr2=0x7fb4dc15e910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:33.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.343+0000 7fb4e1301640 1 -- 172.21.15.16:0/2929386372 shutdown_connections 2023-12-08T08:39:33.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.344+0000 7fb4e1301640 1 -- 172.21.15.16:0/2929386372 wait complete. 2023-12-08T08:39:33.361 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-12-08T08:39:33.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.475+0000 7ff56ce90640 1 Processor -- start 2023-12-08T08:39:33.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.475+0000 7ff56ce90640 1 -- start start 2023-12-08T08:39:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.476+0000 7ff56ce90640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff568143720 0x7ff568071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.476+0000 7ff56ce90640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5680b2870 con 0x7ff5680b2d00 2023-12-08T08:39:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.476+0000 7ff56ce90640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff568071b40 con 0x7ff568071220 2023-12-08T08:39:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.476+0000 7ff56ce90640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5680b1c60 con 0x7ff568143720 2023-12-08T08:39:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.476+0000 7ff565d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff568143720 0x7ff568071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:33.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.476+0000 7ff566d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff5680b2d00 0x7ff5680b3130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33318/0 (socket says 172.21.15.16:33318) 2023-12-08T08:39:33.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.476+0000 7ff565d9b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff568143720 0x7ff568071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47742/0 (socket says 172.21.15.16:47742) 2023-12-08T08:39:33.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.476+0000 7ff566d9d640 1 -- 172.21.15.16:0/884197277 learned_addr learned my addr 172.21.15.16:0/884197277 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:33.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.476+0000 7ff565d9b640 1 -- 172.21.15.16:0/884197277 >> v1:172.21.15.112:6789/0 conn(0x7ff568071220 legacy=0x7ff56815e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:33.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.476+0000 7ff565d9b640 1 -- 172.21.15.16:0/884197277 >> v1:172.21.15.16:6789/0 conn(0x7ff5680b2d00 legacy=0x7ff5680b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:33.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.476+0000 7ff565d9b640 1 -- 172.21.15.16:0/884197277 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5680b1f40 con 0x7ff568143720 2023-12-08T08:39:33.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.477+0000 7ff565d9b640 1 --2- 172.21.15.16:0/884197277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff568143720 0x7ff568071600 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0097a0 tx=0x7ff55c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11c4e048e1e93d8 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:33.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.477+0000 7ff56559a640 1 -- 172.21.15.16:0/884197277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c031830 con 0x7ff568143720 2023-12-08T08:39:33.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.477+0000 7ff56559a640 1 -- 172.21.15.16:0/884197277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55c031990 con 0x7ff568143720 2023-12-08T08:39:33.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.477+0000 7ff56559a640 1 -- 172.21.15.16:0/884197277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c031d20 con 0x7ff568143720 2023-12-08T08:39:33.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.478+0000 7ff56ce90640 1 -- 172.21.15.16:0/884197277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff568143720 msgr2=0x7ff568071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:33.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.478+0000 7ff56ce90640 1 --2- 172.21.15.16:0/884197277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff568143720 0x7ff568071600 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0097a0 tx=0x7ff55c02b8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:33.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.478+0000 7ff56ce90640 1 -- 172.21.15.16:0/884197277 shutdown_connections 2023-12-08T08:39:33.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.478+0000 7ff56ce90640 1 --2- 172.21.15.16:0/884197277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff568143720 0x7ff568071600 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:33.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.478+0000 7ff56ce90640 1 -- 172.21.15.16:0/884197277 >> 172.21.15.16:0/884197277 conn(0x7ff5680ceba0 msgr2=0x7ff5680cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:33.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.478+0000 7ff56ce90640 1 -- 172.21.15.16:0/884197277 shutdown_connections 2023-12-08T08:39:33.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.478+0000 7ff56ce90640 1 -- 172.21.15.16:0/884197277 wait complete. 2023-12-08T08:39:33.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.479+0000 7ff56ce90640 1 Processor -- start 2023-12-08T08:39:33.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.479+0000 7ff56ce90640 1 -- start start 2023-12-08T08:39:33.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.480+0000 7ff56ce90640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff568071220 0x7ff5681f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:33.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.480+0000 7ff56ce90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5680b2d00 0x7ff5681eaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.480+0000 7ff56ce90640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff568143720 0x7ff5681f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.480+0000 7ff56ce90640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5681f4c70 con 0x7ff5680b2d00 2023-12-08T08:39:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.480+0000 7ff56ce90640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5681f4f50 con 0x7ff568143720 2023-12-08T08:39:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.480+0000 7ff56ce90640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5681f0f50 con 0x7ff568071220 2023-12-08T08:39:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.480+0000 7ff565d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5680b2d00 0x7ff5681eaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.480+0000 7ff565d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5680b2d00 0x7ff5681eaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47746/0 (socket says 172.21.15.16:47746) 2023-12-08T08:39:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.480+0000 7ff565d9b640 1 -- 172.21.15.16:0/3898762630 learned_addr learned my addr 172.21.15.16:0/3898762630 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.480+0000 7ff566d9d640 1 --2- 172.21.15.16:0/3898762630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff568143720 0x7ff5681f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.480+0000 7ff56659c640 1 --2- 172.21.15.16:0/3898762630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff568071220 0x7ff5681f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.481+0000 7ff565d9b640 1 -- 172.21.15.16:0/3898762630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff568071220 msgr2=0x7ff5681f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.481+0000 7ff565d9b640 1 --2- 172.21.15.16:0/3898762630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff568071220 0x7ff5681f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.481+0000 7ff565d9b640 1 -- 172.21.15.16:0/3898762630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff568143720 msgr2=0x7ff5681f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:33.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.481+0000 7ff565d9b640 1 --2- 172.21.15.16:0/3898762630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff568143720 0x7ff5681f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.481+0000 7ff565d9b640 1 -- 172.21.15.16:0/3898762630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55c009400 con 0x7ff5680b2d00 2023-12-08T08:39:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.481+0000 7ff565d9b640 1 --2- 172.21.15.16:0/3898762630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5680b2d00 0x7ff5681eaa20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff55c000c00 tx=0x7ff55c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.481+0000 7ff556ffd640 1 -- 172.21.15.16:0/3898762630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c03aa60 con 0x7ff5680b2d00 2023-12-08T08:39:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.481+0000 7ff556ffd640 1 -- 172.21.15.16:0/3898762630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55c03abc0 con 0x7ff5680b2d00 2023-12-08T08:39:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.481+0000 7ff56ce90640 1 -- 172.21.15.16:0/3898762630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5681f1230 con 0x7ff5680b2d00 2023-12-08T08:39:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.481+0000 7ff556ffd640 1 -- 172.21.15.16:0/3898762630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0443f0 con 0x7ff5680b2d00 2023-12-08T08:39:33.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.481+0000 7ff56ce90640 1 -- 172.21.15.16:0/3898762630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5681eaf60 con 0x7ff5680b2d00 2023-12-08T08:39:33.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.482+0000 7ff56ce90640 1 -- 172.21.15.16:0/3898762630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5681f1440 con 0x7ff5680b2d00 2023-12-08T08:39:33.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.483+0000 7ff556ffd640 1 -- 172.21.15.16:0/3898762630 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff55c04d450 con 0x7ff5680b2d00 2023-12-08T08:39:33.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.483+0000 7ff556ffd640 1 --2- 172.21.15.16:0/3898762630 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff54806ae40 0x7ff54806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:33.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.483+0000 7ff556ffd640 1 -- 172.21.15.16:0/3898762630 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5781+0+0 (secure 0 0 0) 0x7ff55c0ab740 con 0x7ff5680b2d00 2023-12-08T08:39:33.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.483+0000 7ff56659c640 1 --2- 172.21.15.16:0/3898762630 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff54806ae40 0x7ff54806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:33.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.484+0000 7ff56659c640 1 --2- 172.21.15.16:0/3898762630 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff54806ae40 0x7ff54806d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff56813f150 tx=0x7ff560007e30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:33.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.487+0000 7ff556ffd640 1 -- 172.21.15.16:0/3898762630 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff55c076850 con 0x7ff5680b2d00 2023-12-08T08:39:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:33.662+0000 7ff56ce90640 1 -- 172.21.15.16:0/3898762630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff568067540 con 0x7ff5680b2d00 2023-12-08T08:39:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.338+0000 7ff556ffd640 1 -- 172.21.15.16:0/3898762630 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7ff55c035020 con 0x7ff5680b2d00 2023-12-08T08:39:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data_ec' removed 2023-12-08T08:39:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.341+0000 7ff56ce90640 1 -- 172.21.15.16:0/3898762630 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff54806ae40 msgr2=0x7ff54806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.341+0000 7ff56ce90640 1 --2- 172.21.15.16:0/3898762630 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff54806ae40 0x7ff54806d300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff56813f150 tx=0x7ff560007e30 comp rx=0 tx=0).stop 2023-12-08T08:39:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.341+0000 7ff56ce90640 1 -- 172.21.15.16:0/3898762630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5680b2d00 msgr2=0x7ff5681eaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.341+0000 7ff56ce90640 1 --2- 172.21.15.16:0/3898762630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5680b2d00 0x7ff5681eaa20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff55c000c00 tx=0x7ff55c0046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.341+0000 7ff56ce90640 1 -- 172.21.15.16:0/3898762630 shutdown_connections 2023-12-08T08:39:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.341+0000 7ff56ce90640 1 --2- 172.21.15.16:0/3898762630 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff54806ae40 0x7ff54806d300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.341+0000 7ff56ce90640 1 --2- 172.21.15.16:0/3898762630 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff568143720 0x7ff5681f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.341+0000 7ff56ce90640 1 --2- 172.21.15.16:0/3898762630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5680b2d00 0x7ff5681eaa20 secure :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff55c000c00 tx=0x7ff55c0046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.341+0000 7ff56ce90640 1 --2- 172.21.15.16:0/3898762630 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff568071220 0x7ff5681f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.341+0000 7ff56ce90640 1 -- 172.21.15.16:0/3898762630 >> 172.21.15.16:0/3898762630 conn(0x7ff5680ceba0 msgr2=0x7ff5680ccbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.343+0000 7ff56ce90640 1 -- 172.21.15.16:0/3898762630 shutdown_connections 2023-12-08T08:39:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.343+0000 7ff56ce90640 1 -- 172.21.15.16:0/3898762630 wait complete. 2023-12-08T08:39:34.362 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-08T08:39:34.362 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-08T08:39:34.363 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-08T08:39:34.363 INFO:teuthology.orchestra.run:waiting for 300 2023-12-08T08:39:34.363 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-08T08:39:34.363 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-08T08:39:34.363 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-08T08:39:34.363 INFO:teuthology.orchestra.run:waiting for 300 2023-12-08T08:39:34.363 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-08T08:39:34.363 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-08T08:39:34.364 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-08T08:39:34.364 INFO:teuthology.orchestra.run:waiting for 300 2023-12-08T08:39:34.364 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-08T08:39:34.364 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-08T08:39:34.364 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-08T08:39:34.364 INFO:teuthology.orchestra.run:waiting for 300 2023-12-08T08:39:34.373 INFO:tasks.ceph.mds.c:Stopped 2023-12-08T08:39:34.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-08T08:39:34.374 INFO:tasks.ceph.mds.a:Stopped 2023-12-08T08:39:34.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-08T08:39:34.377 INFO:tasks.ceph.mds.b:Stopped 2023-12-08T08:39:34.377 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-08T08:39:34.378 INFO:tasks.ceph.mds.d:Stopped 2023-12-08T08:39:34.378 DEBUG:teuthology.orchestra.run.smithi112:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-08T08:39:34.416 INFO:tasks.ceph.mds.c:Started 2023-12-08T08:39:34.416 DEBUG:teuthology.parallel:result is None 2023-12-08T08:39:34.417 INFO:tasks.ceph.mds.a:Started 2023-12-08T08:39:34.417 DEBUG:teuthology.parallel:result is None 2023-12-08T08:39:34.420 INFO:tasks.ceph.mds.b:Started 2023-12-08T08:39:34.420 DEBUG:teuthology.parallel:result is None 2023-12-08T08:39:34.421 INFO:tasks.ceph.mds.d:Started 2023-12-08T08:39:34.422 DEBUG:teuthology.parallel:result is None 2023-12-08T08:39:34.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-08T08:39:34.499 INFO:tasks.ceph.mds.d.smithi112.stdout:starting mds.d at 2023-12-08T08:39:34.500 INFO:tasks.ceph.mds.b.smithi112.stdout:starting mds.b at 2023-12-08T08:39:34.503 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-12-08T08:39:34.503 INFO:tasks.ceph.mds.c.smithi016.stdout:starting mds.c at 2023-12-08T08:39:34.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.536+0000 7f5e79fa5640 1 Processor -- start 2023-12-08T08:39:34.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.537+0000 7f5e79fa5640 1 -- start start 2023-12-08T08:39:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.537+0000 7f5e79fa5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e740cc660 0x7f5e740cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.537+0000 7f5e79fa5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f5e740c63b0 0x7f5e740718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.537+0000 7f5e79fa5640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e740cc130 con 0x7f5e740c5f30 2023-12-08T08:39:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.537+0000 7f5e79fa5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e740cefa0 con 0x7f5e740cc660 2023-12-08T08:39:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.537+0000 7f5e79fa5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7406d460 con 0x7f5e740c63b0 2023-12-08T08:39:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.537+0000 7f5e737fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e740cc660 0x7f5e740cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.538+0000 7f5e737fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e740cc660 0x7f5e740cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47786/0 (socket says 172.21.15.16:47786) 2023-12-08T08:39:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.538+0000 7f5e737fe640 1 -- 172.21.15.16:0/358185290 learned_addr learned my addr 172.21.15.16:0/358185290 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.538+0000 7f5e73fff640 1 --2- 172.21.15.16:0/358185290 >> v2:172.21.15.112:3300/0 conn(0x7f5e740c63b0 0x7f5e740718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.538+0000 7f5e737fe640 1 -- 172.21.15.16:0/358185290 >> v2:172.21.15.112:3300/0 conn(0x7f5e740c63b0 msgr2=0x7f5e740718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.538+0000 7f5e737fe640 1 --2- 172.21.15.16:0/358185290 >> v2:172.21.15.112:3300/0 conn(0x7f5e740c63b0 0x7f5e740718f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.538+0000 7f5e737fe640 1 -- 172.21.15.16:0/358185290 >> v1:172.21.15.112:6789/0 conn(0x7f5e740c5f30 legacy=0x7f5e740711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.538+0000 7f5e737fe640 1 -- 172.21.15.16:0/358185290 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e740ccf80 con 0x7f5e740cc660 2023-12-08T08:39:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.539+0000 7f5e737fe640 1 --2- 172.21.15.16:0/358185290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e740cc660 0x7f5e740cca40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5e740c6d00 tx=0x7f5e6802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344be400f39d243b server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.539+0000 7f5e727fc640 1 -- 172.21.15.16:0/358185290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e68031870 con 0x7f5e740cc660 2023-12-08T08:39:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.539+0000 7f5e727fc640 1 -- 172.21.15.16:0/358185290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e680319d0 con 0x7f5e740cc660 2023-12-08T08:39:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.539+0000 7f5e727fc640 1 -- 172.21.15.16:0/358185290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e68031d60 con 0x7f5e740cc660 2023-12-08T08:39:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.539+0000 7f5e79fa5640 1 -- 172.21.15.16:0/358185290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e740cc660 msgr2=0x7f5e740cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.539+0000 7f5e79fa5640 1 --2- 172.21.15.16:0/358185290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e740cc660 0x7f5e740cca40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5e740c6d00 tx=0x7f5e6802b910 comp rx=0 tx=0).stop 2023-12-08T08:39:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.539+0000 7f5e79fa5640 1 -- 172.21.15.16:0/358185290 shutdown_connections 2023-12-08T08:39:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.539+0000 7f5e79fa5640 1 --2- 172.21.15.16:0/358185290 >> v2:172.21.15.112:3300/0 conn(0x7f5e740c63b0 0x7f5e740718f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.539+0000 7f5e79fa5640 1 --2- 172.21.15.16:0/358185290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e740cc660 0x7f5e740cca40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.539+0000 7f5e79fa5640 1 -- 172.21.15.16:0/358185290 >> 172.21.15.16:0/358185290 conn(0x7f5e740b1890 msgr2=0x7f5e740b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.539+0000 7f5e79fa5640 1 -- 172.21.15.16:0/358185290 shutdown_connections 2023-12-08T08:39:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.539+0000 7f5e79fa5640 1 -- 172.21.15.16:0/358185290 wait complete. 2023-12-08T08:39:34.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.540+0000 7f5e79fa5640 1 Processor -- start 2023-12-08T08:39:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.541+0000 7f5e79fa5640 1 -- start start 2023-12-08T08:39:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.541+0000 7f5e79fa5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e740c5f30 0x7f5e74099a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.541+0000 7f5e79fa5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e740c63b0 0x7f5e7414fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.541+0000 7f5e79fa5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e740cc660 0x7f5e740b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.541+0000 7f5e79fa5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7409c330 con 0x7f5e740c63b0 2023-12-08T08:39:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.541+0000 7f5e79fa5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7409d130 con 0x7f5e740c5f30 2023-12-08T08:39:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.541+0000 7f5e79fa5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7409c560 con 0x7f5e740cc660 2023-12-08T08:39:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.541+0000 7f5e72ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e740c63b0 0x7f5e7414fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.541+0000 7f5e72ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e740c63b0 0x7f5e7414fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47792/0 (socket says 172.21.15.16:47792) 2023-12-08T08:39:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.541+0000 7f5e72ffd640 1 -- 172.21.15.16:0/1884410258 learned_addr learned my addr 172.21.15.16:0/1884410258 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:34.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.542+0000 7f5e737fe640 1 --2- 172.21.15.16:0/1884410258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e740c5f30 0x7f5e74099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.542+0000 7f5e73fff640 1 --2- 172.21.15.16:0/1884410258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e740cc660 0x7f5e740b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.542+0000 7f5e72ffd640 1 -- 172.21.15.16:0/1884410258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e740cc660 msgr2=0x7f5e740b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.542+0000 7f5e72ffd640 1 --2- 172.21.15.16:0/1884410258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e740cc660 0x7f5e740b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.542+0000 7f5e72ffd640 1 -- 172.21.15.16:0/1884410258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e740c5f30 msgr2=0x7f5e74099a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.542+0000 7f5e72ffd640 1 --2- 172.21.15.16:0/1884410258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e740c5f30 0x7f5e74099a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.542+0000 7f5e72ffd640 1 -- 172.21.15.16:0/1884410258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e680094d0 con 0x7f5e740c63b0 2023-12-08T08:39:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.542+0000 7f5e72ffd640 1 --2- 172.21.15.16:0/1884410258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e740c63b0 0x7f5e7414fad0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400b3f0 tx=0x7f5e6400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.542+0000 7f5e5bfff640 1 -- 172.21.15.16:0/1884410258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e64011830 con 0x7f5e740c63b0 2023-12-08T08:39:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.542+0000 7f5e5bfff640 1 -- 172.21.15.16:0/1884410258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e64011990 con 0x7f5e740c63b0 2023-12-08T08:39:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.542+0000 7f5e5bfff640 1 -- 172.21.15.16:0/1884410258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e64011d40 con 0x7f5e740c63b0 2023-12-08T08:39:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.542+0000 7f5e79fa5640 1 -- 172.21.15.16:0/1884410258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7409c800 con 0x7f5e740c63b0 2023-12-08T08:39:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.542+0000 7f5e79fa5640 1 -- 172.21.15.16:0/1884410258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7409cdc0 con 0x7f5e740c63b0 2023-12-08T08:39:34.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.543+0000 7f5e79fa5640 1 -- 172.21.15.16:0/1884410258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7409d2c0 con 0x7f5e740c63b0 2023-12-08T08:39:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.544+0000 7f5e5bfff640 1 -- 172.21.15.16:0/1884410258 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5e6401a400 con 0x7f5e740c63b0 2023-12-08T08:39:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.544+0000 7f5e5bfff640 1 --2- 172.21.15.16:0/1884410258 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5e6006ae70 0x7f5e6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.544+0000 7f5e5bfff640 1 -- 172.21.15.16:0/1884410258 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f5e640892d0 con 0x7f5e740c63b0 2023-12-08T08:39:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.544+0000 7f5e737fe640 1 --2- 172.21.15.16:0/1884410258 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5e6006ae70 0x7f5e6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:34.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.545+0000 7f5e737fe640 1 --2- 172.21.15.16:0/1884410258 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5e6006ae70 0x7f5e6006d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5e68036000 tx=0x7f5e6803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:34.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.548+0000 7f5e5bfff640 1 -- 172.21.15.16:0/1884410258 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5e64054540 con 0x7f5e740c63b0 2023-12-08T08:39:34.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:34.745+0000 7f5e79fa5640 1 -- 172.21.15.16:0/1884410258 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5e74067ad0 con 0x7f5e740c63b0 2023-12-08T08:39:35.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.647+0000 7f5e5bfff640 1 -- 172.21.15.16:0/1884410258 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5e64057b60 con 0x7f5e740c63b0 2023-12-08T08:39:35.649 INFO:teuthology.orchestra.run.smithi016.stderr: removed all blocklist entries 2023-12-08T08:39:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.651+0000 7f5e79fa5640 1 -- 172.21.15.16:0/1884410258 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5e6006ae70 msgr2=0x7f5e6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.651+0000 7f5e79fa5640 1 --2- 172.21.15.16:0/1884410258 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5e6006ae70 0x7f5e6006d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5e68036000 tx=0x7f5e6803c040 comp rx=0 tx=0).stop 2023-12-08T08:39:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.651+0000 7f5e79fa5640 1 -- 172.21.15.16:0/1884410258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e740c63b0 msgr2=0x7f5e7414fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.651+0000 7f5e79fa5640 1 --2- 172.21.15.16:0/1884410258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e740c63b0 0x7f5e7414fad0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400b3f0 tx=0x7f5e6400b8b0 comp rx=0 tx=0).stop 2023-12-08T08:39:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.651+0000 7f5e79fa5640 1 -- 172.21.15.16:0/1884410258 shutdown_connections 2023-12-08T08:39:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.651+0000 7f5e79fa5640 1 --2- 172.21.15.16:0/1884410258 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5e6006ae70 0x7f5e6006d330 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.651+0000 7f5e79fa5640 1 --2- 172.21.15.16:0/1884410258 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5e740cc660 0x7f5e740b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.651+0000 7f5e79fa5640 1 --2- 172.21.15.16:0/1884410258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e740c63b0 0x7f5e7414fad0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.651+0000 7f5e79fa5640 1 --2- 172.21.15.16:0/1884410258 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5e740c5f30 0x7f5e74099a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.651+0000 7f5e79fa5640 1 -- 172.21.15.16:0/1884410258 >> 172.21.15.16:0/1884410258 conn(0x7f5e740b1890 msgr2=0x7f5e740caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.651+0000 7f5e79fa5640 1 -- 172.21.15.16:0/1884410258 shutdown_connections 2023-12-08T08:39:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.651+0000 7f5e79fa5640 1 -- 172.21.15.16:0/1884410258 wait complete. 2023-12-08T08:39:35.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-08T08:39:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.779+0000 7f3712189640 1 Processor -- start 2023-12-08T08:39:35.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.779+0000 7f3712189640 1 -- start start 2023-12-08T08:39:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.780+0000 7f3712189640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f370c06a4d0 0x7f370c06a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.780+0000 7f3712189640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f370c072010 0x7f370c15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.780+0000 7f3712189640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c15fb20 con 0x7f370c15f5b0 2023-12-08T08:39:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.780+0000 7f3712189640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c0cf240 con 0x7f370c072010 2023-12-08T08:39:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.780+0000 7f3712189640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c15b850 con 0x7f370c06a4d0 2023-12-08T08:39:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.780+0000 7f370affd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f370c15f5b0 0x7f370c0ca2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33332/0 (socket says 172.21.15.16:33332) 2023-12-08T08:39:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.780+0000 7f370affd640 1 -- 172.21.15.16:0/1415415481 learned_addr learned my addr 172.21.15.16:0/1415415481 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.780+0000 7f370b7fe640 1 --2- 172.21.15.16:0/1415415481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f370c06a4d0 0x7f370c06a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.780+0000 7f370bfff640 1 --2- 172.21.15.16:0/1415415481 >> v2:172.21.15.112:3300/0 conn(0x7f370c072010 0x7f370c15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.780+0000 7f370a7fc640 1 -- 172.21.15.16:0/1415415481 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67218110 0 0) 0x7f370c15fb20 con 0x7f370c15f5b0 2023-12-08T08:39:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.780+0000 7f370a7fc640 1 -- 172.21.15.16:0/1415415481 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36f4003520 con 0x7f370c15f5b0 2023-12-08T08:39:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.781+0000 7f370a7fc640 1 -- 172.21.15.16:0/1415415481 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2421579183 0 0) 0x7f36f4003520 con 0x7f370c15f5b0 2023-12-08T08:39:35.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.781+0000 7f370bfff640 1 -- 172.21.15.16:0/1415415481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f370c06a4d0 msgr2=0x7f370c06a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.781+0000 7f370bfff640 1 --2- 172.21.15.16:0/1415415481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f370c06a4d0 0x7f370c06a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.781+0000 7f370bfff640 1 -- 172.21.15.16:0/1415415481 >> v1:172.21.15.16:6789/0 conn(0x7f370c15f5b0 legacy=0x7f370c0ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.781+0000 7f370bfff640 1 -- 172.21.15.16:0/1415415481 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c06adf0 con 0x7f370c072010 2023-12-08T08:39:35.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.781+0000 7f370b7fe640 1 --2- 172.21.15.16:0/1415415481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f370c06a4d0 0x7f370c06a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:39:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.782+0000 7f370bfff640 1 --2- 172.21.15.16:0/1415415481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f370c072010 0x7f370c15bca0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3704009dc0 tx=0x7f370402bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6708dd0951e1c38 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.782+0000 7f370a7fc640 1 -- 172.21.15.16:0/1415415481 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3700001670 con 0x7f370c072010 2023-12-08T08:39:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.782+0000 7f370a7fc640 1 -- 172.21.15.16:0/1415415481 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370402c880 con 0x7f370c072010 2023-12-08T08:39:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.782+0000 7f370a7fc640 1 -- 172.21.15.16:0/1415415481 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370402cc10 con 0x7f370c072010 2023-12-08T08:39:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.782+0000 7f3712189640 1 -- 172.21.15.16:0/1415415481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f370c072010 msgr2=0x7f370c15bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.782+0000 7f3712189640 1 --2- 172.21.15.16:0/1415415481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f370c072010 0x7f370c15bca0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3704009dc0 tx=0x7f370402bd50 comp rx=0 tx=0).stop 2023-12-08T08:39:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.782+0000 7f3712189640 1 -- 172.21.15.16:0/1415415481 shutdown_connections 2023-12-08T08:39:35.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.782+0000 7f3712189640 1 --2- 172.21.15.16:0/1415415481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f370c072010 0x7f370c15bca0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.782+0000 7f3712189640 1 --2- 172.21.15.16:0/1415415481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f370c06a4d0 0x7f370c06a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.783+0000 7f3712189640 1 -- 172.21.15.16:0/1415415481 >> 172.21.15.16:0/1415415481 conn(0x7f370c0b1890 msgr2=0x7f370c0b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.783+0000 7f3712189640 1 -- 172.21.15.16:0/1415415481 shutdown_connections 2023-12-08T08:39:35.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.783+0000 7f3712189640 1 -- 172.21.15.16:0/1415415481 wait complete. 2023-12-08T08:39:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.784+0000 7f3712189640 1 Processor -- start 2023-12-08T08:39:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.784+0000 7f3712189640 1 -- start start 2023-12-08T08:39:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f3712189640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f370c06a4d0 0x7f370c0b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f3712189640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f370c072010 0x7f370c156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f3712189640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c15f5b0 0x7f370c0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f3712189640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0c7870 con 0x7f370c15f5b0 2023-12-08T08:39:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f3712189640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c0b2ec0 con 0x7f370c06a4d0 2023-12-08T08:39:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f3712189640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c156db0 con 0x7f370c072010 2023-12-08T08:39:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f370bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c15f5b0 0x7f370c0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f370bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c15f5b0 0x7f370c0adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47794/0 (socket says 172.21.15.16:47794) 2023-12-08T08:39:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f370bfff640 1 -- 172.21.15.16:0/4257103772 learned_addr learned my addr 172.21.15.16:0/4257103772 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f370b7fe640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f370c06a4d0 0x7f370c0b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f370affd640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f370c072010 0x7f370c156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f370bfff640 1 -- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f370c072010 msgr2=0x7f370c156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f370bfff640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f370c072010 0x7f370c156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f370bfff640 1 -- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f370c06a4d0 msgr2=0x7f370c0b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f370bfff640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f370c06a4d0 0x7f370c0b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f370bfff640 1 -- 172.21.15.16:0/4257103772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc009400 con 0x7f370c15f5b0 2023-12-08T08:39:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f370b7fe640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f370c06a4d0 0x7f370c0b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f370affd640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f370c072010 0x7f370c156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f370bfff640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c15f5b0 0x7f370c0adda0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3704009ee0 tx=0x7f3704004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.785+0000 7f36f3fff640 1 -- 172.21.15.16:0/4257103772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704034e50 con 0x7f370c15f5b0 2023-12-08T08:39:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.786+0000 7f36f3fff640 1 -- 172.21.15.16:0/4257103772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370402ce90 con 0x7f370c15f5b0 2023-12-08T08:39:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.786+0000 7f3712189640 1 -- 172.21.15.16:0/4257103772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3704009a70 con 0x7f370c15f5b0 2023-12-08T08:39:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.786+0000 7f36f3fff640 1 -- 172.21.15.16:0/4257103772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704040670 con 0x7f370c15f5b0 2023-12-08T08:39:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.786+0000 7f3712189640 1 -- 172.21.15.16:0/4257103772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c0c6ed0 con 0x7f370c15f5b0 2023-12-08T08:39:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.787+0000 7f3712189640 1 -- 172.21.15.16:0/4257103772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c0c79b0 con 0x7f370c15f5b0 2023-12-08T08:39:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.787+0000 7f36f3fff640 1 -- 172.21.15.16:0/4257103772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f37040407d0 con 0x7f370c15f5b0 2023-12-08T08:39:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.787+0000 7f36f3fff640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f36f806ae70 0x7f36f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.788+0000 7f36f3fff640 1 -- 172.21.15.16:0/4257103772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f37040ac110 con 0x7f370c15f5b0 2023-12-08T08:39:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.788+0000 7f370b7fe640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f36f806ae70 0x7f36f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.788+0000 7f370b7fe640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f36f806ae70 0x7f36f806d330 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f370c1f4150 tx=0x7f36fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:35.792+0000 7f36f3fff640 1 -- 172.21.15.16:0/4257103772 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f370c0c79b0 con 0x7f370c15f5b0 2023-12-08T08:39:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.040+0000 7f3712189640 1 -- 172.21.15.16:0/4257103772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f370c0c79b0 con 0x7f370c15f5b0 2023-12-08T08:39:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.040+0000 7f36f3fff640 1 -- 172.21.15.16:0/4257103772 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f370c0c79b0 con 0x7f370c15f5b0 2023-12-08T08:39:36.042 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:36.042 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-12-08T08:39:36.042 INFO:teuthology.orchestra.run.smithi016.stdout: "auth_dump": [ 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.a", 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC41XJlEfknLhAAU0MfMMmtFok98FUnhfdwNg==", 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.b", 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC41XJl9kd7OxAACKRXjQrIJORGtWKvnIEB/A==", 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-12-08T08:39:36.043 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-12-08T08:39:36.044 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.044 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.044 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.044 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.c", 2023-12-08T08:39:36.044 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC41XJleAE4NhAA6vzrXiYiWRXoRHnl8grXgA==", 2023-12-08T08:39:36.044 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.044 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-12-08T08:39:36.044 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-12-08T08:39:36.044 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-12-08T08:39:36.044 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.044 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.044 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mds.d", 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC51XJlxKNJBxAAU7UWljdMrMl28PCh3lciDw==", 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.0", 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC61XJlChJvMhAA27OGl0JnI2oX15qX88jApw==", 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-12-08T08:39:36.045 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-12-08T08:39:36.046 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.046 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.046 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.046 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.1", 2023-12-08T08:39:36.046 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC91XJlST/oMBAA/jWxkH8dw+39wzBQJ2pBsQ==", 2023-12-08T08:39:36.046 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.046 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-12-08T08:39:36.046 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-12-08T08:39:36.046 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.046 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.047 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.047 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.2", 2023-12-08T08:39:36.047 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDA1XJlEG+SLxAAgAN/6Fv/H7bMQHjlmUHgnQ==", 2023-12-08T08:39:36.047 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.047 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-12-08T08:39:36.047 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-12-08T08:39:36.047 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.047 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.047 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.047 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.3", 2023-12-08T08:39:36.047 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDD1XJl4kQQLhAAsevNUXS8KlnqlhydabjVmw==", 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.4", 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDJ1XJlUSUBOBAAjg1B3YnxHRRfUyfjtVcsbA==", 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.049 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.049 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.5", 2023-12-08T08:39:36.049 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDM1XJlrpjLNhAAZ9K/AYfVZ6CEHwqDKED/+w==", 2023-12-08T08:39:36.049 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.049 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-12-08T08:39:36.049 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-12-08T08:39:36.049 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.049 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.049 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.049 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.6", 2023-12-08T08:39:36.049 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDP1XJlGLJKNRAAP42GAoFBFBgKqP8646NM3w==", 2023-12-08T08:39:36.049 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.050 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-12-08T08:39:36.050 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-12-08T08:39:36.050 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.050 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.050 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.050 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "osd.7", 2023-12-08T08:39:36.050 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDS1XJlUZDEMxAAE5FgV0KgG4sKzIPFzbxKzA==", 2023-12-08T08:39:36.050 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.050 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow profile osd", 2023-12-08T08:39:36.050 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile osd", 2023-12-08T08:39:36.050 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.051 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.051 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.051 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.0", 2023-12-08T08:39:36.051 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC51XJlZAHPCxAAZLN3jMh+Z9XRet/zdd6MzQ==", 2023-12-08T08:39:36.051 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.051 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-12-08T08:39:36.051 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-12-08T08:39:36.051 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-12-08T08:39:36.051 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-12-08T08:39:36.051 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.051 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.051 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.1", 2023-12-08T08:39:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC51XJllr1UExAAo9PcR6ozHGuR4DIMLF0vlg==", 2023-12-08T08:39:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-12-08T08:39:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-12-08T08:39:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-12-08T08:39:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-12-08T08:39:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.2", 2023-12-08T08:39:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC51XJljiayDxAAetWjq+LR+1gYuybmNhrl7A==", 2023-12-08T08:39:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-12-08T08:39:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-12-08T08:39:36.053 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-12-08T08:39:36.053 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-12-08T08:39:36.053 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.053 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.053 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.053 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.3", 2023-12-08T08:39:36.053 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC51XJl2R4aGRAABYH/VhDDrsPHG9bbsDa52A==", 2023-12-08T08:39:36.053 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.053 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow", 2023-12-08T08:39:36.053 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow r", 2023-12-08T08:39:36.053 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow rw", 2023-12-08T08:39:36.053 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow rwx" 2023-12-08T08:39:36.053 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.053 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.054 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.054 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.admin", 2023-12-08T08:39:36.054 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC41XJlmZ4qEBAAXeuHA73CaY3xR9ZyBLHyDQ==", 2023-12-08T08:39:36.054 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.054 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-12-08T08:39:36.054 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": "allow *", 2023-12-08T08:39:36.054 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow *", 2023-12-08T08:39:36.054 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.054 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.054 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.054 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mds", 2023-12-08T08:39:36.054 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1XJlIzhWMxAAl0sMCzbA1zBSD+wQ/E/eNg==", 2023-12-08T08:39:36.055 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.055 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mds" 2023-12-08T08:39:36.055 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.055 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.055 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-mgr", 2023-12-08T08:39:36.055 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1XJll0BWMxAAJ7y+gMFxpxPwLRrEpbkL5g==", 2023-12-08T08:39:36.055 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.055 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-mgr" 2023-12-08T08:39:36.055 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.055 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.055 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-osd", 2023-12-08T08:39:36.055 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1XJlQ0hWMxAAt5DrhiylCmkLMXqF6q+9FA==", 2023-12-08T08:39:36.056 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.056 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-osd" 2023-12-08T08:39:36.056 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.056 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.056 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd", 2023-12-08T08:39:36.056 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1XJl309WMxAA11VhGo6DPDRmEIiCb37n/g==", 2023-12-08T08:39:36.056 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.056 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd" 2023-12-08T08:39:36.056 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.056 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.057 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-12-08T08:39:36.057 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1XJlo1hWMxAA8iwGmPWpf0beMqwnMfkK0g==", 2023-12-08T08:39:36.057 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.057 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-12-08T08:39:36.057 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.057 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.057 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "client.bootstrap-rgw", 2023-12-08T08:39:36.057 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQDi1XJl9mJWMxAAxZaPGyyRXd6D+tp5YBdk+w==", 2023-12-08T08:39:36.057 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.057 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile bootstrap-rgw" 2023-12-08T08:39:36.057 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.058 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.058 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.x", 2023-12-08T08:39:36.058 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC41XJllUExKhAAqR4aRqeRBt4ibrlTNOD0xQ==", 2023-12-08T08:39:36.058 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.058 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-12-08T08:39:36.058 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-12-08T08:39:36.058 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.058 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:36.058 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:36.058 INFO:teuthology.orchestra.run.smithi016.stdout: "entity": "mgr.y", 2023-12-08T08:39:36.058 INFO:teuthology.orchestra.run.smithi016.stdout: "key": "AQC41XJlcSdvJhAAXSnQAgTJ8oR3ZaObFc0fWw==", 2023-12-08T08:39:36.058 INFO:teuthology.orchestra.run.smithi016.stdout: "caps": { 2023-12-08T08:39:36.059 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": "allow *", 2023-12-08T08:39:36.059 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": "allow profile mgr", 2023-12-08T08:39:36.059 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": "allow *" 2023-12-08T08:39:36.059 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.059 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:36.059 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-12-08T08:39:36.059 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-12-08T08:39:36.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.042+0000 7f3712189640 1 -- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f36f806ae70 msgr2=0x7f36f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.042+0000 7f3712189640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f36f806ae70 0x7f36f806d330 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f370c1f4150 tx=0x7f36fc008040 comp rx=0 tx=0).stop 2023-12-08T08:39:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.042+0000 7f3712189640 1 -- 172.21.15.16:0/4257103772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c15f5b0 msgr2=0x7f370c0adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.042+0000 7f3712189640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c15f5b0 0x7f370c0adda0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3704009ee0 tx=0x7f3704004a40 comp rx=0 tx=0).stop 2023-12-08T08:39:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.042+0000 7f3712189640 1 -- 172.21.15.16:0/4257103772 shutdown_connections 2023-12-08T08:39:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.042+0000 7f3712189640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f36f806ae70 0x7f36f806d330 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.042+0000 7f3712189640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f370c15f5b0 0x7f370c0adda0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.042+0000 7f3712189640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f370c072010 0x7f370c156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.042+0000 7f3712189640 1 --2- 172.21.15.16:0/4257103772 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f370c06a4d0 0x7f370c0b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.042+0000 7f3712189640 1 -- 172.21.15.16:0/4257103772 >> 172.21.15.16:0/4257103772 conn(0x7f370c0b1890 msgr2=0x7f370c06dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.042+0000 7f3712189640 1 -- 172.21.15.16:0/4257103772 shutdown_connections 2023-12-08T08:39:36.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.042+0000 7f3712189640 1 -- 172.21.15.16:0/4257103772 wait complete. 2023-12-08T08:39:36.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-12-08T08:39:36.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.168+0000 7f1b4f9c5640 1 Processor -- start 2023-12-08T08:39:36.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.168+0000 7f1b4f9c5640 1 -- start start 2023-12-08T08:39:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.169+0000 7f1b4f9c5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b480cf760 0x7f1b4806a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.169+0000 7f1b4f9c5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b4815e150 0x7f1b48161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.169+0000 7f1b4f9c5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4806b070 con 0x7f1b480cf380 2023-12-08T08:39:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.169+0000 7f1b4f9c5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b48072290 con 0x7f1b4815e150 2023-12-08T08:39:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.169+0000 7f1b4f9c5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480d0d00 con 0x7f1b480cf760 2023-12-08T08:39:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.169+0000 7f1b4d761640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1b480cf380 0x7f1b48071b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33342/0 (socket says 172.21.15.16:33342) 2023-12-08T08:39:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.169+0000 7f1b4d761640 1 -- 172.21.15.16:0/3545683655 learned_addr learned my addr 172.21.15.16:0/3545683655 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.169+0000 7f1b4df62640 1 --2- 172.21.15.16:0/3545683655 >> v2:172.21.15.16:3300/0 conn(0x7f1b4815e150 0x7f1b48161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.169+0000 7f1b4df62640 1 -- 172.21.15.16:0/3545683655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b480cf760 msgr2=0x7f1b4806a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:36.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.169+0000 7f1b4df62640 1 --2- 172.21.15.16:0/3545683655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b480cf760 0x7f1b4806a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.169+0000 7f1b4df62640 1 -- 172.21.15.16:0/3545683655 >> v1:172.21.15.16:6789/0 conn(0x7f1b480cf380 legacy=0x7f1b48071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.169+0000 7f1b4df62640 1 -- 172.21.15.16:0/3545683655 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4806aab0 con 0x7f1b4815e150 2023-12-08T08:39:36.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.169+0000 7f1b3ffff640 1 -- 172.21.15.16:0/3545683655 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626071224 0 0) 0x7f1b4806b070 con 0x7f1b480cf380 2023-12-08T08:39:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.170+0000 7f1b4df62640 1 --2- 172.21.15.16:0/3545683655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b4815e150 0x7f1b48161690 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009dc0 tx=0x7f1b4002be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3495b556cf12a963 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.170+0000 7f1b3ffff640 1 -- 172.21.15.16:0/3545683655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4002cbb0 con 0x7f1b4815e150 2023-12-08T08:39:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.170+0000 7f1b3ffff640 1 -- 172.21.15.16:0/3545683655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4002cd10 con 0x7f1b4815e150 2023-12-08T08:39:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.170+0000 7f1b3ffff640 1 -- 172.21.15.16:0/3545683655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b40034660 con 0x7f1b4815e150 2023-12-08T08:39:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.170+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3545683655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b4815e150 msgr2=0x7f1b48161690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.170+0000 7f1b4f9c5640 1 --2- 172.21.15.16:0/3545683655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b4815e150 0x7f1b48161690 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009dc0 tx=0x7f1b4002be00 comp rx=0 tx=0).stop 2023-12-08T08:39:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.170+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3545683655 shutdown_connections 2023-12-08T08:39:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.170+0000 7f1b4f9c5640 1 --2- 172.21.15.16:0/3545683655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b4815e150 0x7f1b48161690 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.170+0000 7f1b4f9c5640 1 --2- 172.21.15.16:0/3545683655 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b480cf760 0x7f1b4806a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.170+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3545683655 >> 172.21.15.16:0/3545683655 conn(0x7f1b480cfda0 msgr2=0x7f1b480d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.171+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3545683655 shutdown_connections 2023-12-08T08:39:36.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.171+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3545683655 wait complete. 2023-12-08T08:39:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.172+0000 7f1b4f9c5640 1 Processor -- start 2023-12-08T08:39:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.172+0000 7f1b4f9c5640 1 -- start start 2023-12-08T08:39:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.172+0000 7f1b4f9c5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b480cf380 0x7f1b4815c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.172+0000 7f1b4f9c5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b480cf760 0x7f1b48157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.172+0000 7f1b4f9c5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b4815e150 0x7f1b480b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.172+0000 7f1b4f9c5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4815c680 con 0x7f1b480cf380 2023-12-08T08:39:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.172+0000 7f1b4f9c5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480ca380 con 0x7f1b480cf760 2023-12-08T08:39:36.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.172+0000 7f1b4f9c5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48157ea0 con 0x7f1b4815e150 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b4d761640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b480cf380 0x7f1b4815c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b4d761640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b480cf380 0x7f1b4815c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47820/0 (socket says 172.21.15.16:47820) 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b4d761640 1 -- 172.21.15.16:0/3158248743 learned_addr learned my addr 172.21.15.16:0/3158248743 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b4df62640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b4815e150 0x7f1b480b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b4cf60640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b480cf760 0x7f1b48157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b4d761640 1 -- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b4815e150 msgr2=0x7f1b480b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b4d761640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b4815e150 0x7f1b480b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b4d761640 1 -- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b480cf760 msgr2=0x7f1b48157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b4d761640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b480cf760 0x7f1b48157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b4d761640 1 -- 172.21.15.16:0/3158248743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b40009a70 con 0x7f1b480cf380 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b4cf60640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b480cf760 0x7f1b48157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b4df62640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b4815e150 0x7f1b480b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b4d761640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b480cf380 0x7f1b4815c140 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400c4d0 tx=0x7f1b4400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b3dffb640 1 -- 172.21.15.16:0/3158248743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b440074e0 con 0x7f1b480cf380 2023-12-08T08:39:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b3dffb640 1 -- 172.21.15.16:0/3158248743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b44007660 con 0x7f1b480cf380 2023-12-08T08:39:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3158248743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b480ca5d0 con 0x7f1b480cf380 2023-12-08T08:39:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.173+0000 7f1b3dffb640 1 -- 172.21.15.16:0/3158248743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44004a20 con 0x7f1b480cf380 2023-12-08T08:39:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.174+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3158248743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b480cab90 con 0x7f1b480cf380 2023-12-08T08:39:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.175+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3158248743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b480cadb0 con 0x7f1b480cf380 2023-12-08T08:39:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.175+0000 7f1b3dffb640 1 -- 172.21.15.16:0/3158248743 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1b44004b80 con 0x7f1b480cf380 2023-12-08T08:39:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.175+0000 7f1b3dffb640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1b2c06ae70 0x7f1b2c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.175+0000 7f1b3dffb640 1 -- 172.21.15.16:0/3158248743 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f1b44088520 con 0x7f1b480cf380 2023-12-08T08:39:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.175+0000 7f1b4cf60640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1b2c06ae70 0x7f1b2c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.176+0000 7f1b4cf60640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1b2c06ae70 0x7f1b2c06d330 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1b480b5d70 tx=0x7f1b38008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.180+0000 7f1b3dffb640 1 -- 172.21.15.16:0/3158248743 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1b44053870 con 0x7f1b480cf380 2023-12-08T08:39:36.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.421+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3158248743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f1b48067430 con 0x7f1b480cf380 2023-12-08T08:39:36.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.423+0000 7f1b3dffb640 1 -- 172.21.15.16:0/3158248743 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1b440180a0 con 0x7f1b480cf380 2023-12-08T08:39:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.425+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1b2c06ae70 msgr2=0x7f1b2c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.425+0000 7f1b4f9c5640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1b2c06ae70 0x7f1b2c06d330 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1b480b5d70 tx=0x7f1b38008040 comp rx=0 tx=0).stop 2023-12-08T08:39:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.425+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3158248743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b480cf380 msgr2=0x7f1b4815c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.425+0000 7f1b4f9c5640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b480cf380 0x7f1b4815c140 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400c4d0 tx=0x7f1b4400c990 comp rx=0 tx=0).stop 2023-12-08T08:39:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.426+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3158248743 shutdown_connections 2023-12-08T08:39:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.426+0000 7f1b4f9c5640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1b2c06ae70 0x7f1b2c06d330 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.426+0000 7f1b4f9c5640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1b4815e150 0x7f1b480b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.426+0000 7f1b4f9c5640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1b480cf760 0x7f1b48157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.426+0000 7f1b4f9c5640 1 --2- 172.21.15.16:0/3158248743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b480cf380 0x7f1b4815c140 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.426+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3158248743 >> 172.21.15.16:0/3158248743 conn(0x7f1b480cfda0 msgr2=0x7f1b4815d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:36.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.426+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3158248743 shutdown_connections 2023-12-08T08:39:36.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.426+0000 7f1b4f9c5640 1 -- 172.21.15.16:0/3158248743 wait complete. 2023-12-08T08:39:36.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-12-08T08:39:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.544+0000 7f62e0aa6640 1 Processor -- start 2023-12-08T08:39:36.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.544+0000 7f62e0aa6640 1 -- start start 2023-12-08T08:39:36.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.545+0000 7f62e0aa6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f62dc140a10 0x7f62dc0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.545+0000 7f62e0aa6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62dc071e50 0x7f62dc0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.545+0000 7f62e0aa6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62dc070db0 con 0x7f62dc0b1f30 2023-12-08T08:39:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.545+0000 7f62e0aa6640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62dc070ba0 con 0x7f62dc140a10 2023-12-08T08:39:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.545+0000 7f62e0aa6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc071090 con 0x7f62dc071e50 2023-12-08T08:39:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.545+0000 7f62dad9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f62dc0b1f30 0x7f62dc0b23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51500/0 (socket says 172.21.15.16:51500) 2023-12-08T08:39:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.545+0000 7f62dad9d640 1 -- 172.21.15.16:0/1474646076 learned_addr learned my addr 172.21.15.16:0/1474646076 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.545+0000 7f62d9d9b640 1 --2- 172.21.15.16:0/1474646076 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62dc071e50 0x7f62dc0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.545+0000 7f62da59c640 1 --2- 172.21.15.16:0/1474646076 >> v2:172.21.15.112:3300/0 conn(0x7f62dc140a10 0x7f62dc0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.545+0000 7f62d959a640 1 -- 172.21.15.16:0/1474646076 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151094733 0 0) 0x7f62dc070db0 con 0x7f62dc0b1f30 2023-12-08T08:39:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.545+0000 7f62d959a640 1 -- 172.21.15.16:0/1474646076 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62c8003520 con 0x7f62dc0b1f30 2023-12-08T08:39:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.546+0000 7f62d9d9b640 1 -- 172.21.15.16:0/1474646076 >> v2:172.21.15.112:3300/0 conn(0x7f62dc140a10 msgr2=0x7f62dc0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.546+0000 7f62d9d9b640 1 --2- 172.21.15.16:0/1474646076 >> v2:172.21.15.112:3300/0 conn(0x7f62dc140a10 0x7f62dc0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.546+0000 7f62d959a640 1 -- 172.21.15.16:0/1474646076 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3357073585 0 0) 0x7f62c8003520 con 0x7f62dc0b1f30 2023-12-08T08:39:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.546+0000 7f62d9d9b640 1 -- 172.21.15.16:0/1474646076 >> v1:172.21.15.16:6789/0 conn(0x7f62dc0b1f30 legacy=0x7f62dc0b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.546+0000 7f62d9d9b640 1 -- 172.21.15.16:0/1474646076 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62dc071370 con 0x7f62dc071e50 2023-12-08T08:39:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.546+0000 7f62d9d9b640 1 --2- 172.21.15.16:0/1474646076 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62dc071e50 0x7f62dc0ce210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f62c40097a0 tx=0x7f62c402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=809968c0acf51899 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.547+0000 7f62d959a640 1 -- 172.21.15.16:0/1474646076 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c402f050 con 0x7f62dc071e50 2023-12-08T08:39:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.547+0000 7f62d959a640 1 -- 172.21.15.16:0/1474646076 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c4031500 con 0x7f62dc071e50 2023-12-08T08:39:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.547+0000 7f62d959a640 1 -- 172.21.15.16:0/1474646076 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d4001670 con 0x7f62dc071e50 2023-12-08T08:39:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.547+0000 7f62e0aa6640 1 -- 172.21.15.16:0/1474646076 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62dc071e50 msgr2=0x7f62dc0ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.547+0000 7f62e0aa6640 1 --2- 172.21.15.16:0/1474646076 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62dc071e50 0x7f62dc0ce210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f62c40097a0 tx=0x7f62c402b820 comp rx=0 tx=0).stop 2023-12-08T08:39:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.547+0000 7f62e0aa6640 1 -- 172.21.15.16:0/1474646076 shutdown_connections 2023-12-08T08:39:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.547+0000 7f62e0aa6640 1 --2- 172.21.15.16:0/1474646076 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62dc071e50 0x7f62dc0ce210 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.547+0000 7f62e0aa6640 1 --2- 172.21.15.16:0/1474646076 >> v2:172.21.15.112:3300/0 conn(0x7f62dc140a10 0x7f62dc0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.547+0000 7f62e0aa6640 1 -- 172.21.15.16:0/1474646076 >> 172.21.15.16:0/1474646076 conn(0x7f62dc0b09f0 msgr2=0x7f62dc0b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.547+0000 7f62e0aa6640 1 -- 172.21.15.16:0/1474646076 shutdown_connections 2023-12-08T08:39:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.547+0000 7f62e0aa6640 1 -- 172.21.15.16:0/1474646076 wait complete. 2023-12-08T08:39:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.549+0000 7f62e0aa6640 1 Processor -- start 2023-12-08T08:39:36.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.549+0000 7f62e0aa6640 1 -- start start 2023-12-08T08:39:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.549+0000 7f62e0aa6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc071e50 0x7f62dc159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.549+0000 7f62e0aa6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62dc0b1f30 0x7f62dc06ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.549+0000 7f62e0aa6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62dc140a10 0x7f62dc06cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.549+0000 7f62e0aa6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc0c9f60 con 0x7f62dc071e50 2023-12-08T08:39:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.549+0000 7f62e0aa6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc0c8ec0 con 0x7f62dc0b1f30 2023-12-08T08:39:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.549+0000 7f62e0aa6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc0c91a0 con 0x7f62dc140a10 2023-12-08T08:39:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.549+0000 7f62da59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc071e50 0x7f62dc159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.549+0000 7f62da59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc071e50 0x7f62dc159b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34488/0 (socket says 172.21.15.16:34488) 2023-12-08T08:39:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.549+0000 7f62da59c640 1 -- 172.21.15.16:0/2389731595 learned_addr learned my addr 172.21.15.16:0/2389731595 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.550+0000 7f62dad9d640 1 --2- 172.21.15.16:0/2389731595 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62dc140a10 0x7f62dc06cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.550+0000 7f62da59c640 1 -- 172.21.15.16:0/2389731595 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62dc140a10 msgr2=0x7f62dc06cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.550+0000 7f62d9d9b640 1 --2- 172.21.15.16:0/2389731595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62dc0b1f30 0x7f62dc06ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.550+0000 7f62da59c640 1 --2- 172.21.15.16:0/2389731595 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62dc140a10 0x7f62dc06cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.550+0000 7f62da59c640 1 -- 172.21.15.16:0/2389731595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62dc0b1f30 msgr2=0x7f62dc06ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.550+0000 7f62da59c640 1 --2- 172.21.15.16:0/2389731595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62dc0b1f30 0x7f62dc06ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.550+0000 7f62da59c640 1 -- 172.21.15.16:0/2389731595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c4009400 con 0x7f62dc071e50 2023-12-08T08:39:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.550+0000 7f62da59c640 1 --2- 172.21.15.16:0/2389731595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc071e50 0x7f62dc159b40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00efc0 tx=0x7f62cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.550+0000 7f62d2ffd640 1 -- 172.21.15.16:0/2389731595 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc009b80 con 0x7f62dc071e50 2023-12-08T08:39:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.550+0000 7f62d2ffd640 1 -- 172.21.15.16:0/2389731595 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62cc00f040 con 0x7f62dc071e50 2023-12-08T08:39:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.550+0000 7f62d2ffd640 1 -- 172.21.15.16:0/2389731595 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62cc0146a0 con 0x7f62dc071e50 2023-12-08T08:39:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.550+0000 7f62e0aa6640 1 -- 172.21.15.16:0/2389731595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc0c9480 con 0x7f62dc071e50 2023-12-08T08:39:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.550+0000 7f62e0aa6640 1 -- 172.21.15.16:0/2389731595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc0c9a40 con 0x7f62dc071e50 2023-12-08T08:39:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.551+0000 7f62e0aa6640 1 -- 172.21.15.16:0/2389731595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62dc067430 con 0x7f62dc071e50 2023-12-08T08:39:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.551+0000 7f62d2ffd640 1 -- 172.21.15.16:0/2389731595 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f62cc014800 con 0x7f62dc071e50 2023-12-08T08:39:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.552+0000 7f62d2ffd640 1 --2- 172.21.15.16:0/2389731595 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f62bc06ae40 0x7f62bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.552+0000 7f62d2ffd640 1 -- 172.21.15.16:0/2389731595 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f62cc089f30 con 0x7f62dc071e50 2023-12-08T08:39:36.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.552+0000 7f62d9d9b640 1 --2- 172.21.15.16:0/2389731595 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f62bc06ae40 0x7f62bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.553+0000 7f62d9d9b640 1 --2- 172.21.15.16:0/2389731595 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f62bc06ae40 0x7f62bc06d300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f62c4035000 tx=0x7f62c4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:36.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.556+0000 7f62d2ffd640 1 -- 172.21.15.16:0/2389731595 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f62cc0551f0 con 0x7f62dc071e50 2023-12-08T08:39:36.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.805+0000 7f62e0aa6640 1 -- 172.21.15.16:0/2389731595 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f62dc0732a0 con 0x7f62dc071e50 2023-12-08T08:39:36.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.807+0000 7f62d2ffd640 1 -- 172.21.15.16:0/2389731595 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f62cc058810 con 0x7f62dc071e50 2023-12-08T08:39:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.808+0000 7f62e0aa6640 1 -- 172.21.15.16:0/2389731595 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f62bc06ae40 msgr2=0x7f62bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.808+0000 7f62e0aa6640 1 --2- 172.21.15.16:0/2389731595 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f62bc06ae40 0x7f62bc06d300 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f62c4035000 tx=0x7f62c4008030 comp rx=0 tx=0).stop 2023-12-08T08:39:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.808+0000 7f62e0aa6640 1 -- 172.21.15.16:0/2389731595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc071e50 msgr2=0x7f62dc159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.808+0000 7f62e0aa6640 1 --2- 172.21.15.16:0/2389731595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc071e50 0x7f62dc159b40 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00efc0 tx=0x7f62cc00c490 comp rx=0 tx=0).stop 2023-12-08T08:39:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.808+0000 7f62e0aa6640 1 -- 172.21.15.16:0/2389731595 shutdown_connections 2023-12-08T08:39:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.808+0000 7f62e0aa6640 1 --2- 172.21.15.16:0/2389731595 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f62bc06ae40 0x7f62bc06d300 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.808+0000 7f62e0aa6640 1 --2- 172.21.15.16:0/2389731595 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f62dc140a10 0x7f62dc06cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.808+0000 7f62e0aa6640 1 --2- 172.21.15.16:0/2389731595 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f62dc0b1f30 0x7f62dc06ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.808+0000 7f62e0aa6640 1 --2- 172.21.15.16:0/2389731595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62dc071e50 0x7f62dc159b40 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.808+0000 7f62e0aa6640 1 -- 172.21.15.16:0/2389731595 >> 172.21.15.16:0/2389731595 conn(0x7f62dc0b09f0 msgr2=0x7f62dc0cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.809+0000 7f62e0aa6640 1 -- 172.21.15.16:0/2389731595 shutdown_connections 2023-12-08T08:39:36.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.809+0000 7f62e0aa6640 1 -- 172.21.15.16:0/2389731595 wait complete. 2023-12-08T08:39:36.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-12-08T08:39:36.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.928+0000 7fa732fe3640 1 Processor -- start 2023-12-08T08:39:36.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.928+0000 7fa732fe3640 1 -- start start 2023-12-08T08:39:36.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.928+0000 7fa732fe3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa72c0d1d60 0x7fa72c0d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.928+0000 7fa732fe3640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa72c15f890 0x7fa72c15d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.928+0000 7fa732fe3640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c15ee90 con 0x7fa72c15d580 2023-12-08T08:39:36.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.928+0000 7fa732fe3640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72c0d0fe0 con 0x7fa72c15f890 2023-12-08T08:39:36.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.928+0000 7fa732fe3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0d1280 con 0x7fa72c0d1d60 2023-12-08T08:39:36.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.929+0000 7fa731580640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa72c15f890 0x7fa72c15d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.929+0000 7fa731580640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa72c15f890 0x7fa72c15d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34504/0 (socket says 172.21.15.16:34504) 2023-12-08T08:39:36.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.929+0000 7fa731580640 1 -- 172.21.15.16:0/2145114429 learned_addr learned my addr 172.21.15.16:0/2145114429 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:36.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.929+0000 7fa72bfff640 1 --2- 172.21.15.16:0/2145114429 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa72c0d1d60 0x7fa72c0d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.929+0000 7fa731580640 1 -- 172.21.15.16:0/2145114429 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa72c0d1d60 msgr2=0x7fa72c0d3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:36.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.929+0000 7fa731580640 1 --2- 172.21.15.16:0/2145114429 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa72c0d1d60 0x7fa72c0d3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.929+0000 7fa731580640 1 -- 172.21.15.16:0/2145114429 >> v1:172.21.15.112:6789/0 conn(0x7fa72c15d580 legacy=0x7fa72c15f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:36.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.929+0000 7fa731580640 1 -- 172.21.15.16:0/2145114429 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c0d1560 con 0x7fa72c15f890 2023-12-08T08:39:36.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.930+0000 7fa731580640 1 --2- 172.21.15.16:0/2145114429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c15f890 0x7fa72c15d960 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa724009e50 tx=0x7fa72402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc39fe1495caff56 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:36.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.930+0000 7fa72b7fe640 1 -- 172.21.15.16:0/2145114429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72402c9e0 con 0x7fa72c15f890 2023-12-08T08:39:36.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.930+0000 7fa72b7fe640 1 -- 172.21.15.16:0/2145114429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72402cb40 con 0x7fa72c15f890 2023-12-08T08:39:36.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.930+0000 7fa72b7fe640 1 -- 172.21.15.16:0/2145114429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7240343f0 con 0x7fa72c15f890 2023-12-08T08:39:36.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.930+0000 7fa732fe3640 1 -- 172.21.15.16:0/2145114429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c15f890 msgr2=0x7fa72c15d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.930+0000 7fa732fe3640 1 --2- 172.21.15.16:0/2145114429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c15f890 0x7fa72c15d960 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa724009e50 tx=0x7fa72402be40 comp rx=0 tx=0).stop 2023-12-08T08:39:36.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.930+0000 7fa732fe3640 1 -- 172.21.15.16:0/2145114429 shutdown_connections 2023-12-08T08:39:36.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.930+0000 7fa732fe3640 1 --2- 172.21.15.16:0/2145114429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c15f890 0x7fa72c15d960 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.930+0000 7fa732fe3640 1 --2- 172.21.15.16:0/2145114429 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa72c0d1d60 0x7fa72c0d3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.930+0000 7fa732fe3640 1 -- 172.21.15.16:0/2145114429 >> 172.21.15.16:0/2145114429 conn(0x7fa72c0d2fb0 msgr2=0x7fa72c0d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:36.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.930+0000 7fa732fe3640 1 -- 172.21.15.16:0/2145114429 shutdown_connections 2023-12-08T08:39:36.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.930+0000 7fa732fe3640 1 -- 172.21.15.16:0/2145114429 wait complete. 2023-12-08T08:39:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.932+0000 7fa732fe3640 1 Processor -- start 2023-12-08T08:39:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.932+0000 7fa732fe3640 1 -- start start 2023-12-08T08:39:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa732fe3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c0d1d60 0x7fa72c15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa732fe3640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa72c15d580 0x7fa72c0cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa732fe3640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa72c15f890 0x7fa72c0b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa732fe3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0c7bd0 con 0x7fa72c0d1d60 2023-12-08T08:39:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa732fe3640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c15b750 con 0x7fa72c15d580 2023-12-08T08:39:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa732fe3640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0c6fc0 con 0x7fa72c15f890 2023-12-08T08:39:36.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa730d7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c0d1d60 0x7fa72c15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa730d7f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c0d1d60 0x7fa72c15b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34510/0 (socket says 172.21.15.16:34510) 2023-12-08T08:39:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa730d7f640 1 -- 172.21.15.16:0/4156840936 learned_addr learned my addr 172.21.15.16:0/4156840936 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa731580640 1 --2- 172.21.15.16:0/4156840936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa72c15f890 0x7fa72c0b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa730d7f640 1 -- 172.21.15.16:0/4156840936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa72c15f890 msgr2=0x7fa72c0b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa730d7f640 1 --2- 172.21.15.16:0/4156840936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa72c15f890 0x7fa72c0b5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa730d7f640 1 -- 172.21.15.16:0/4156840936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa72c15d580 msgr2=0x7fa72c0cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa72bfff640 1 --2- 172.21.15.16:0/4156840936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa72c15d580 0x7fa72c0cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa730d7f640 1 --2- 172.21.15.16:0/4156840936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa72c15d580 0x7fa72c0cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa730d7f640 1 -- 172.21.15.16:0/4156840936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa724009b30 con 0x7fa72c0d1d60 2023-12-08T08:39:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa730d7f640 1 --2- 172.21.15.16:0/4156840936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c0d1d60 0x7fa72c15b210 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa72000c430 tx=0x7fa72000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa7297fa640 1 -- 172.21.15.16:0/4156840936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720011790 con 0x7fa72c0d1d60 2023-12-08T08:39:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa7297fa640 1 -- 172.21.15.16:0/4156840936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7200118f0 con 0x7fa72c0d1d60 2023-12-08T08:39:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa7297fa640 1 -- 172.21.15.16:0/4156840936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720011ce0 con 0x7fa72c0d1d60 2023-12-08T08:39:36.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa732fe3640 1 -- 172.21.15.16:0/4156840936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c0c72a0 con 0x7fa72c0d1d60 2023-12-08T08:39:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.933+0000 7fa732fe3640 1 -- 172.21.15.16:0/4156840936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c0c7860 con 0x7fa72c0d1d60 2023-12-08T08:39:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.934+0000 7fa732fe3640 1 -- 172.21.15.16:0/4156840936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72c0d09d0 con 0x7fa72c0d1d60 2023-12-08T08:39:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.935+0000 7fa7297fa640 1 -- 172.21.15.16:0/4156840936 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa72001c460 con 0x7fa72c0d1d60 2023-12-08T08:39:36.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.935+0000 7fa7297fa640 1 --2- 172.21.15.16:0/4156840936 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa71006ae70 0x7fa71006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:36.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.935+0000 7fa7297fa640 1 -- 172.21.15.16:0/4156840936 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fa7200893d0 con 0x7fa72c0d1d60 2023-12-08T08:39:36.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.935+0000 7fa72bfff640 1 --2- 172.21.15.16:0/4156840936 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa71006ae70 0x7fa71006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:36.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.936+0000 7fa72bfff640 1 --2- 172.21.15.16:0/4156840936 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa71006ae70 0x7fa71006d330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa718006fd0 tx=0x7fa718008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:36.939+0000 7fa7297fa640 1 -- 172.21.15.16:0/4156840936 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa7200545f0 con 0x7fa72c0d1d60 2023-12-08T08:39:37.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.184+0000 7fa732fe3640 1 -- 172.21.15.16:0/4156840936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fa72c067430 con 0x7fa72c0d1d60 2023-12-08T08:39:37.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.186+0000 7fa7297fa640 1 -- 172.21.15.16:0/4156840936 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa720057c10 con 0x7fa72c0d1d60 2023-12-08T08:39:37.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.188+0000 7fa732fe3640 1 -- 172.21.15.16:0/4156840936 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa71006ae70 msgr2=0x7fa71006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.188+0000 7fa732fe3640 1 --2- 172.21.15.16:0/4156840936 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa71006ae70 0x7fa71006d330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa718006fd0 tx=0x7fa718008040 comp rx=0 tx=0).stop 2023-12-08T08:39:37.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.188+0000 7fa732fe3640 1 -- 172.21.15.16:0/4156840936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c0d1d60 msgr2=0x7fa72c15b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.188+0000 7fa732fe3640 1 --2- 172.21.15.16:0/4156840936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c0d1d60 0x7fa72c15b210 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa72000c430 tx=0x7fa72000c8f0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.188+0000 7fa732fe3640 1 -- 172.21.15.16:0/4156840936 shutdown_connections 2023-12-08T08:39:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.188+0000 7fa732fe3640 1 --2- 172.21.15.16:0/4156840936 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fa71006ae70 0x7fa71006d330 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.188+0000 7fa732fe3640 1 --2- 172.21.15.16:0/4156840936 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fa72c15f890 0x7fa72c0b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.188+0000 7fa732fe3640 1 --2- 172.21.15.16:0/4156840936 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fa72c15d580 0x7fa72c0cac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.188+0000 7fa732fe3640 1 --2- 172.21.15.16:0/4156840936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c0d1d60 0x7fa72c15b210 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.188+0000 7fa732fe3640 1 -- 172.21.15.16:0/4156840936 >> 172.21.15.16:0/4156840936 conn(0x7fa72c0d2fb0 msgr2=0x7fa72c06b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.188+0000 7fa732fe3640 1 -- 172.21.15.16:0/4156840936 shutdown_connections 2023-12-08T08:39:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.188+0000 7fa732fe3640 1 -- 172.21.15.16:0/4156840936 wait complete. 2023-12-08T08:39:37.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-12-08T08:39:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.306+0000 7f7198d01640 1 Processor -- start 2023-12-08T08:39:37.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.306+0000 7f7198d01640 1 -- start start 2023-12-08T08:39:37.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.307+0000 7f7198d01640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f71940ad290 0x7f719406ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:37.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.307+0000 7f7198d01640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719406aee0 con 0x7f71940ad710 2023-12-08T08:39:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.307+0000 7f7198d01640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719406bd40 con 0x7f7194160ca0 2023-12-08T08:39:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.307+0000 7f7198d01640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f719406b130 con 0x7f71940ad290 2023-12-08T08:39:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.307+0000 7f7192d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f71940ad710 0x7f719406d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51502/0 (socket says 172.21.15.16:51502) 2023-12-08T08:39:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.307+0000 7f7192d9d640 1 -- 172.21.15.16:0/3890840221 learned_addr learned my addr 172.21.15.16:0/3890840221 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.307+0000 7f7191d9b640 1 --2- 172.21.15.16:0/3890840221 >> v2:172.21.15.112:3300/0 conn(0x7f71940ad290 0x7f719406ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.307+0000 7f719159a640 1 -- 172.21.15.16:0/3890840221 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571956483 0 0) 0x7f719406aee0 con 0x7f71940ad710 2023-12-08T08:39:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.307+0000 7f719159a640 1 -- 172.21.15.16:0/3890840221 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7180003520 con 0x7f71940ad710 2023-12-08T08:39:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f719159a640 1 -- 172.21.15.16:0/3890840221 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1542747347 0 0) 0x7f7180003520 con 0x7f71940ad710 2023-12-08T08:39:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f719159a640 1 -- 172.21.15.16:0/3890840221 >> v2:172.21.15.112:3300/0 conn(0x7f71940ad290 msgr2=0x7f719406ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f719159a640 1 --2- 172.21.15.16:0/3890840221 >> v2:172.21.15.112:3300/0 conn(0x7f71940ad290 0x7f719406ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f719159a640 1 -- 172.21.15.16:0/3890840221 >> v1:172.21.15.112:6789/0 conn(0x7f7194160ca0 legacy=0x7f7194161080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f719159a640 1 -- 172.21.15.16:0/3890840221 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719406b410 con 0x7f71940ad710 2023-12-08T08:39:37.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f719159a640 1 -- 172.21.15.16:0/3890840221 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7f71840019f0 con 0x7f71940ad710 2023-12-08T08:39:37.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f719159a640 1 -- 172.21.15.16:0/3890840221 >> v1:172.21.15.16:6789/0 conn(0x7f71940ad710 legacy=0x7f719406d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f719159a640 1 --2- 172.21.15.16:0/3890840221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7180003970 0x7f7180003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:37.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f719159a640 1 --2- 172.21.15.16:0/3890840221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7180004370 0x7f7180004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:37.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f719159a640 1 --2- 172.21.15.16:0/3890840221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7180005040 0x7f718000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:37.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f719159a640 1 -- 172.21.15.16:0/3890840221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719406b130 con 0x7f7180005040 2023-12-08T08:39:37.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f719159a640 1 -- 172.21.15.16:0/3890840221 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718000bfa0 con 0x7f7180004370 2023-12-08T08:39:37.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f719159a640 1 -- 172.21.15.16:0/3890840221 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718000c160 con 0x7f7180003970 2023-12-08T08:39:37.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f7192d9d640 1 --2- 172.21.15.16:0/3890840221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7180005040 0x7f718000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:37.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f719259c640 1 --2- 172.21.15.16:0/3890840221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7180003970 0x7f7180003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:37.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f7191d9b640 1 --2- 172.21.15.16:0/3890840221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7180004370 0x7f7180004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:37.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f7192d9d640 1 -- 172.21.15.16:0/3890840221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7180003970 msgr2=0x7f7180003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:37.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f7192d9d640 1 --2- 172.21.15.16:0/3890840221 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f7180003970 0x7f7180003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f7192d9d640 1 -- 172.21.15.16:0/3890840221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7180004370 msgr2=0x7f7180004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:37.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f7192d9d640 1 --2- 172.21.15.16:0/3890840221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7180004370 0x7f7180004840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f7192d9d640 1 -- 172.21.15.16:0/3890840221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7180003520 con 0x7f7180005040 2023-12-08T08:39:37.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.308+0000 7f7191d9b640 1 --2- 172.21.15.16:0/3890840221 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f7180004370 0x7f7180004840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:37.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.309+0000 7f7192d9d640 1 -- 172.21.15.16:0/3890840221 reap_dead start 2023-12-08T08:39:37.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.309+0000 7f7192d9d640 1 --2- 172.21.15.16:0/3890840221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7180005040 0x7f718000b980 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f718400a830 tx=0x7f71800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184f25b2072b560a server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.309+0000 7f719159a640 1 -- 172.21.15.16:0/3890840221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718402cea0 con 0x7f7180005040 2023-12-08T08:39:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.309+0000 7f719159a640 1 -- 172.21.15.16:0/3890840221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718402f070 con 0x7f7180005040 2023-12-08T08:39:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.309+0000 7f719159a640 1 -- 172.21.15.16:0/3890840221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7184034890 con 0x7f7180005040 2023-12-08T08:39:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.310+0000 7f7198d01640 1 -- 172.21.15.16:0/3890840221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7180005040 msgr2=0x7f718000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.310+0000 7f7198d01640 1 --2- 172.21.15.16:0/3890840221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7180005040 0x7f718000b980 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f718400a830 tx=0x7f71800057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.310+0000 7f7198d01640 1 -- 172.21.15.16:0/3890840221 shutdown_connections 2023-12-08T08:39:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.310+0000 7f7198d01640 1 --2- 172.21.15.16:0/3890840221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7180005040 0x7f718000b980 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.310+0000 7f7198d01640 1 -- 172.21.15.16:0/3890840221 >> 172.21.15.16:0/3890840221 conn(0x7f71940cf470 msgr2=0x7f71940cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.310+0000 7f7198d01640 1 -- 172.21.15.16:0/3890840221 shutdown_connections 2023-12-08T08:39:37.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.310+0000 7f7198d01640 1 -- 172.21.15.16:0/3890840221 wait complete. 2023-12-08T08:39:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.311+0000 7f7198d01640 1 Processor -- start 2023-12-08T08:39:37.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.311+0000 7f7198d01640 1 -- start start 2023-12-08T08:39:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f7198d01640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7180005040 0x7f719413b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f7198d01640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71941f6180 0x7f71941f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f7198d01640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71940b2fe0 0x7f71940b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f7198d01640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71941f6b40 con 0x7f7180005040 2023-12-08T08:39:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f7198d01640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719414e830 con 0x7f71941f6180 2023-12-08T08:39:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f7198d01640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71940b3d60 con 0x7f71940b2fe0 2023-12-08T08:39:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f719259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7180005040 0x7f719413b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f719259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7180005040 0x7f719413b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34520/0 (socket says 172.21.15.16:34520) 2023-12-08T08:39:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f719259c640 1 -- 172.21.15.16:0/3813239470 learned_addr learned my addr 172.21.15.16:0/3813239470 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f7192d9d640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71940b2fe0 0x7f71940b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f7191d9b640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71941f6180 0x7f71941f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:37.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f719259c640 1 -- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71940b2fe0 msgr2=0x7f71940b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f719259c640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71940b2fe0 0x7f71940b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f719259c640 1 -- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71941f6180 msgr2=0x7f71941f6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f719259c640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71941f6180 0x7f71941f6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f719259c640 1 -- 172.21.15.16:0/3813239470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719414e1e0 con 0x7f7180005040 2023-12-08T08:39:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f7192d9d640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71940b2fe0 0x7f71940b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.312+0000 7f7191d9b640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71941f6180 0x7f71941f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.313+0000 7f719259c640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7180005040 0x7f719413b270 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f718c00f7f0 tx=0x7f718c00fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.313+0000 7f718affd640 1 -- 172.21.15.16:0/3813239470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c00fea0 con 0x7f7180005040 2023-12-08T08:39:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.313+0000 7f718affd640 1 -- 172.21.15.16:0/3813239470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718c0052e0 con 0x7f7180005040 2023-12-08T08:39:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.313+0000 7f718affd640 1 -- 172.21.15.16:0/3813239470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718c00d910 con 0x7f7180005040 2023-12-08T08:39:37.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.313+0000 7f7198d01640 1 -- 172.21.15.16:0/3813239470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719414e4c0 con 0x7f7180005040 2023-12-08T08:39:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.313+0000 7f7198d01640 1 -- 172.21.15.16:0/3813239470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719414de40 con 0x7f7180005040 2023-12-08T08:39:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.314+0000 7f7198d01640 1 -- 172.21.15.16:0/3813239470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71940acc40 con 0x7f7180005040 2023-12-08T08:39:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.314+0000 7f718affd640 1 -- 172.21.15.16:0/3813239470 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f718c00da70 con 0x7f7180005040 2023-12-08T08:39:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.315+0000 7f718affd640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f717406ae40 0x7f717406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.315+0000 7f718affd640 1 -- 172.21.15.16:0/3813239470 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f718c089480 con 0x7f7180005040 2023-12-08T08:39:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.315+0000 7f7191d9b640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f717406ae40 0x7f717406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.315+0000 7f7191d9b640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f717406ae40 0x7f717406d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f719414fdd0 tx=0x7f717c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:37.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.319+0000 7f718affd640 1 -- 172.21.15.16:0/3813239470 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f718c0546a0 con 0x7f7180005040 2023-12-08T08:39:37.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.564+0000 7f7198d01640 1 -- 172.21.15.16:0/3813239470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f7194067430 con 0x7f7180005040 2023-12-08T08:39:37.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.566+0000 7f718affd640 1 -- 172.21.15.16:0/3813239470 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f718c057cc0 con 0x7f7180005040 2023-12-08T08:39:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.568+0000 7f7198d01640 1 -- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f717406ae40 msgr2=0x7f717406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.568+0000 7f7198d01640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f717406ae40 0x7f717406d300 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f719414fdd0 tx=0x7f717c008040 comp rx=0 tx=0).stop 2023-12-08T08:39:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.568+0000 7f7198d01640 1 -- 172.21.15.16:0/3813239470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7180005040 msgr2=0x7f719413b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.568+0000 7f7198d01640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7180005040 0x7f719413b270 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f718c00f7f0 tx=0x7f718c00fcb0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.568+0000 7f7198d01640 1 -- 172.21.15.16:0/3813239470 shutdown_connections 2023-12-08T08:39:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.568+0000 7f7198d01640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f717406ae40 0x7f717406d300 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.568+0000 7f7198d01640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f71940b2fe0 0x7f71940b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.568+0000 7f7198d01640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f71941f6180 0x7f71941f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.568+0000 7f7198d01640 1 --2- 172.21.15.16:0/3813239470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7180005040 0x7f719413b270 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.568+0000 7f7198d01640 1 -- 172.21.15.16:0/3813239470 >> 172.21.15.16:0/3813239470 conn(0x7f71940cf470 msgr2=0x7f7194161080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.568+0000 7f7198d01640 1 -- 172.21.15.16:0/3813239470 shutdown_connections 2023-12-08T08:39:37.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.568+0000 7f7198d01640 1 -- 172.21.15.16:0/3813239470 wait complete. 2023-12-08T08:39:37.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-12-08T08:39:37.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.686+0000 7f04432e0640 1 Processor -- start 2023-12-08T08:39:37.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.687+0000 7f04432e0640 1 -- start start 2023-12-08T08:39:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.687+0000 7f04432e0640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f043c1608a0 0x7f043c0d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.687+0000 7f04432e0640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f043c0b1d70 0x7f043c0d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.687+0000 7f04432e0640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043c0d20b0 con 0x7f043c0afce0 2023-12-08T08:39:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.687+0000 7f04432e0640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f043c0d0620 con 0x7f043c1608a0 2023-12-08T08:39:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.687+0000 7f04432e0640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f043c0d1dd0 con 0x7f043c0b1d70 2023-12-08T08:39:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.687+0000 7f044107c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f043c1608a0 0x7f043c0d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.687+0000 7f044107c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f043c1608a0 0x7f043c0d0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34536/0 (socket says 172.21.15.16:34536) 2023-12-08T08:39:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.687+0000 7f044107c640 1 -- 172.21.15.16:0/476034224 learned_addr learned my addr 172.21.15.16:0/476034224 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.688+0000 7f044087b640 1 --2- 172.21.15.16:0/476034224 >> v2:172.21.15.112:3300/0 conn(0x7f043c0b1d70 0x7f043c0d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.688+0000 7f044107c640 1 -- 172.21.15.16:0/476034224 >> v2:172.21.15.112:3300/0 conn(0x7f043c0b1d70 msgr2=0x7f043c0d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.688+0000 7f044107c640 1 --2- 172.21.15.16:0/476034224 >> v2:172.21.15.112:3300/0 conn(0x7f043c0b1d70 0x7f043c0d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.688+0000 7f044107c640 1 -- 172.21.15.16:0/476034224 >> v1:172.21.15.112:6789/0 conn(0x7f043c0afce0 legacy=0x7f043c0b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.688+0000 7f044107c640 1 -- 172.21.15.16:0/476034224 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c0cff60 con 0x7f043c1608a0 2023-12-08T08:39:37.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.688+0000 7f044087b640 1 --2- 172.21.15.16:0/476034224 >> v2:172.21.15.112:3300/0 conn(0x7f043c0b1d70 0x7f043c0d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.688+0000 7f044107c640 1 --2- 172.21.15.16:0/476034224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c1608a0 0x7f043c0d0f30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f043c0d4130 tx=0x7f042c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f30a4cfe8a235e server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.689+0000 7f043bfff640 1 -- 172.21.15.16:0/476034224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c02ca80 con 0x7f043c1608a0 2023-12-08T08:39:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.689+0000 7f043bfff640 1 -- 172.21.15.16:0/476034224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042c02cbe0 con 0x7f043c1608a0 2023-12-08T08:39:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.689+0000 7f043bfff640 1 -- 172.21.15.16:0/476034224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f042c0343f0 con 0x7f043c1608a0 2023-12-08T08:39:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.689+0000 7f04432e0640 1 -- 172.21.15.16:0/476034224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c1608a0 msgr2=0x7f043c0d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.689+0000 7f04432e0640 1 --2- 172.21.15.16:0/476034224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c1608a0 0x7f043c0d0f30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f043c0d4130 tx=0x7f042c02bee0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.689+0000 7f04432e0640 1 -- 172.21.15.16:0/476034224 shutdown_connections 2023-12-08T08:39:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.689+0000 7f04432e0640 1 --2- 172.21.15.16:0/476034224 >> v2:172.21.15.112:3300/0 conn(0x7f043c0b1d70 0x7f043c0d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.689+0000 7f04432e0640 1 --2- 172.21.15.16:0/476034224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c1608a0 0x7f043c0d0f30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.689+0000 7f04432e0640 1 -- 172.21.15.16:0/476034224 >> 172.21.15.16:0/476034224 conn(0x7f043c0d3030 msgr2=0x7f043c0d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.689+0000 7f04432e0640 1 -- 172.21.15.16:0/476034224 shutdown_connections 2023-12-08T08:39:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.689+0000 7f04432e0640 1 -- 172.21.15.16:0/476034224 wait complete. 2023-12-08T08:39:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.690+0000 7f04432e0640 1 Processor -- start 2023-12-08T08:39:37.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.690+0000 7f04432e0640 1 -- start start 2023-12-08T08:39:37.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.691+0000 7f04432e0640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f043c0afce0 0x7f043c0b7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:37.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.691+0000 7f04432e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c0b1d70 0x7f043c15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:37.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.691+0000 7f04432e0640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f043c1608a0 0x7f043c0c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:37.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.691+0000 7f04432e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c1f2c70 con 0x7f043c0b1d70 2023-12-08T08:39:37.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.691+0000 7f04432e0640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c15aaf0 con 0x7f043c1608a0 2023-12-08T08:39:37.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.691+0000 7f04432e0640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043c0c7050 con 0x7f043c0afce0 2023-12-08T08:39:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.691+0000 7f044087b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c0b1d70 0x7f043c15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.691+0000 7f044087b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c0b1d70 0x7f043c15a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34544/0 (socket says 172.21.15.16:34544) 2023-12-08T08:39:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.691+0000 7f044087b640 1 -- 172.21.15.16:0/909150111 learned_addr learned my addr 172.21.15.16:0/909150111 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.691+0000 7f044107c640 1 --2- 172.21.15.16:0/909150111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f043c0afce0 0x7f043c0b7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.691+0000 7f044187d640 1 --2- 172.21.15.16:0/909150111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f043c1608a0 0x7f043c0c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.692+0000 7f044087b640 1 -- 172.21.15.16:0/909150111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f043c0afce0 msgr2=0x7f043c0b7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.692+0000 7f044087b640 1 --2- 172.21.15.16:0/909150111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f043c0afce0 0x7f043c0b7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.692+0000 7f044087b640 1 -- 172.21.15.16:0/909150111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f043c1608a0 msgr2=0x7f043c0c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.692+0000 7f044087b640 1 --2- 172.21.15.16:0/909150111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f043c1608a0 0x7f043c0c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.692+0000 7f044087b640 1 -- 172.21.15.16:0/909150111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c009b30 con 0x7f043c0b1d70 2023-12-08T08:39:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.692+0000 7f044087b640 1 --2- 172.21.15.16:0/909150111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c0b1d70 0x7f043c15a5b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f043000d7d0 tx=0x7f043000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.692+0000 7f0439ffb640 1 -- 172.21.15.16:0/909150111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430004030 con 0x7f043c0b1d70 2023-12-08T08:39:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.692+0000 7f0439ffb640 1 -- 172.21.15.16:0/909150111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0430004190 con 0x7f043c0b1d70 2023-12-08T08:39:37.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.692+0000 7f0439ffb640 1 -- 172.21.15.16:0/909150111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430004520 con 0x7f043c0b1d70 2023-12-08T08:39:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.692+0000 7f04432e0640 1 -- 172.21.15.16:0/909150111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043c0c7300 con 0x7f043c0b1d70 2023-12-08T08:39:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.692+0000 7f04432e0640 1 -- 172.21.15.16:0/909150111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043c0c78c0 con 0x7f043c0b1d70 2023-12-08T08:39:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.693+0000 7f04432e0640 1 -- 172.21.15.16:0/909150111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043c0c7ae0 con 0x7f043c0b1d70 2023-12-08T08:39:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.693+0000 7f0439ffb640 1 -- 172.21.15.16:0/909150111 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0430009640 con 0x7f043c0b1d70 2023-12-08T08:39:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.694+0000 7f0439ffb640 1 --2- 172.21.15.16:0/909150111 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f042806ae40 0x7f042806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:37.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.694+0000 7f0439ffb640 1 -- 172.21.15.16:0/909150111 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f04300893a0 con 0x7f043c0b1d70 2023-12-08T08:39:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.694+0000 7f044107c640 1 --2- 172.21.15.16:0/909150111 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f042806ae40 0x7f042806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:37.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.695+0000 7f044107c640 1 --2- 172.21.15.16:0/909150111 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f042806ae40 0x7f042806d300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f042c035000 tx=0x7f042c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:37.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.698+0000 7f0439ffb640 1 -- 172.21.15.16:0/909150111 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f04300547a0 con 0x7f043c0b1d70 2023-12-08T08:39:37.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.942+0000 7f04432e0640 1 -- 172.21.15.16:0/909150111 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f043c067430 con 0x7f043c0b1d70 2023-12-08T08:39:37.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.943+0000 7f0439ffb640 1 -- 172.21.15.16:0/909150111 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0430057dc0 con 0x7f043c0b1d70 2023-12-08T08:39:37.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.945+0000 7f04432e0640 1 -- 172.21.15.16:0/909150111 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f042806ae40 msgr2=0x7f042806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.945+0000 7f04432e0640 1 --2- 172.21.15.16:0/909150111 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f042806ae40 0x7f042806d300 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f042c035000 tx=0x7f042c030000 comp rx=0 tx=0).stop 2023-12-08T08:39:37.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.945+0000 7f04432e0640 1 -- 172.21.15.16:0/909150111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c0b1d70 msgr2=0x7f043c15a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:37.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.945+0000 7f04432e0640 1 --2- 172.21.15.16:0/909150111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c0b1d70 0x7f043c15a5b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f043000d7d0 tx=0x7f043000dc90 comp rx=0 tx=0).stop 2023-12-08T08:39:37.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.945+0000 7f04432e0640 1 -- 172.21.15.16:0/909150111 shutdown_connections 2023-12-08T08:39:37.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.945+0000 7f04432e0640 1 --2- 172.21.15.16:0/909150111 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f042806ae40 0x7f042806d300 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.945+0000 7f04432e0640 1 --2- 172.21.15.16:0/909150111 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f043c1608a0 0x7f043c0c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.945+0000 7f04432e0640 1 --2- 172.21.15.16:0/909150111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c0b1d70 0x7f043c15a5b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.945+0000 7f04432e0640 1 --2- 172.21.15.16:0/909150111 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f043c0afce0 0x7f043c0b7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:37.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.945+0000 7f04432e0640 1 -- 172.21.15.16:0/909150111 >> 172.21.15.16:0/909150111 conn(0x7f043c0d3030 msgr2=0x7f043c0b00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:37.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.946+0000 7f04432e0640 1 -- 172.21.15.16:0/909150111 shutdown_connections 2023-12-08T08:39:37.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:37.946+0000 7f04432e0640 1 -- 172.21.15.16:0/909150111 wait complete. 2023-12-08T08:39:37.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-12-08T08:39:38.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.064+0000 7f7896197640 1 Processor -- start 2023-12-08T08:39:38.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.064+0000 7f7896197640 1 -- start start 2023-12-08T08:39:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.064+0000 7f7896197640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78900b0cf0 0x7f78900d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.064+0000 7f7896197640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78900d20b0 con 0x7f78900b1d70 2023-12-08T08:39:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.064+0000 7f7896197640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78900d1b80 con 0x7f78901608a0 2023-12-08T08:39:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.064+0000 7f7896197640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78900d0620 con 0x7f78900b0cf0 2023-12-08T08:39:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.064+0000 7f788effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78900b0cf0 0x7f78900d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.065+0000 7f788ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f78900b1d70 0x7f78900cdc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51504/0 (socket says 172.21.15.16:51504) 2023-12-08T08:39:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.065+0000 7f788effd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78900b0cf0 0x7f78900d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34558/0 (socket says 172.21.15.16:34558) 2023-12-08T08:39:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.065+0000 7f788ffff640 1 -- 172.21.15.16:0/2244472662 learned_addr learned my addr 172.21.15.16:0/2244472662 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.065+0000 7f788effd640 1 -- 172.21.15.16:0/2244472662 >> v1:172.21.15.112:6789/0 conn(0x7f78901608a0 legacy=0x7f78900d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.065+0000 7f788effd640 1 -- 172.21.15.16:0/2244472662 >> v1:172.21.15.16:6789/0 conn(0x7f78900b1d70 legacy=0x7f78900cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.065+0000 7f788effd640 1 -- 172.21.15.16:0/2244472662 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78900d1dd0 con 0x7f78900b0cf0 2023-12-08T08:39:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.066+0000 7f788effd640 1 --2- 172.21.15.16:0/2244472662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78900b0cf0 0x7f78900d1640 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f78840097a0 tx=0x7f788402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44bcea848ac6a47 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.066+0000 7f788e7fc640 1 -- 172.21.15.16:0/2244472662 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788402e070 con 0x7f78900b0cf0 2023-12-08T08:39:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.066+0000 7f788e7fc640 1 -- 172.21.15.16:0/2244472662 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78840315a0 con 0x7f78900b0cf0 2023-12-08T08:39:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.066+0000 7f788e7fc640 1 -- 172.21.15.16:0/2244472662 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7884031930 con 0x7f78900b0cf0 2023-12-08T08:39:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.066+0000 7f7896197640 1 -- 172.21.15.16:0/2244472662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78900b0cf0 msgr2=0x7f78900d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.066+0000 7f7896197640 1 --2- 172.21.15.16:0/2244472662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78900b0cf0 0x7f78900d1640 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f78840097a0 tx=0x7f788402b840 comp rx=0 tx=0).stop 2023-12-08T08:39:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.066+0000 7f7896197640 1 -- 172.21.15.16:0/2244472662 shutdown_connections 2023-12-08T08:39:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.066+0000 7f7896197640 1 --2- 172.21.15.16:0/2244472662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78900b0cf0 0x7f78900d1640 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.066+0000 7f7896197640 1 -- 172.21.15.16:0/2244472662 >> 172.21.15.16:0/2244472662 conn(0x7f78900d3030 msgr2=0x7f78900d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.066+0000 7f7896197640 1 -- 172.21.15.16:0/2244472662 shutdown_connections 2023-12-08T08:39:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.066+0000 7f7896197640 1 -- 172.21.15.16:0/2244472662 wait complete. 2023-12-08T08:39:38.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.067+0000 7f7896197640 1 Processor -- start 2023-12-08T08:39:38.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.068+0000 7f7896197640 1 -- start start 2023-12-08T08:39:38.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.068+0000 7f7896197640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78900b0cf0 0x7f78900cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.068+0000 7f7896197640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78900b1d70 0x7f789015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.068+0000 7f7896197640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78901608a0 0x7f78900b79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.068+0000 7f7896197640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78901f3cf0 con 0x7f78900b0cf0 2023-12-08T08:39:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.068+0000 7f7896197640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78900cb000 con 0x7f78901608a0 2023-12-08T08:39:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.068+0000 7f7896197640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78900cb2e0 con 0x7f78900b1d70 2023-12-08T08:39:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.068+0000 7f788f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78900b0cf0 0x7f78900cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.068+0000 7f788f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78900b0cf0 0x7f78900cbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34566/0 (socket says 172.21.15.16:34566) 2023-12-08T08:39:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.068+0000 7f788f7fe640 1 -- 172.21.15.16:0/3472041617 learned_addr learned my addr 172.21.15.16:0/3472041617 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.069+0000 7f788effd640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78900b1d70 0x7f789015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.069+0000 7f788ffff640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78901608a0 0x7f78900b79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.069+0000 7f788f7fe640 1 -- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78900b1d70 msgr2=0x7f789015c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.069+0000 7f788f7fe640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78900b1d70 0x7f789015c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.069+0000 7f788f7fe640 1 -- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78901608a0 msgr2=0x7f78900b79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.069+0000 7f788f7fe640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78901608a0 0x7f78900b79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.069+0000 7f788f7fe640 1 -- 172.21.15.16:0/3472041617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7884009400 con 0x7f78900b0cf0 2023-12-08T08:39:38.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.069+0000 7f788effd640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78900b1d70 0x7f789015c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.069+0000 7f788f7fe640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78900b0cf0 0x7f78900cbdc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f788000b3f0 tx=0x7f788000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.069+0000 7f7877fff640 1 -- 172.21.15.16:0/3472041617 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880011830 con 0x7f78900b0cf0 2023-12-08T08:39:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.069+0000 7f7877fff640 1 -- 172.21.15.16:0/3472041617 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7880011990 con 0x7f78900b0cf0 2023-12-08T08:39:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.069+0000 7f7896197640 1 -- 172.21.15.16:0/3472041617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78900cb5c0 con 0x7f78900b0cf0 2023-12-08T08:39:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.069+0000 7f7877fff640 1 -- 172.21.15.16:0/3472041617 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880011d40 con 0x7f78900b0cf0 2023-12-08T08:39:38.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.069+0000 7f7896197640 1 -- 172.21.15.16:0/3472041617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789015c6e0 con 0x7f78900b0cf0 2023-12-08T08:39:38.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.070+0000 7f7896197640 1 -- 172.21.15.16:0/3472041617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78900cbae0 con 0x7f78900b0cf0 2023-12-08T08:39:38.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.071+0000 7f7877fff640 1 -- 172.21.15.16:0/3472041617 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f788001a400 con 0x7f78900b0cf0 2023-12-08T08:39:38.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.071+0000 7f7877fff640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f787c06ae40 0x7f787c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.071+0000 7f7877fff640 1 -- 172.21.15.16:0/3472041617 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f78800891b0 con 0x7f78900b0cf0 2023-12-08T08:39:38.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.071+0000 7f788effd640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f787c06ae40 0x7f787c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.072+0000 7f788effd640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f787c06ae40 0x7f787c06d300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7884036000 tx=0x7f788403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.075+0000 7f7877fff640 1 -- 172.21.15.16:0/3472041617 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78900cbae0 con 0x7f78900b0cf0 2023-12-08T08:39:38.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.317+0000 7f7896197640 1 -- 172.21.15.16:0/3472041617 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f78900cbae0 con 0x7f78900b0cf0 2023-12-08T08:39:38.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.319+0000 7f7877fff640 1 -- 172.21.15.16:0/3472041617 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7f78900cbae0 con 0x7f78900b0cf0 2023-12-08T08:39:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.321+0000 7f7896197640 1 -- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f787c06ae40 msgr2=0x7f787c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.321+0000 7f7896197640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f787c06ae40 0x7f787c06d300 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7884036000 tx=0x7f788403d040 comp rx=0 tx=0).stop 2023-12-08T08:39:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.321+0000 7f7896197640 1 -- 172.21.15.16:0/3472041617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78900b0cf0 msgr2=0x7f78900cbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.321+0000 7f7896197640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78900b0cf0 0x7f78900cbdc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f788000b3f0 tx=0x7f788000b8b0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.321+0000 7f7896197640 1 -- 172.21.15.16:0/3472041617 shutdown_connections 2023-12-08T08:39:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.321+0000 7f7896197640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f787c06ae40 0x7f787c06d300 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.321+0000 7f7896197640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f78901608a0 0x7f78900b79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.321+0000 7f7896197640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f78900b1d70 0x7f789015c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.321+0000 7f7896197640 1 --2- 172.21.15.16:0/3472041617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78900b0cf0 0x7f78900cbdc0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.321+0000 7f7896197640 1 -- 172.21.15.16:0/3472041617 >> 172.21.15.16:0/3472041617 conn(0x7f78900d3030 msgr2=0x7f78900cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.321+0000 7f7896197640 1 -- 172.21.15.16:0/3472041617 shutdown_connections 2023-12-08T08:39:38.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.321+0000 7f7896197640 1 -- 172.21.15.16:0/3472041617 wait complete. 2023-12-08T08:39:38.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:38.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.439+0000 7fb1d2bbb640 1 Processor -- start 2023-12-08T08:39:38.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.439+0000 7fb1d2bbb640 1 -- start start 2023-12-08T08:39:38.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1d2bbb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1cc0d1900 0x7fb1cc0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1d2bbb640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1cc0cf380 con 0x7fb1cc0cd760 2023-12-08T08:39:38.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1d2bbb640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1cc0cce00 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1d2bbb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc0cea50 con 0x7fb1cc0d1900 2023-12-08T08:39:38.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1d1158640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb1cc0cd760 0x7fb1cc0b3c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51512/0 (socket says 172.21.15.16:51512) 2023-12-08T08:39:38.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1d1158640 1 -- 172.21.15.16:0/3380209582 learned_addr learned my addr 172.21.15.16:0/3380209582 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:38.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1cbfff640 1 --2- 172.21.15.16:0/3380209582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1cc0d1900 0x7fb1cc0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1cb7fe640 1 -- 172.21.15.16:0/3380209582 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509221687 0 0) 0x7fb1cc0cf380 con 0x7fb1cc0cd760 2023-12-08T08:39:38.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1cb7fe640 1 -- 172.21.15.16:0/3380209582 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1bc003520 con 0x7fb1cc0cd760 2023-12-08T08:39:38.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1cb7fe640 1 -- 172.21.15.16:0/3380209582 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 29766913 0 0) 0x7fb1cc0cce00 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1cb7fe640 1 -- 172.21.15.16:0/3380209582 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1cc0cf380 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1cbfff640 1 -- 172.21.15.16:0/3380209582 >> v1:172.21.15.112:6789/0 conn(0x7fb1cc15b0f0 legacy=0x7fb1cc0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1cb7fe640 1 -- 172.21.15.16:0/3380209582 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1911449379 0 0) 0x7fb1bc003520 con 0x7fb1cc0cd760 2023-12-08T08:39:38.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1cbfff640 1 -- 172.21.15.16:0/3380209582 >> v1:172.21.15.16:6789/0 conn(0x7fb1cc0cd760 legacy=0x7fb1cc0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.440+0000 7fb1cbfff640 1 -- 172.21.15.16:0/3380209582 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1cc0ced30 con 0x7fb1cc0d1900 2023-12-08T08:39:38.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.441+0000 7fb1cbfff640 1 --2- 172.21.15.16:0/3380209582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1cc0d1900 0x7fb1cc0cf810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80097d0 tx=0x7fb1b802b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4eefa0f970e045b server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:38.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.441+0000 7fb1cb7fe640 1 -- 172.21.15.16:0/3380209582 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c40019f0 con 0x7fb1cc0d1900 2023-12-08T08:39:38.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.441+0000 7fb1cb7fe640 1 -- 172.21.15.16:0/3380209582 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b802e050 con 0x7fb1cc0d1900 2023-12-08T08:39:38.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.441+0000 7fb1cb7fe640 1 -- 172.21.15.16:0/3380209582 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8030710 con 0x7fb1cc0d1900 2023-12-08T08:39:38.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.442+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/3380209582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1cc0d1900 msgr2=0x7fb1cc0cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.442+0000 7fb1d2bbb640 1 --2- 172.21.15.16:0/3380209582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1cc0d1900 0x7fb1cc0cf810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb1b80097d0 tx=0x7fb1b802b770 comp rx=0 tx=0).stop 2023-12-08T08:39:38.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.442+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/3380209582 shutdown_connections 2023-12-08T08:39:38.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.442+0000 7fb1d2bbb640 1 --2- 172.21.15.16:0/3380209582 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1cc0d1900 0x7fb1cc0cf810 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.442+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/3380209582 >> 172.21.15.16:0/3380209582 conn(0x7fb1cc0d0de0 msgr2=0x7fb1cc0d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:38.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.442+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/3380209582 shutdown_connections 2023-12-08T08:39:38.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.442+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/3380209582 wait complete. 2023-12-08T08:39:38.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.443+0000 7fb1d2bbb640 1 Processor -- start 2023-12-08T08:39:38.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.443+0000 7fb1d2bbb640 1 -- start start 2023-12-08T08:39:38.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.444+0000 7fb1d2bbb640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1cc0cd760 0x7fb1cc15a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.444+0000 7fb1d2bbb640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1cc0d1900 0x7fb1cc09c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.444+0000 7fb1d2bbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1cc15b0f0 0x7fb1cc0c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.444+0000 7fb1d2bbb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc15a930 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.444+0000 7fb1d2bbb640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc09e950 con 0x7fb1cc0cd760 2023-12-08T08:39:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.444+0000 7fb1d2bbb640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc09bfa0 con 0x7fb1cc0d1900 2023-12-08T08:39:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.444+0000 7fb1d1158640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1cc15b0f0 0x7fb1cc0c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.444+0000 7fb1d1158640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1cc15b0f0 0x7fb1cc0c6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34580/0 (socket says 172.21.15.16:34580) 2023-12-08T08:39:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.444+0000 7fb1d1158640 1 -- 172.21.15.16:0/1536254137 learned_addr learned my addr 172.21.15.16:0/1536254137 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.444+0000 7fb1cbfff640 1 --2- 172.21.15.16:0/1536254137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1cc0d1900 0x7fb1cc09c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.444+0000 7fb1d0957640 1 --2- 172.21.15.16:0/1536254137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1cc0cd760 0x7fb1cc15a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.444+0000 7fb1d1158640 1 -- 172.21.15.16:0/1536254137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1cc0d1900 msgr2=0x7fb1cc09c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.444+0000 7fb1d1158640 1 --2- 172.21.15.16:0/1536254137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1cc0d1900 0x7fb1cc09c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.445+0000 7fb1d1158640 1 -- 172.21.15.16:0/1536254137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1cc0cd760 msgr2=0x7fb1cc15a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.445+0000 7fb1d1158640 1 --2- 172.21.15.16:0/1536254137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1cc0cd760 0x7fb1cc15a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.445+0000 7fb1d1158640 1 -- 172.21.15.16:0/1536254137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b8009400 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.445+0000 7fb1d1158640 1 --2- 172.21.15.16:0/1536254137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1cc15b0f0 0x7fb1cc0c6ae0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400cc20 tx=0x7fb1c400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.445+0000 7fb1c97fa640 1 -- 172.21.15.16:0/1536254137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c400ce30 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.445+0000 7fb1c97fa640 1 -- 172.21.15.16:0/1536254137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c4012400 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.445+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/1536254137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1cc09c560 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.445+0000 7fb1c97fa640 1 -- 172.21.15.16:0/1536254137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c40127b0 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.445+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/1536254137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1cc09ba40 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.446+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/1536254137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1cc067430 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.447+0000 7fb1c97fa640 1 -- 172.21.15.16:0/1536254137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb1c4012910 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.447+0000 7fb1c97fa640 1 --2- 172.21.15.16:0/1536254137 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb1b006ae40 0x7fb1b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.447+0000 7fb1c97fa640 1 -- 172.21.15.16:0/1536254137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb1c4089a40 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.447+0000 7fb1d0957640 1 --2- 172.21.15.16:0/1536254137 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb1b006ae40 0x7fb1b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.448+0000 7fb1d0957640 1 --2- 172.21.15.16:0/1536254137 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb1b006ae40 0x7fb1b006d300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00011e0 tx=0x7fb1c0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:38.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.451+0000 7fb1c97fa640 1 -- 172.21.15.16:0/1536254137 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb1c408be60 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.627+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/1536254137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1cc072520 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.628+0000 7fb1c97fa640 1 -- 172.21.15.16:0/1536254137 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13062 (secure 0 0 0) 0x7fb1c4054db0 con 0x7fb1cc15b0f0 2023-12-08T08:39:38.630 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:38.631 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":35,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:35.643587+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.630+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/1536254137 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb1b006ae40 msgr2=0x7fb1b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.630+0000 7fb1d2bbb640 1 --2- 172.21.15.16:0/1536254137 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb1b006ae40 0x7fb1b006d300 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00011e0 tx=0x7fb1c0009040 comp rx=0 tx=0).stop 2023-12-08T08:39:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.630+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/1536254137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1cc15b0f0 msgr2=0x7fb1cc0c6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.630+0000 7fb1d2bbb640 1 --2- 172.21.15.16:0/1536254137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1cc15b0f0 0x7fb1cc0c6ae0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb1c400cc20 tx=0x7fb1c400a430 comp rx=0 tx=0).stop 2023-12-08T08:39:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.630+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/1536254137 shutdown_connections 2023-12-08T08:39:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.630+0000 7fb1d2bbb640 1 --2- 172.21.15.16:0/1536254137 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb1b006ae40 0x7fb1b006d300 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.630+0000 7fb1d2bbb640 1 --2- 172.21.15.16:0/1536254137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1cc15b0f0 0x7fb1cc0c6ae0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.630+0000 7fb1d2bbb640 1 --2- 172.21.15.16:0/1536254137 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb1cc0d1900 0x7fb1cc09c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.630+0000 7fb1d2bbb640 1 --2- 172.21.15.16:0/1536254137 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb1cc0cd760 0x7fb1cc15a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.630+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/1536254137 >> 172.21.15.16:0/1536254137 conn(0x7fb1cc0d0de0 msgr2=0x7fb1cc15caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.630+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/1536254137 shutdown_connections 2023-12-08T08:39:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.630+0000 7fb1d2bbb640 1 -- 172.21.15.16:0/1536254137 wait complete. 2023-12-08T08:39:38.643 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-08T08:38:07.031785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-08T08:39:38.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-08T08:39:38.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.751+0000 7fe6b6dd7640 1 Processor -- start 2023-12-08T08:39:38.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.751+0000 7fe6b6dd7640 1 -- start start 2023-12-08T08:39:38.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.752+0000 7fe6b6dd7640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6b0160980 0x7fe6b00cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.752+0000 7fe6b6dd7640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fe6b006b6e0 0x7fe6b006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.752+0000 7fe6b6dd7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6b00d0000 0x7fe6b006c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.752+0000 7fe6b6dd7640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b00d04f0 con 0x7fe6b0160980 2023-12-08T08:39:38.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.752+0000 7fe6b6dd7640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b00b1260 con 0x7fe6b006b6e0 2023-12-08T08:39:38.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.752+0000 7fe6b6dd7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b00b0930 con 0x7fe6b00d0000 2023-12-08T08:39:38.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.752+0000 7fe6b4b73640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6b0160980 0x7fe6b00cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.752+0000 7fe6b4b73640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6b0160980 0x7fe6b00cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34582/0 (socket says 172.21.15.16:34582) 2023-12-08T08:39:38.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.752+0000 7fe6b4b73640 1 -- 172.21.15.16:0/4194759084 learned_addr learned my addr 172.21.15.16:0/4194759084 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:38.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.753+0000 7fe6affff640 1 --2- 172.21.15.16:0/4194759084 >> v2:172.21.15.112:3300/0 conn(0x7fe6b006b6e0 0x7fe6b006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.753+0000 7fe6b5374640 1 --2- 172.21.15.16:0/4194759084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6b00d0000 0x7fe6b006c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.753+0000 7fe6b4b73640 1 -- 172.21.15.16:0/4194759084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6b00d0000 msgr2=0x7fe6b006c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:38.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.753+0000 7fe6b4b73640 1 --2- 172.21.15.16:0/4194759084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6b00d0000 0x7fe6b006c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.753+0000 7fe6b4b73640 1 -- 172.21.15.16:0/4194759084 >> v2:172.21.15.112:3300/0 conn(0x7fe6b006b6e0 msgr2=0x7fe6b006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:38.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.753+0000 7fe6b4b73640 1 --2- 172.21.15.16:0/4194759084 >> v2:172.21.15.112:3300/0 conn(0x7fe6b006b6e0 0x7fe6b006bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.753+0000 7fe6b4b73640 1 -- 172.21.15.16:0/4194759084 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b00b0c10 con 0x7fe6b0160980 2023-12-08T08:39:38.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.753+0000 7fe6b4b73640 1 --2- 172.21.15.16:0/4194759084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b0160980 0x7fe6b00cfac0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe6b006bc50 tx=0x7fe6a802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d6ed0cf9709d48 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:38.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.754+0000 7fe6af7fe640 1 -- 172.21.15.16:0/4194759084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a802cb00 con 0x7fe6b0160980 2023-12-08T08:39:38.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.754+0000 7fe6af7fe640 1 -- 172.21.15.16:0/4194759084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a802cc60 con 0x7fe6b0160980 2023-12-08T08:39:38.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.754+0000 7fe6af7fe640 1 -- 172.21.15.16:0/4194759084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80343f0 con 0x7fe6b0160980 2023-12-08T08:39:38.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.754+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/4194759084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b0160980 msgr2=0x7fe6b00cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.754+0000 7fe6b6dd7640 1 --2- 172.21.15.16:0/4194759084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b0160980 0x7fe6b00cfac0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe6b006bc50 tx=0x7fe6a802bf60 comp rx=0 tx=0).stop 2023-12-08T08:39:38.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.754+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/4194759084 shutdown_connections 2023-12-08T08:39:38.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.754+0000 7fe6b6dd7640 1 --2- 172.21.15.16:0/4194759084 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6b00d0000 0x7fe6b006c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.754+0000 7fe6b6dd7640 1 --2- 172.21.15.16:0/4194759084 >> v2:172.21.15.112:3300/0 conn(0x7fe6b006b6e0 0x7fe6b006bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.754+0000 7fe6b6dd7640 1 --2- 172.21.15.16:0/4194759084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b0160980 0x7fe6b00cfac0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.754+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/4194759084 >> 172.21.15.16:0/4194759084 conn(0x7fe6b00d0e30 msgr2=0x7fe6b00d1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:38.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.754+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/4194759084 shutdown_connections 2023-12-08T08:39:38.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.754+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/4194759084 wait complete. 2023-12-08T08:39:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.755+0000 7fe6b6dd7640 1 Processor -- start 2023-12-08T08:39:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.755+0000 7fe6b6dd7640 1 -- start start 2023-12-08T08:39:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.756+0000 7fe6b6dd7640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b006b6e0 0x7fe6b0156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.756+0000 7fe6b6dd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b00d0000 0x7fe6b0151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.756+0000 7fe6b6dd7640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6b0160980 0x7fe6b01593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.756+0000 7fe6b6dd7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b0156b80 con 0x7fe6b00d0000 2023-12-08T08:39:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.756+0000 7fe6b6dd7640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b00c6250 con 0x7fe6b006b6e0 2023-12-08T08:39:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.756+0000 7fe6b6dd7640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b01523d0 con 0x7fe6b0160980 2023-12-08T08:39:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.756+0000 7fe6affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b00d0000 0x7fe6b0151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.756+0000 7fe6affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b00d0000 0x7fe6b0151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34594/0 (socket says 172.21.15.16:34594) 2023-12-08T08:39:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6affff640 1 -- 172.21.15.16:0/2723021489 learned_addr learned my addr 172.21.15.16:0/2723021489 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6b4b73640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b006b6e0 0x7fe6b0156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6b5374640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6b0160980 0x7fe6b01593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6affff640 1 -- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6b0160980 msgr2=0x7fe6b01593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6affff640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6b0160980 0x7fe6b01593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6affff640 1 -- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b006b6e0 msgr2=0x7fe6b0156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6affff640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b006b6e0 0x7fe6b0156640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6affff640 1 -- 172.21.15.16:0/2723021489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a8009b30 con 0x7fe6b00d0000 2023-12-08T08:39:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6b5374640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6b0160980 0x7fe6b01593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6affff640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b00d0000 0x7fe6b0151e90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400e890 tx=0x7fe6a400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:38.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6ad7fa640 1 -- 172.21.15.16:0/2723021489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a400c940 con 0x7fe6b00d0000 2023-12-08T08:39:38.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6ad7fa640 1 -- 172.21.15.16:0/2723021489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a400caa0 con 0x7fe6b00d0000 2023-12-08T08:39:38.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/2723021489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b00c5c90 con 0x7fe6b00d0000 2023-12-08T08:39:38.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6ad7fa640 1 -- 172.21.15.16:0/2723021489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a400ce30 con 0x7fe6b00d0000 2023-12-08T08:39:38.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.757+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/2723021489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b0158bc0 con 0x7fe6b00d0000 2023-12-08T08:39:38.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.758+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/2723021489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b0067ad0 con 0x7fe6b00d0000 2023-12-08T08:39:38.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.759+0000 7fe6ad7fa640 1 -- 172.21.15.16:0/2723021489 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe6a4013470 con 0x7fe6b00d0000 2023-12-08T08:39:38.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.759+0000 7fe6ad7fa640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe69c06ae40 0x7fe69c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:38.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.759+0000 7fe6ad7fa640 1 -- 172.21.15.16:0/2723021489 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe6a4089580 con 0x7fe6b00d0000 2023-12-08T08:39:38.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.759+0000 7fe6b4b73640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe69c06ae40 0x7fe69c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:38.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.760+0000 7fe6b4b73640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe69c06ae40 0x7fe69c06d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8035000 tx=0x7fe6a8030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:38.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.763+0000 7fe6ad7fa640 1 -- 172.21.15.16:0/2723021489 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe6a40548d0 con 0x7fe6b00d0000 2023-12-08T08:39:38.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.938+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/2723021489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6b00c7c70 con 0x7fe6b00d0000 2023-12-08T08:39:38.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.939+0000 7fe6ad7fa640 1 -- 172.21.15.16:0/2723021489 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6a4057ef0 con 0x7fe6b00d0000 2023-12-08T08:39:38.941 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-12-08T08:39:38.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.941+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe69c06ae40 msgr2=0x7fe69c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.941+0000 7fe6b6dd7640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe69c06ae40 0x7fe69c06d300 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe6a8035000 tx=0x7fe6a8030000 comp rx=0 tx=0).stop 2023-12-08T08:39:38.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.941+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/2723021489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b00d0000 msgr2=0x7fe6b0151e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:38.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.941+0000 7fe6b6dd7640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b00d0000 0x7fe6b0151e90 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400e890 tx=0x7fe6a400ed50 comp rx=0 tx=0).stop 2023-12-08T08:39:38.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.941+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/2723021489 shutdown_connections 2023-12-08T08:39:38.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.941+0000 7fe6b6dd7640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fe69c06ae40 0x7fe69c06d300 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.941+0000 7fe6b6dd7640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fe6b0160980 0x7fe6b01593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.941+0000 7fe6b6dd7640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b00d0000 0x7fe6b0151e90 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.941+0000 7fe6b6dd7640 1 --2- 172.21.15.16:0/2723021489 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fe6b006b6e0 0x7fe6b0156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:38.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.941+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/2723021489 >> 172.21.15.16:0/2723021489 conn(0x7fe6b00d0e30 msgr2=0x7fe6b006f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:38.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.941+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/2723021489 shutdown_connections 2023-12-08T08:39:38.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:38.941+0000 7fe6b6dd7640 1 -- 172.21.15.16:0/2723021489 wait complete. 2023-12-08T08:39:38.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-08T08:39:39.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.059+0000 7fb946ecf640 1 Processor -- start 2023-12-08T08:39:39.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.059+0000 7fb946ecf640 1 -- start start 2023-12-08T08:39:39.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.059+0000 7fb946ecf640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb9401608a0 0x7fb9400d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:39.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.059+0000 7fb946ecf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9400b0cf0 0x7fb9400cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:39.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.059+0000 7fb946ecf640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9400d20b0 con 0x7fb9400b1d70 2023-12-08T08:39:39.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.059+0000 7fb946ecf640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9400d0620 con 0x7fb9401608a0 2023-12-08T08:39:39.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.059+0000 7fb946ecf640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9400d1b80 con 0x7fb9400b0cf0 2023-12-08T08:39:39.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.060+0000 7fb944c6b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fb9401608a0 0x7fb9400d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:39.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.060+0000 7fb94546c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9400b0cf0 0x7fb9400cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:39.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.060+0000 7fb93ffff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fb9400b1d70 0x7fb9400d1470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:39134/0 (socket says 172.21.15.16:39134) 2023-12-08T08:39:39.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.060+0000 7fb93ffff640 1 -- 172.21.15.16:0/2502167481 learned_addr learned my addr 172.21.15.16:0/2502167481 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:39.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.060+0000 7fb93f7fe640 1 -- 172.21.15.16:0/2502167481 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3675479619 0 0) 0x7fb9400d20b0 con 0x7fb9400b1d70 2023-12-08T08:39:39.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.060+0000 7fb93f7fe640 1 -- 172.21.15.16:0/2502167481 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb930003520 con 0x7fb9400b1d70 2023-12-08T08:39:39.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.060+0000 7fb944c6b640 1 -- 172.21.15.16:0/2502167481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9400b0cf0 msgr2=0x7fb9400cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:39.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.060+0000 7fb944c6b640 1 --2- 172.21.15.16:0/2502167481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9400b0cf0 0x7fb9400cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.060+0000 7fb944c6b640 1 -- 172.21.15.16:0/2502167481 >> v1:172.21.15.112:6789/0 conn(0x7fb9400b1d70 legacy=0x7fb9400d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:39.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.060+0000 7fb944c6b640 1 -- 172.21.15.16:0/2502167481 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9400d1dd0 con 0x7fb9401608a0 2023-12-08T08:39:39.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.060+0000 7fb94546c640 1 --2- 172.21.15.16:0/2502167481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9400b0cf0 0x7fb9400cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:39:39.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.061+0000 7fb944c6b640 1 --2- 172.21.15.16:0/2502167481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9401608a0 0x7fb9400d0f30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb9340097a0 tx=0x7fb93402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d685a681e6da141d server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:39.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.061+0000 7fb93f7fe640 1 -- 172.21.15.16:0/2502167481 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93402e070 con 0x7fb9401608a0 2023-12-08T08:39:39.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.061+0000 7fb93f7fe640 1 -- 172.21.15.16:0/2502167481 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9340315a0 con 0x7fb9401608a0 2023-12-08T08:39:39.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.061+0000 7fb93f7fe640 1 -- 172.21.15.16:0/2502167481 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934031930 con 0x7fb9401608a0 2023-12-08T08:39:39.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.062+0000 7fb946ecf640 1 -- 172.21.15.16:0/2502167481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9401608a0 msgr2=0x7fb9400d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:39.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.062+0000 7fb946ecf640 1 --2- 172.21.15.16:0/2502167481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9401608a0 0x7fb9400d0f30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb9340097a0 tx=0x7fb93402b840 comp rx=0 tx=0).stop 2023-12-08T08:39:39.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.062+0000 7fb946ecf640 1 -- 172.21.15.16:0/2502167481 shutdown_connections 2023-12-08T08:39:39.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.062+0000 7fb946ecf640 1 --2- 172.21.15.16:0/2502167481 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9400b0cf0 0x7fb9400cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.062+0000 7fb946ecf640 1 --2- 172.21.15.16:0/2502167481 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9401608a0 0x7fb9400d0f30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.062+0000 7fb946ecf640 1 -- 172.21.15.16:0/2502167481 >> 172.21.15.16:0/2502167481 conn(0x7fb9400d3030 msgr2=0x7fb9400d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:39.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.062+0000 7fb946ecf640 1 -- 172.21.15.16:0/2502167481 shutdown_connections 2023-12-08T08:39:39.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.062+0000 7fb946ecf640 1 -- 172.21.15.16:0/2502167481 wait complete. 2023-12-08T08:39:39.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.063+0000 7fb946ecf640 1 Processor -- start 2023-12-08T08:39:39.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.063+0000 7fb946ecf640 1 -- start start 2023-12-08T08:39:39.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb946ecf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9400b0cf0 0x7fb940151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:39.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb946ecf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9400b1d70 0x7fb94015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:39.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb946ecf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9401608a0 0x7fb9400c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:39.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb946ecf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb940151c80 con 0x7fb9400b0cf0 2023-12-08T08:39:39.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb946ecf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94015cc00 con 0x7fb9401608a0 2023-12-08T08:39:39.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb946ecf640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94015cee0 con 0x7fb9400b1d70 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb944c6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9400b0cf0 0x7fb940151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb944c6b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9400b0cf0 0x7fb940151740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34606/0 (socket says 172.21.15.16:34606) 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb944c6b640 1 -- 172.21.15.16:0/3261383838 learned_addr learned my addr 172.21.15.16:0/3261383838 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb93ffff640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9400b1d70 0x7fb94015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb94546c640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9401608a0 0x7fb9400c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb944c6b640 1 -- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9400b1d70 msgr2=0x7fb94015dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb944c6b640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9400b1d70 0x7fb94015dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb944c6b640 1 -- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9401608a0 msgr2=0x7fb9400c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb944c6b640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9401608a0 0x7fb9400c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb944c6b640 1 -- 172.21.15.16:0/3261383838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb938009a70 con 0x7fb9400b0cf0 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.064+0000 7fb93ffff640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9400b1d70 0x7fb94015dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.065+0000 7fb944c6b640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9400b0cf0 0x7fb940151740 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb934034040 tx=0x7fb9340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.065+0000 7fb93d7fa640 1 -- 172.21.15.16:0/3261383838 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934032840 con 0x7fb9400b0cf0 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.065+0000 7fb93d7fa640 1 -- 172.21.15.16:0/3261383838 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb934031f00 con 0x7fb9400b0cf0 2023-12-08T08:39:39.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.065+0000 7fb946ecf640 1 -- 172.21.15.16:0/3261383838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb934009400 con 0x7fb9400b0cf0 2023-12-08T08:39:39.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.065+0000 7fb93d7fa640 1 -- 172.21.15.16:0/3261383838 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934032c00 con 0x7fb9400b0cf0 2023-12-08T08:39:39.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.065+0000 7fb946ecf640 1 -- 172.21.15.16:0/3261383838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94015d4a0 con 0x7fb9400b0cf0 2023-12-08T08:39:39.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.066+0000 7fb946ecf640 1 -- 172.21.15.16:0/3261383838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb940067430 con 0x7fb9400b0cf0 2023-12-08T08:39:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.066+0000 7fb93d7fa640 1 -- 172.21.15.16:0/3261383838 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb93404e430 con 0x7fb9400b0cf0 2023-12-08T08:39:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.067+0000 7fb93d7fa640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb92406ae40 0x7fb92406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.067+0000 7fb93d7fa640 1 -- 172.21.15.16:0/3261383838 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fb9340ab1b0 con 0x7fb9400b0cf0 2023-12-08T08:39:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.067+0000 7fb93ffff640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb92406ae40 0x7fb92406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:39.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.068+0000 7fb93ffff640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb92406ae40 0x7fb92406d300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb9400cd7f0 tx=0x7fb92c007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:39.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.071+0000 7fb93d7fa640 1 -- 172.21.15.16:0/3261383838 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb934076520 con 0x7fb9400b0cf0 2023-12-08T08:39:39.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.251+0000 7fb946ecf640 1 -- 172.21.15.16:0/3261383838 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb9400c66e0 con 0x7fb9400b0cf0 2023-12-08T08:39:39.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.251+0000 7fb93d7fa640 1 -- 172.21.15.16:0/3261383838 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb934079b40 con 0x7fb9400b0cf0 2023-12-08T08:39:39.254 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:39.254 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-12-08T08:39:39.254 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-12-08T08:39:39.254 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-12-08T08:39:39.254 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2023-12-08T08:39:39.254 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:39.254 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-12-08T08:39:39.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.253+0000 7fb946ecf640 1 -- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb92406ae40 msgr2=0x7fb92406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:39.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.253+0000 7fb946ecf640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb92406ae40 0x7fb92406d300 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb9400cd7f0 tx=0x7fb92c007d30 comp rx=0 tx=0).stop 2023-12-08T08:39:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.253+0000 7fb946ecf640 1 -- 172.21.15.16:0/3261383838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9400b0cf0 msgr2=0x7fb940151740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.253+0000 7fb946ecf640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9400b0cf0 0x7fb940151740 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb934034040 tx=0x7fb9340046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.253+0000 7fb946ecf640 1 -- 172.21.15.16:0/3261383838 shutdown_connections 2023-12-08T08:39:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.253+0000 7fb946ecf640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fb92406ae40 0x7fb92406d300 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.253+0000 7fb946ecf640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fb9401608a0 0x7fb9400c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.253+0000 7fb946ecf640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fb9400b1d70 0x7fb94015dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.253+0000 7fb946ecf640 1 --2- 172.21.15.16:0/3261383838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9400b0cf0 0x7fb940151740 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.253+0000 7fb946ecf640 1 -- 172.21.15.16:0/3261383838 >> 172.21.15.16:0/3261383838 conn(0x7fb9400d3030 msgr2=0x7fb9400cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.253+0000 7fb946ecf640 1 -- 172.21.15.16:0/3261383838 shutdown_connections 2023-12-08T08:39:39.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.253+0000 7fb946ecf640 1 -- 172.21.15.16:0/3261383838 wait complete. 2023-12-08T08:39:39.266 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-08T08:39:39.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-08T08:39:39.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.375+0000 7f50fd66b640 1 Processor -- start 2023-12-08T08:39:39.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.375+0000 7f50fd66b640 1 -- start start 2023-12-08T08:39:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.376+0000 7f50fd66b640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f50f80d1d90 0x7f50f80d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.376+0000 7f50fd66b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f50f80cf960 0x7f50f80cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.376+0000 7f50fd66b640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50f80ce680 con 0x7f50f80d19b0 2023-12-08T08:39:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.376+0000 7f50fd66b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f80b4210 con 0x7f50f80cf960 2023-12-08T08:39:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.376+0000 7f50fd66b640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50f80d02b0 con 0x7f50f80d1d90 2023-12-08T08:39:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.376+0000 7f50f77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f50f80cf960 0x7f50f80cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.376+0000 7f50f77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f50f80cf960 0x7f50f80cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34620/0 (socket says 172.21.15.16:34620) 2023-12-08T08:39:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.376+0000 7f50f77fe640 1 -- 172.21.15.16:0/1454047108 learned_addr learned my addr 172.21.15.16:0/1454047108 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.376+0000 7f50f67fc640 1 --2- 172.21.15.16:0/1454047108 >> v2:172.21.15.112:3300/0 conn(0x7f50f80d1d90 0x7f50f80d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:39.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.376+0000 7f50f77fe640 1 -- 172.21.15.16:0/1454047108 >> v2:172.21.15.112:3300/0 conn(0x7f50f80d1d90 msgr2=0x7f50f80d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:39.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.376+0000 7f50f77fe640 1 --2- 172.21.15.16:0/1454047108 >> v2:172.21.15.112:3300/0 conn(0x7f50f80d1d90 0x7f50f80d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.376+0000 7f50f77fe640 1 -- 172.21.15.16:0/1454047108 >> v1:172.21.15.112:6789/0 conn(0x7f50f80d19b0 legacy=0x7f50f80d36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:39.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.376+0000 7f50f77fe640 1 -- 172.21.15.16:0/1454047108 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f80ce900 con 0x7f50f80cf960 2023-12-08T08:39:39.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.376+0000 7f50f67fc640 1 --2- 172.21.15.16:0/1454047108 >> v2:172.21.15.112:3300/0 conn(0x7f50f80d1d90 0x7f50f80d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:39.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.377+0000 7f50f77fe640 1 --2- 172.21.15.16:0/1454047108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f80cf960 0x7f50f80cfd70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f50f80cd670 tx=0x7f50f002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdae6b2b156c491 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:39.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.377+0000 7f50f5ffb640 1 -- 172.21.15.16:0/1454047108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f002cb20 con 0x7f50f80cf960 2023-12-08T08:39:39.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.377+0000 7f50f5ffb640 1 -- 172.21.15.16:0/1454047108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f002cc80 con 0x7f50f80cf960 2023-12-08T08:39:39.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.377+0000 7f50f5ffb640 1 -- 172.21.15.16:0/1454047108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f00343f0 con 0x7f50f80cf960 2023-12-08T08:39:39.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.378+0000 7f50fd66b640 1 -- 172.21.15.16:0/1454047108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f80cf960 msgr2=0x7f50f80cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:39.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.378+0000 7f50fd66b640 1 --2- 172.21.15.16:0/1454047108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f80cf960 0x7f50f80cfd70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f50f80cd670 tx=0x7f50f002bf80 comp rx=0 tx=0).stop 2023-12-08T08:39:39.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.378+0000 7f50fd66b640 1 -- 172.21.15.16:0/1454047108 shutdown_connections 2023-12-08T08:39:39.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.378+0000 7f50fd66b640 1 --2- 172.21.15.16:0/1454047108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f80cf960 0x7f50f80cfd70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.378+0000 7f50fd66b640 1 --2- 172.21.15.16:0/1454047108 >> v2:172.21.15.112:3300/0 conn(0x7f50f80d1d90 0x7f50f80d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.378+0000 7f50fd66b640 1 -- 172.21.15.16:0/1454047108 >> 172.21.15.16:0/1454047108 conn(0x7f50f809c360 msgr2=0x7f50f80d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:39.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.378+0000 7f50fd66b640 1 -- 172.21.15.16:0/1454047108 shutdown_connections 2023-12-08T08:39:39.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.378+0000 7f50fd66b640 1 -- 172.21.15.16:0/1454047108 wait complete. 2023-12-08T08:39:39.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.379+0000 7f50fd66b640 1 Processor -- start 2023-12-08T08:39:39.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.379+0000 7f50fd66b640 1 -- start start 2023-12-08T08:39:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50fd66b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50f80cf960 0x7f50f8150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50fd66b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50f80d19b0 0x7f50f81f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50fd66b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f80d1d90 0x7f50f815a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50fd66b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f81508d0 con 0x7f50f80d1d90 2023-12-08T08:39:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50fd66b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f80b6640 con 0x7f50f80d19b0 2023-12-08T08:39:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50fd66b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50f80b6920 con 0x7f50f80cf960 2023-12-08T08:39:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f80d1d90 0x7f50f815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f80d1d90 0x7f50f815a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34624/0 (socket says 172.21.15.16:34624) 2023-12-08T08:39:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50f77fe640 1 -- 172.21.15.16:0/2085299089 learned_addr learned my addr 172.21.15.16:0/2085299089 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50f6ffd640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50f80cf960 0x7f50f8150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50f67fc640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50f80d19b0 0x7f50f81f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50f77fe640 1 -- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50f80cf960 msgr2=0x7f50f8150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50f77fe640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50f80cf960 0x7f50f8150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50f77fe640 1 -- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50f80d19b0 msgr2=0x7f50f81f31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50f77fe640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50f80d19b0 0x7f50f81f31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50f77fe640 1 -- 172.21.15.16:0/2085299089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f0009b30 con 0x7f50f80d1d90 2023-12-08T08:39:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50f67fc640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50f80d19b0 0x7f50f81f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.380+0000 7f50f77fe640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f80d1d90 0x7f50f815a680 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f50f0036040 tx=0x7f50f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.381+0000 7f50df7fe640 1 -- 172.21.15.16:0/2085299089 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0007e70 con 0x7f50f80d1d90 2023-12-08T08:39:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.381+0000 7f50df7fe640 1 -- 172.21.15.16:0/2085299089 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f0009510 con 0x7f50f80d1d90 2023-12-08T08:39:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.381+0000 7f50fd66b640 1 -- 172.21.15.16:0/2085299089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f80b6c00 con 0x7f50f80d1d90 2023-12-08T08:39:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.381+0000 7f50df7fe640 1 -- 172.21.15.16:0/2085299089 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f003e690 con 0x7f50f80d1d90 2023-12-08T08:39:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.381+0000 7f50fd66b640 1 -- 172.21.15.16:0/2085299089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f80b71c0 con 0x7f50f80d1d90 2023-12-08T08:39:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.381+0000 7f50fd66b640 1 -- 172.21.15.16:0/2085299089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f80b73b0 con 0x7f50f80d1d90 2023-12-08T08:39:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.382+0000 7f50df7fe640 1 -- 172.21.15.16:0/2085299089 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f50f003e7f0 con 0x7f50f80d1d90 2023-12-08T08:39:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.382+0000 7f50df7fe640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f50e406ae40 0x7f50e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.383+0000 7f50df7fe640 1 -- 172.21.15.16:0/2085299089 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f50f00abfd0 con 0x7f50f80d1d90 2023-12-08T08:39:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.383+0000 7f50f6ffd640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f50e406ae40 0x7f50e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.383+0000 7f50f6ffd640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f50e406ae40 0x7f50e406d300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f50e8000f60 tx=0x7f50e8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.387+0000 7f50df7fe640 1 -- 172.21.15.16:0/2085299089 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50f00773b0 con 0x7f50f80d1d90 2023-12-08T08:39:39.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:39.562+0000 7f50fd66b640 1 -- 172.21.15.16:0/2085299089 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f50f8067540 con 0x7f50f80d1d90 2023-12-08T08:39:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.365+0000 7f50df7fe640 1 -- 172.21.15.16:0/2085299089 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7f50f007a9d0 con 0x7f50f80d1d90 2023-12-08T08:39:40.367 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-12-08T08:39:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.368+0000 7f50fd66b640 1 -- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f50e406ae40 msgr2=0x7f50e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.368+0000 7f50fd66b640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f50e406ae40 0x7f50e406d300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f50e8000f60 tx=0x7f50e8009040 comp rx=0 tx=0).stop 2023-12-08T08:39:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.368+0000 7f50fd66b640 1 -- 172.21.15.16:0/2085299089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f80d1d90 msgr2=0x7f50f815a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.368+0000 7f50fd66b640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f80d1d90 0x7f50f815a680 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f50f0036040 tx=0x7f50f0004a40 comp rx=0 tx=0).stop 2023-12-08T08:39:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.368+0000 7f50fd66b640 1 -- 172.21.15.16:0/2085299089 shutdown_connections 2023-12-08T08:39:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.368+0000 7f50fd66b640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f50e406ae40 0x7f50e406d300 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.368+0000 7f50fd66b640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50f80d1d90 0x7f50f815a680 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:40.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.368+0000 7f50fd66b640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f50f80d19b0 0x7f50f81f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.368+0000 7f50fd66b640 1 --2- 172.21.15.16:0/2085299089 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f50f80cf960 0x7f50f8150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.368+0000 7f50fd66b640 1 -- 172.21.15.16:0/2085299089 >> 172.21.15.16:0/2085299089 conn(0x7f50f809c360 msgr2=0x7f50f80afc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.368+0000 7f50fd66b640 1 -- 172.21.15.16:0/2085299089 shutdown_connections 2023-12-08T08:39:40.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.369+0000 7f50fd66b640 1 -- 172.21.15.16:0/2085299089 wait complete. 2023-12-08T08:39:40.388 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-08T08:39:40.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.505+0000 7f1d930c9640 1 Processor -- start 2023-12-08T08:39:40.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.505+0000 7f1d930c9640 1 -- start start 2023-12-08T08:39:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.506+0000 7f1d930c9640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d8c15d6f0 0x7f1d8c0ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.506+0000 7f1d930c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d8c0cde70 0x7f1d8c0cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.506+0000 7f1d930c9640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1d8c0ccd20 0x7f1d8c0cf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.506+0000 7f1d930c9640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8c0ce350 con 0x7f1d8c15d6f0 2023-12-08T08:39:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.506+0000 7f1d930c9640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8c0ce630 con 0x7f1d8c0ccd20 2023-12-08T08:39:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.506+0000 7f1d930c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c0cbd00 con 0x7f1d8c0cde70 2023-12-08T08:39:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.506+0000 7f1d90e65640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d8c15d6f0 0x7f1d8c0ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.506+0000 7f1d90e65640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d8c15d6f0 0x7f1d8c0ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34628/0 (socket says 172.21.15.16:34628) 2023-12-08T08:39:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.506+0000 7f1d90e65640 1 -- 172.21.15.16:0/1961142907 learned_addr learned my addr 172.21.15.16:0/1961142907 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.506+0000 7f1d8bfff640 1 --2- 172.21.15.16:0/1961142907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d8c0cde70 0x7f1d8c0cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.506+0000 7f1d91666640 1 --2- 172.21.15.16:0/1961142907 >> v2:172.21.15.112:3300/0 conn(0x7f1d8c0ccd20 0x7f1d8c0cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.507+0000 7f1d90e65640 1 -- 172.21.15.16:0/1961142907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d8c0cde70 msgr2=0x7f1d8c0cc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.507+0000 7f1d90e65640 1 --2- 172.21.15.16:0/1961142907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d8c0cde70 0x7f1d8c0cc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.507+0000 7f1d90e65640 1 -- 172.21.15.16:0/1961142907 >> v2:172.21.15.112:3300/0 conn(0x7f1d8c0ccd20 msgr2=0x7f1d8c0cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.507+0000 7f1d90e65640 1 --2- 172.21.15.16:0/1961142907 >> v2:172.21.15.112:3300/0 conn(0x7f1d8c0ccd20 0x7f1d8c0cf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.507+0000 7f1d90e65640 1 -- 172.21.15.16:0/1961142907 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c0cbfe0 con 0x7f1d8c15d6f0 2023-12-08T08:39:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.507+0000 7f1d8bfff640 1 --2- 172.21.15.16:0/1961142907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d8c0cde70 0x7f1d8c0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.507+0000 7f1d91666640 1 --2- 172.21.15.16:0/1961142907 >> v2:172.21.15.112:3300/0 conn(0x7f1d8c0ccd20 0x7f1d8c0cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.507+0000 7f1d90e65640 1 --2- 172.21.15.16:0/1961142907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c15d6f0 0x7f1d8c0ceb50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c072100 tx=0x7f1d8402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b6d69016a87d35 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:40.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.507+0000 7f1d8b7fe640 1 -- 172.21.15.16:0/1961142907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8402c9e0 con 0x7f1d8c15d6f0 2023-12-08T08:39:40.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.508+0000 7f1d8b7fe640 1 -- 172.21.15.16:0/1961142907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8402cb40 con 0x7f1d8c15d6f0 2023-12-08T08:39:40.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.508+0000 7f1d8b7fe640 1 -- 172.21.15.16:0/1961142907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d840343f0 con 0x7f1d8c15d6f0 2023-12-08T08:39:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.508+0000 7f1d930c9640 1 -- 172.21.15.16:0/1961142907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c15d6f0 msgr2=0x7f1d8c0ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.508+0000 7f1d930c9640 1 --2- 172.21.15.16:0/1961142907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c15d6f0 0x7f1d8c0ceb50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c072100 tx=0x7f1d8402be40 comp rx=0 tx=0).stop 2023-12-08T08:39:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.508+0000 7f1d930c9640 1 -- 172.21.15.16:0/1961142907 shutdown_connections 2023-12-08T08:39:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.508+0000 7f1d930c9640 1 --2- 172.21.15.16:0/1961142907 >> v2:172.21.15.112:3300/0 conn(0x7f1d8c0ccd20 0x7f1d8c0cf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.508+0000 7f1d930c9640 1 --2- 172.21.15.16:0/1961142907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d8c0cde70 0x7f1d8c0cc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.508+0000 7f1d930c9640 1 --2- 172.21.15.16:0/1961142907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c15d6f0 0x7f1d8c0ceb50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.508+0000 7f1d930c9640 1 -- 172.21.15.16:0/1961142907 >> 172.21.15.16:0/1961142907 conn(0x7f1d8c0d0d70 msgr2=0x7f1d8c0d1180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.508+0000 7f1d930c9640 1 -- 172.21.15.16:0/1961142907 shutdown_connections 2023-12-08T08:39:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.508+0000 7f1d930c9640 1 -- 172.21.15.16:0/1961142907 wait complete. 2023-12-08T08:39:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.509+0000 7f1d930c9640 1 Processor -- start 2023-12-08T08:39:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.509+0000 7f1d930c9640 1 -- start start 2023-12-08T08:39:40.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.510+0000 7f1d930c9640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d8c0ccd20 0x7f1d8c15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:40.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.510+0000 7f1d930c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c0cde70 0x7f1d8c09b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:40.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.510+0000 7f1d930c9640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d8c15d6f0 0x7f1d8c1f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.510+0000 7f1d930c9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c09e350 con 0x7f1d8c0cde70 2023-12-08T08:39:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.510+0000 7f1d930c9640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c15a510 con 0x7f1d8c0ccd20 2023-12-08T08:39:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.510+0000 7f1d930c9640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c15ce60 con 0x7f1d8c15d6f0 2023-12-08T08:39:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.510+0000 7f1d8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c0cde70 0x7f1d8c09b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.510+0000 7f1d8bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c0cde70 0x7f1d8c09b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34630/0 (socket says 172.21.15.16:34630) 2023-12-08T08:39:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.510+0000 7f1d8bfff640 1 -- 172.21.15.16:0/3442477367 learned_addr learned my addr 172.21.15.16:0/3442477367 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.510+0000 7f1d91666640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d8c15d6f0 0x7f1d8c1f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.510+0000 7f1d90e65640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d8c0ccd20 0x7f1d8c15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.511+0000 7f1d8bfff640 1 -- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d8c15d6f0 msgr2=0x7f1d8c1f2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.511+0000 7f1d8bfff640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d8c15d6f0 0x7f1d8c1f2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.511+0000 7f1d8bfff640 1 -- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d8c0ccd20 msgr2=0x7f1d8c15c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.511+0000 7f1d8bfff640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d8c0ccd20 0x7f1d8c15c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.511+0000 7f1d8bfff640 1 -- 172.21.15.16:0/3442477367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d84009b30 con 0x7f1d8c0cde70 2023-12-08T08:39:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.511+0000 7f1d90e65640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d8c0ccd20 0x7f1d8c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.511+0000 7f1d8bfff640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c0cde70 0x7f1d8c09b600 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00c7f0 tx=0x7f1d7c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.511+0000 7f1d897fa640 1 -- 172.21.15.16:0/3442477367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c007a30 con 0x7f1d8c0cde70 2023-12-08T08:39:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.511+0000 7f1d897fa640 1 -- 172.21.15.16:0/3442477367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c007b90 con 0x7f1d8c0cde70 2023-12-08T08:39:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.511+0000 7f1d930c9640 1 -- 172.21.15.16:0/3442477367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d8c159be0 con 0x7f1d8c0cde70 2023-12-08T08:39:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.511+0000 7f1d897fa640 1 -- 172.21.15.16:0/3442477367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c00ce40 con 0x7f1d8c0cde70 2023-12-08T08:39:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.511+0000 7f1d930c9640 1 -- 172.21.15.16:0/3442477367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8c15a1a0 con 0x7f1d8c0cde70 2023-12-08T08:39:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.512+0000 7f1d930c9640 1 -- 172.21.15.16:0/3442477367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d8c15a6c0 con 0x7f1d8c0cde70 2023-12-08T08:39:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.512+0000 7f1d897fa640 1 -- 172.21.15.16:0/3442477367 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d7c0113f0 con 0x7f1d8c0cde70 2023-12-08T08:39:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.513+0000 7f1d897fa640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1d6c06ae40 0x7f1d6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.513+0000 7f1d897fa640 1 -- 172.21.15.16:0/3442477367 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5547+0+0 (secure 0 0 0) 0x7f1d7c089610 con 0x7f1d8c0cde70 2023-12-08T08:39:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.513+0000 7f1d90e65640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1d6c06ae40 0x7f1d6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.514+0000 7f1d90e65640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1d6c06ae40 0x7f1d6c06d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1d84035000 tx=0x7f1d84030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:40.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.517+0000 7f1d897fa640 1 -- 172.21.15.16:0/3442477367 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d8c15a6c0 con 0x7f1d8c0cde70 2023-12-08T08:39:40.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:39:40.589 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:39:40.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:40.695+0000 7f1d930c9640 1 -- 172.21.15.16:0/3442477367 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1d8c0b2170 con 0x7f1d8c0cde70 2023-12-08T08:39:41.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.370+0000 7f1d897fa640 1 -- 172.21.15.16:0/3442477367 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1d7c0582a0 con 0x7f1d8c0cde70 2023-12-08T08:39:41.373 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-12-08T08:39:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.374+0000 7f1d930c9640 1 -- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1d6c06ae40 msgr2=0x7f1d6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.374+0000 7f1d930c9640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1d6c06ae40 0x7f1d6c06d300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1d84035000 tx=0x7f1d84030000 comp rx=0 tx=0).stop 2023-12-08T08:39:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.374+0000 7f1d930c9640 1 -- 172.21.15.16:0/3442477367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c0cde70 msgr2=0x7f1d8c09b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.374+0000 7f1d930c9640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c0cde70 0x7f1d8c09b600 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00c7f0 tx=0x7f1d7c00cbf0 comp rx=0 tx=0).stop 2023-12-08T08:39:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.375+0000 7f1d930c9640 1 -- 172.21.15.16:0/3442477367 shutdown_connections 2023-12-08T08:39:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.375+0000 7f1d930c9640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1d6c06ae40 0x7f1d6c06d300 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.375+0000 7f1d930c9640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1d8c15d6f0 0x7f1d8c1f2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.375+0000 7f1d930c9640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d8c0cde70 0x7f1d8c09b600 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.375+0000 7f1d930c9640 1 --2- 172.21.15.16:0/3442477367 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1d8c0ccd20 0x7f1d8c15c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.375+0000 7f1d930c9640 1 -- 172.21.15.16:0/3442477367 >> 172.21.15.16:0/3442477367 conn(0x7f1d8c0d0d70 msgr2=0x7f1d8c06f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.376+0000 7f1d930c9640 1 -- 172.21.15.16:0/3442477367 shutdown_connections 2023-12-08T08:39:41.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.376+0000 7f1d930c9640 1 -- 172.21.15.16:0/3442477367 wait complete. 2023-12-08T08:39:41.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-08T08:39:41.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.517+0000 7f2aedad4640 1 Processor -- start 2023-12-08T08:39:41.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.517+0000 7f2aedad4640 1 -- start start 2023-12-08T08:39:41.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.518+0000 7f2aedad4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ae8145d80 0x7f2ae8071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:41.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.518+0000 7f2aedad4640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae80b4e00 con 0x7f2ae80b4920 2023-12-08T08:39:41.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.518+0000 7f2aedad4640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae80b50e0 con 0x7f2ae806a4d0 2023-12-08T08:39:41.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.518+0000 7f2aedad4640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae806cfb0 con 0x7f2ae8145d80 2023-12-08T08:39:41.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.518+0000 7f2ae77fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ae8145d80 0x7f2ae8071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:41.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.518+0000 7f2ae6ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2ae80b4920 0x7f2ae80acd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51528/0 (socket says 172.21.15.16:51528) 2023-12-08T08:39:41.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.518+0000 7f2ae6ffd640 1 -- 172.21.15.16:0/1898065085 learned_addr learned my addr 172.21.15.16:0/1898065085 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:41.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.518+0000 7f2ae77fe640 1 -- 172.21.15.16:0/1898065085 >> v1:172.21.15.112:6789/0 conn(0x7f2ae806a4d0 legacy=0x7f2ae806da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:41.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.518+0000 7f2ae77fe640 1 -- 172.21.15.16:0/1898065085 >> v1:172.21.15.16:6789/0 conn(0x7f2ae80b4920 legacy=0x7f2ae80acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:41.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.518+0000 7f2ae77fe640 1 -- 172.21.15.16:0/1898065085 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae806d290 con 0x7f2ae8145d80 2023-12-08T08:39:41.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.518+0000 7f2ae67fc640 1 -- 172.21.15.16:0/1898065085 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810788184 0 0) 0x7f2ae80b4e00 con 0x7f2ae80b4920 2023-12-08T08:39:41.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.519+0000 7f2ae77fe640 1 --2- 172.21.15.16:0/1898065085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae8145d80 0x7f2ae8071ae0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80097a0 tx=0x7f2ad802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1896c316c1698486 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:41.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.519+0000 7f2ae67fc640 1 -- 172.21.15.16:0/1898065085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8030610 con 0x7f2ae8145d80 2023-12-08T08:39:41.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.519+0000 7f2ae67fc640 1 -- 172.21.15.16:0/1898065085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad8030770 con 0x7f2ae8145d80 2023-12-08T08:39:41.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.519+0000 7f2ae67fc640 1 -- 172.21.15.16:0/1898065085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8030b00 con 0x7f2ae8145d80 2023-12-08T08:39:41.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.520+0000 7f2aedad4640 1 -- 172.21.15.16:0/1898065085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae8145d80 msgr2=0x7f2ae8071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:41.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.520+0000 7f2aedad4640 1 --2- 172.21.15.16:0/1898065085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae8145d80 0x7f2ae8071ae0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80097a0 tx=0x7f2ad802b960 comp rx=0 tx=0).stop 2023-12-08T08:39:41.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.520+0000 7f2aedad4640 1 -- 172.21.15.16:0/1898065085 shutdown_connections 2023-12-08T08:39:41.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.520+0000 7f2aedad4640 1 --2- 172.21.15.16:0/1898065085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae8145d80 0x7f2ae8071ae0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:41.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.520+0000 7f2aedad4640 1 -- 172.21.15.16:0/1898065085 >> 172.21.15.16:0/1898065085 conn(0x7f2ae806b7c0 msgr2=0x7f2ae806bbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:41.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.520+0000 7f2aedad4640 1 -- 172.21.15.16:0/1898065085 shutdown_connections 2023-12-08T08:39:41.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.520+0000 7f2aedad4640 1 -- 172.21.15.16:0/1898065085 wait complete. 2023-12-08T08:39:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.521+0000 7f2aedad4640 1 Processor -- start 2023-12-08T08:39:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.521+0000 7f2aedad4640 1 -- start start 2023-12-08T08:39:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2aedad4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae806a4d0 0x7f2ae815d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2aedad4640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ae80b4920 0x7f2ae80cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2aedad4640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ae8145d80 0x7f2ae815ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2aedad4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae815e9f0 con 0x7f2ae806a4d0 2023-12-08T08:39:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2aedad4640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae8140c20 con 0x7f2ae8145d80 2023-12-08T08:39:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2aedad4640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae815dd70 con 0x7f2ae80b4920 2023-12-08T08:39:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2ae77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae806a4d0 0x7f2ae815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2ae77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae806a4d0 0x7f2ae815d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34640/0 (socket says 172.21.15.16:34640) 2023-12-08T08:39:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2ae77fe640 1 -- 172.21.15.16:0/2080995448 learned_addr learned my addr 172.21.15.16:0/2080995448 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2ae7fff640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ae8145d80 0x7f2ae815ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2ae6ffd640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ae80b4920 0x7f2ae80cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2ae77fe640 1 -- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ae80b4920 msgr2=0x7f2ae80cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2ae77fe640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ae80b4920 0x7f2ae80cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2ae77fe640 1 -- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ae8145d80 msgr2=0x7f2ae815ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2ae77fe640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ae8145d80 0x7f2ae815ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2ae77fe640 1 -- 172.21.15.16:0/2080995448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad8009400 con 0x7f2ae806a4d0 2023-12-08T08:39:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2ae7fff640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ae8145d80 0x7f2ae815ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.522+0000 7f2ae77fe640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae806a4d0 0x7f2ae815d790 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80098c0 tx=0x7f2ad80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.523+0000 7f2acffff640 1 -- 172.21.15.16:0/2080995448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad803e070 con 0x7f2ae806a4d0 2023-12-08T08:39:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.523+0000 7f2acffff640 1 -- 172.21.15.16:0/2080995448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad8039a10 con 0x7f2ae806a4d0 2023-12-08T08:39:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.523+0000 7f2aedad4640 1 -- 172.21.15.16:0/2080995448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae815e050 con 0x7f2ae806a4d0 2023-12-08T08:39:41.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.523+0000 7f2acffff640 1 -- 172.21.15.16:0/2080995448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8039da0 con 0x7f2ae806a4d0 2023-12-08T08:39:41.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.523+0000 7f2aedad4640 1 -- 172.21.15.16:0/2080995448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae815e610 con 0x7f2ae806a4d0 2023-12-08T08:39:41.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.524+0000 7f2aedad4640 1 -- 172.21.15.16:0/2080995448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae8071460 con 0x7f2ae806a4d0 2023-12-08T08:39:41.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.524+0000 7f2acffff640 1 -- 172.21.15.16:0/2080995448 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ad804b400 con 0x7f2ae806a4d0 2023-12-08T08:39:41.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.525+0000 7f2acffff640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2ad406ae40 0x7f2ad406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:41.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.525+0000 7f2acffff640 1 -- 172.21.15.16:0/2080995448 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f2ad80abbc0 con 0x7f2ae806a4d0 2023-12-08T08:39:41.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.525+0000 7f2ae6ffd640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2ad406ae40 0x7f2ad406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:41.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.525+0000 7f2ae6ffd640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2ad406ae40 0x7f2ad406d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8141270 tx=0x7f2adc007b90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.529+0000 7f2acffff640 1 -- 172.21.15.16:0/2080995448 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ad8076c20 con 0x7f2ae806a4d0 2023-12-08T08:39:41.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:41.742+0000 7f2aedad4640 1 -- 172.21.15.16:0/2080995448 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2ae8067430 con 0x7f2ae806a4d0 2023-12-08T08:39:42.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.372+0000 7f2acffff640 1 -- 172.21.15.16:0/2080995448 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2ad8034020 con 0x7f2ae806a4d0 2023-12-08T08:39:42.374 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 5 and data pool 6 2023-12-08T08:39:42.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.376+0000 7f2aedad4640 1 -- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2ad406ae40 msgr2=0x7f2ad406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:42.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.376+0000 7f2aedad4640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2ad406ae40 0x7f2ad406d300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8141270 tx=0x7f2adc007b90 comp rx=0 tx=0).stop 2023-12-08T08:39:42.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.376+0000 7f2aedad4640 1 -- 172.21.15.16:0/2080995448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae806a4d0 msgr2=0x7f2ae815d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:42.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.376+0000 7f2aedad4640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae806a4d0 0x7f2ae815d790 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2ad80098c0 tx=0x7f2ad80046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.376+0000 7f2aedad4640 1 -- 172.21.15.16:0/2080995448 shutdown_connections 2023-12-08T08:39:42.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.376+0000 7f2aedad4640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2ad406ae40 0x7f2ad406d300 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.376+0000 7f2aedad4640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2ae8145d80 0x7f2ae815ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.376+0000 7f2aedad4640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2ae80b4920 0x7f2ae80cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.376+0000 7f2aedad4640 1 --2- 172.21.15.16:0/2080995448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ae806a4d0 0x7f2ae815d790 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.376+0000 7f2aedad4640 1 -- 172.21.15.16:0/2080995448 >> 172.21.15.16:0/2080995448 conn(0x7f2ae806b7c0 msgr2=0x7f2ae806bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:42.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.377+0000 7f2aedad4640 1 -- 172.21.15.16:0/2080995448 shutdown_connections 2023-12-08T08:39:42.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.377+0000 7f2aedad4640 1 -- 172.21.15.16:0/2080995448 wait complete. 2023-12-08T08:39:42.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:42.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.517+0000 7fc118e31640 1 Processor -- start 2023-12-08T08:39:42.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.517+0000 7fc118e31640 1 -- start start 2023-12-08T08:39:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc118e31640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1140cbd10 0x7fc114070e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc118e31640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1140720d0 con 0x7fc11415b750 2023-12-08T08:39:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc118e31640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1140cc890 con 0x7fc114158ee0 2023-12-08T08:39:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc118e31640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1140714c0 con 0x7fc1140cbd10 2023-12-08T08:39:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc112d9d640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc11415b750 0x7fc11406c480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51532/0 (socket says 172.21.15.16:51532) 2023-12-08T08:39:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc112d9d640 1 -- 172.21.15.16:0/2621136788 learned_addr learned my addr 172.21.15.16:0/2621136788 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc10bfff640 1 --2- 172.21.15.16:0/2621136788 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1140cbd10 0x7fc114070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc111d9b640 1 -- 172.21.15.16:0/2621136788 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578522517 0 0) 0x7fc1140720d0 con 0x7fc11415b750 2023-12-08T08:39:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc111d9b640 1 -- 172.21.15.16:0/2621136788 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc100003520 con 0x7fc11415b750 2023-12-08T08:39:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc111d9b640 1 -- 172.21.15.16:0/2621136788 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1128719827 0 0) 0x7fc100003520 con 0x7fc11415b750 2023-12-08T08:39:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc111d9b640 1 -- 172.21.15.16:0/2621136788 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1140cbd10 msgr2=0x7fc114070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:42.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc111d9b640 1 --2- 172.21.15.16:0/2621136788 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1140cbd10 0x7fc114070e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc111d9b640 1 -- 172.21.15.16:0/2621136788 >> v1:172.21.15.112:6789/0 conn(0x7fc114158ee0 legacy=0x7fc1140cb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc111d9b640 1 -- 172.21.15.16:0/2621136788 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1140717a0 con 0x7fc11415b750 2023-12-08T08:39:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.518+0000 7fc111d9b640 1 -- 172.21.15.16:0/2621136788 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7fc10c0019f0 con 0x7fc11415b750 2023-12-08T08:39:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc111d9b640 1 -- 172.21.15.16:0/2621136788 >> v1:172.21.15.16:6789/0 conn(0x7fc11415b750 legacy=0x7fc11406c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc111d9b640 1 --2- 172.21.15.16:0/2621136788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1000039a0 0x7fc100003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc111d9b640 1 --2- 172.21.15.16:0/2621136788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1000047c0 0x7fc100004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc111d9b640 1 --2- 172.21.15.16:0/2621136788 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1140cbd10 0x7fc1000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc111d9b640 1 -- 172.21.15.16:0/2621136788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1140714c0 con 0x7fc1000039a0 2023-12-08T08:39:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc111d9b640 1 -- 172.21.15.16:0/2621136788 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10000bab0 con 0x7fc1000047c0 2023-12-08T08:39:42.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc111d9b640 1 -- 172.21.15.16:0/2621136788 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc10000bc70 con 0x7fc1140cbd10 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc11259c640 1 --2- 172.21.15.16:0/2621136788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1000039a0 0x7fc100003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc10bfff640 1 --2- 172.21.15.16:0/2621136788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1000047c0 0x7fc100004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc112d9d640 1 --2- 172.21.15.16:0/2621136788 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1140cbd10 0x7fc1000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc11259c640 1 -- 172.21.15.16:0/2621136788 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1140cbd10 msgr2=0x7fc1000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc11259c640 1 --2- 172.21.15.16:0/2621136788 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc1140cbd10 0x7fc1000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc11259c640 1 -- 172.21.15.16:0/2621136788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1000047c0 msgr2=0x7fc100004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc11259c640 1 --2- 172.21.15.16:0/2621136788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1000047c0 0x7fc100004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc11259c640 1 -- 172.21.15.16:0/2621136788 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc100003520 con 0x7fc1000039a0 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.519+0000 7fc10bfff640 1 --2- 172.21.15.16:0/2621136788 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1000047c0 0x7fc100004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.520+0000 7fc11259c640 1 --2- 172.21.15.16:0/2621136788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1000039a0 0x7fc100003dc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc104001750 tx=0x7fc1000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388d13c009e3b2ce server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.520+0000 7fc111d9b640 1 -- 172.21.15.16:0/2621136788 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1040052d0 con 0x7fc1000039a0 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.520+0000 7fc111d9b640 1 -- 172.21.15.16:0/2621136788 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10402e050 con 0x7fc1000039a0 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.520+0000 7fc111d9b640 1 -- 172.21.15.16:0/2621136788 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1040035f0 con 0x7fc1000039a0 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.520+0000 7fc118e31640 1 -- 172.21.15.16:0/2621136788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1000039a0 msgr2=0x7fc100003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:42.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.520+0000 7fc118e31640 1 --2- 172.21.15.16:0/2621136788 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1000039a0 0x7fc100003dc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc104001750 tx=0x7fc1000057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.520+0000 7fc112d9d640 1 -- 172.21.15.16:0/2621136788 reap_dead start 2023-12-08T08:39:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.520+0000 7fc118e31640 1 -- 172.21.15.16:0/2621136788 shutdown_connections 2023-12-08T08:39:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.520+0000 7fc118e31640 1 -- 172.21.15.16:0/2621136788 >> 172.21.15.16:0/2621136788 conn(0x7fc1140b09f0 msgr2=0x7fc1140b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.520+0000 7fc118e31640 1 -- 172.21.15.16:0/2621136788 shutdown_connections 2023-12-08T08:39:42.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.520+0000 7fc118e31640 1 -- 172.21.15.16:0/2621136788 wait complete. 2023-12-08T08:39:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.522+0000 7fc118e31640 1 Processor -- start 2023-12-08T08:39:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.522+0000 7fc118e31640 1 -- start start 2023-12-08T08:39:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.522+0000 7fc118e31640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1000039a0 0x7fc11414d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.522+0000 7fc118e31640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11409c1a0 0x7fc11415d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.522+0000 7fc118e31640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc11409c620 0x7fc11415d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.522+0000 7fc118e31640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11414f830 con 0x7fc11409c1a0 2023-12-08T08:39:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.522+0000 7fc118e31640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11414df20 con 0x7fc1000039a0 2023-12-08T08:39:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.522+0000 7fc118e31640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc114150320 con 0x7fc11409c620 2023-12-08T08:39:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc10bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11409c1a0 0x7fc11415d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc10bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11409c1a0 0x7fc11415d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34656/0 (socket says 172.21.15.16:34656) 2023-12-08T08:39:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc10bfff640 1 -- 172.21.15.16:0/3072314889 learned_addr learned my addr 172.21.15.16:0/3072314889 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:42.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc11259c640 1 --2- 172.21.15.16:0/3072314889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1000039a0 0x7fc11414d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc112d9d640 1 --2- 172.21.15.16:0/3072314889 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc11409c620 0x7fc11415d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc10bfff640 1 -- 172.21.15.16:0/3072314889 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc11409c620 msgr2=0x7fc11415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc10bfff640 1 --2- 172.21.15.16:0/3072314889 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc11409c620 0x7fc11415d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc10bfff640 1 -- 172.21.15.16:0/3072314889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1000039a0 msgr2=0x7fc11414d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc10bfff640 1 --2- 172.21.15.16:0/3072314889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1000039a0 0x7fc11414d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc10bfff640 1 -- 172.21.15.16:0/3072314889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11414fa80 con 0x7fc11409c1a0 2023-12-08T08:39:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc10bfff640 1 --2- 172.21.15.16:0/3072314889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11409c1a0 0x7fc11415d110 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0116d0 tx=0x7fc0fc011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc10affd640 1 -- 172.21.15.16:0/3072314889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc00f9c0 con 0x7fc11409c1a0 2023-12-08T08:39:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc10affd640 1 -- 172.21.15.16:0/3072314889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc00fb20 con 0x7fc11409c1a0 2023-12-08T08:39:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc118e31640 1 -- 172.21.15.16:0/3072314889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11414fd60 con 0x7fc11409c1a0 2023-12-08T08:39:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc10affd640 1 -- 172.21.15.16:0/3072314889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc00fed0 con 0x7fc11409c1a0 2023-12-08T08:39:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.523+0000 7fc118e31640 1 -- 172.21.15.16:0/3072314889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11409bc80 con 0x7fc11409c1a0 2023-12-08T08:39:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.524+0000 7fc118e31640 1 -- 172.21.15.16:0/3072314889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11409bae0 con 0x7fc11409c1a0 2023-12-08T08:39:42.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.525+0000 7fc10affd640 1 -- 172.21.15.16:0/3072314889 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc0fc015460 con 0x7fc11409c1a0 2023-12-08T08:39:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.525+0000 7fc10affd640 1 --2- 172.21.15.16:0/3072314889 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc0f406ae40 0x7fc0f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.525+0000 7fc10affd640 1 -- 172.21.15.16:0/3072314889 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc0fc089f30 con 0x7fc11409c1a0 2023-12-08T08:39:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.525+0000 7fc11259c640 1 --2- 172.21.15.16:0/3072314889 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc0f406ae40 0x7fc0f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.526+0000 7fc11259c640 1 --2- 172.21.15.16:0/3072314889 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc0f406ae40 0x7fc0f406d300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc104033000 tx=0x7fc104009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:42.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.529+0000 7fc10affd640 1 -- 172.21.15.16:0/3072314889 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc11409bae0 con 0x7fc11409c1a0 2023-12-08T08:39:42.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.706+0000 7fc118e31640 1 -- 172.21.15.16:0/3072314889 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc11409bae0 con 0x7fc11409c1a0 2023-12-08T08:39:42.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.708+0000 7fc10affd640 1 -- 172.21.15.16:0/3072314889 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16787 (secure 0 0 0) 0x7fc11409bae0 con 0x7fc11409c1a0 2023-12-08T08:39:42.710 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:42.710 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":38,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:41.743005+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:39:39.593361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-12-08T08:39:40.726565+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:42.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.709+0000 7fc118e31640 1 -- 172.21.15.16:0/3072314889 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc0f406ae40 msgr2=0x7fc0f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:42.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.709+0000 7fc118e31640 1 --2- 172.21.15.16:0/3072314889 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc0f406ae40 0x7fc0f406d300 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc104033000 tx=0x7fc104009030 comp rx=0 tx=0).stop 2023-12-08T08:39:42.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.709+0000 7fc118e31640 1 -- 172.21.15.16:0/3072314889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11409c1a0 msgr2=0x7fc11415d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:42.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.709+0000 7fc118e31640 1 --2- 172.21.15.16:0/3072314889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11409c1a0 0x7fc11415d110 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0116d0 tx=0x7fc0fc011b90 comp rx=0 tx=0).stop 2023-12-08T08:39:42.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.709+0000 7fc118e31640 1 -- 172.21.15.16:0/3072314889 shutdown_connections 2023-12-08T08:39:42.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.709+0000 7fc118e31640 1 --2- 172.21.15.16:0/3072314889 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fc0f406ae40 0x7fc0f406d300 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.709+0000 7fc118e31640 1 --2- 172.21.15.16:0/3072314889 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fc11409c620 0x7fc11415d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.709+0000 7fc118e31640 1 --2- 172.21.15.16:0/3072314889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11409c1a0 0x7fc11415d110 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.709+0000 7fc118e31640 1 --2- 172.21.15.16:0/3072314889 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fc1000039a0 0x7fc11414d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.709+0000 7fc118e31640 1 -- 172.21.15.16:0/3072314889 >> 172.21.15.16:0/3072314889 conn(0x7fc1140b09f0 msgr2=0x7fc1140acbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:42.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.710+0000 7fc118e31640 1 -- 172.21.15.16:0/3072314889 shutdown_connections 2023-12-08T08:39:42.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.710+0000 7fc118e31640 1 -- 172.21.15.16:0/3072314889 wait complete. 2023-12-08T08:39:42.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:42.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.827+0000 7fbc5a761640 1 Processor -- start 2023-12-08T08:39:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.827+0000 7fbc5a761640 1 -- start start 2023-12-08T08:39:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.828+0000 7fbc5a761640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbc541608a0 0x7fbc540d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.828+0000 7fbc5a761640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fbc540b0cf0 0x7fbc540cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.828+0000 7fbc5a761640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc540d20b0 con 0x7fbc540b1d70 2023-12-08T08:39:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.828+0000 7fbc5a761640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc540d0620 con 0x7fbc541608a0 2023-12-08T08:39:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.828+0000 7fbc5a761640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc540d1b80 con 0x7fbc540b0cf0 2023-12-08T08:39:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.828+0000 7fbc53fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbc541608a0 0x7fbc540d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.828+0000 7fbc537fe640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbc540b1d70 0x7fbc540d1470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51534/0 (socket says 172.21.15.16:51534) 2023-12-08T08:39:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.828+0000 7fbc537fe640 1 -- 172.21.15.16:0/281306096 learned_addr learned my addr 172.21.15.16:0/281306096 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.828+0000 7fbc53fff640 1 --2- 172.21.15.16:0/281306096 >> v2:172.21.15.16:3300/0 conn(0x7fbc541608a0 0x7fbc540d0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34658/0 (socket says 172.21.15.16:34658) 2023-12-08T08:39:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.829+0000 7fbc58cfe640 1 --2- 172.21.15.16:0/281306096 >> v2:172.21.15.112:3300/0 conn(0x7fbc540b0cf0 0x7fbc540cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.829+0000 7fbc53fff640 1 -- 172.21.15.16:0/281306096 >> v2:172.21.15.112:3300/0 conn(0x7fbc540b0cf0 msgr2=0x7fbc540cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.829+0000 7fbc53fff640 1 --2- 172.21.15.16:0/281306096 >> v2:172.21.15.112:3300/0 conn(0x7fbc540b0cf0 0x7fbc540cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.829+0000 7fbc53fff640 1 -- 172.21.15.16:0/281306096 >> v1:172.21.15.16:6789/0 conn(0x7fbc540b1d70 legacy=0x7fbc540d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.829+0000 7fbc53fff640 1 -- 172.21.15.16:0/281306096 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc540d1dd0 con 0x7fbc541608a0 2023-12-08T08:39:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.829+0000 7fbc58cfe640 1 --2- 172.21.15.16:0/281306096 >> v2:172.21.15.112:3300/0 conn(0x7fbc540b0cf0 0x7fbc540cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.829+0000 7fbc53fff640 1 --2- 172.21.15.16:0/281306096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc541608a0 0x7fbc540d0f30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbc54073100 tx=0x7fbc4802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb4eb9e5edff225 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.830+0000 7fbc52ffd640 1 -- 172.21.15.16:0/281306096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4802e070 con 0x7fbc541608a0 2023-12-08T08:39:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.830+0000 7fbc52ffd640 1 -- 172.21.15.16:0/281306096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc480315a0 con 0x7fbc541608a0 2023-12-08T08:39:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.830+0000 7fbc52ffd640 1 -- 172.21.15.16:0/281306096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc48031930 con 0x7fbc541608a0 2023-12-08T08:39:42.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.830+0000 7fbc5a761640 1 -- 172.21.15.16:0/281306096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc541608a0 msgr2=0x7fbc540d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:42.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.830+0000 7fbc5a761640 1 --2- 172.21.15.16:0/281306096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc541608a0 0x7fbc540d0f30 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fbc54073100 tx=0x7fbc4802b840 comp rx=0 tx=0).stop 2023-12-08T08:39:42.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.830+0000 7fbc5a761640 1 -- 172.21.15.16:0/281306096 shutdown_connections 2023-12-08T08:39:42.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.830+0000 7fbc5a761640 1 --2- 172.21.15.16:0/281306096 >> v2:172.21.15.112:3300/0 conn(0x7fbc540b0cf0 0x7fbc540cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.830+0000 7fbc5a761640 1 --2- 172.21.15.16:0/281306096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc541608a0 0x7fbc540d0f30 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.830+0000 7fbc5a761640 1 -- 172.21.15.16:0/281306096 >> 172.21.15.16:0/281306096 conn(0x7fbc540d3030 msgr2=0x7fbc540d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:42.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.830+0000 7fbc5a761640 1 -- 172.21.15.16:0/281306096 shutdown_connections 2023-12-08T08:39:42.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.830+0000 7fbc5a761640 1 -- 172.21.15.16:0/281306096 wait complete. 2023-12-08T08:39:42.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.831+0000 7fbc5a761640 1 Processor -- start 2023-12-08T08:39:42.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.832+0000 7fbc5a761640 1 -- start start 2023-12-08T08:39:42.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.832+0000 7fbc5a761640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc540b0cf0 0x7fbc540ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:42.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.832+0000 7fbc5a761640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc540b1d70 0x7fbc5414c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:42.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.832+0000 7fbc5a761640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc541608a0 0x7fbc5414cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:42.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.832+0000 7fbc5a761640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5414d150 con 0x7fbc541608a0 2023-12-08T08:39:42.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.832+0000 7fbc5a761640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540ca790 con 0x7fbc540b0cf0 2023-12-08T08:39:42.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.832+0000 7fbc5a761640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc540b6d10 con 0x7fbc540b1d70 2023-12-08T08:39:42.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.832+0000 7fbc58cfe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc541608a0 0x7fbc5414cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:42.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.832+0000 7fbc58cfe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc541608a0 0x7fbc5414cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34668/0 (socket says 172.21.15.16:34668) 2023-12-08T08:39:42.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.832+0000 7fbc58cfe640 1 -- 172.21.15.16:0/1032161742 learned_addr learned my addr 172.21.15.16:0/1032161742 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:42.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.833+0000 7fbc537fe640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc540b1d70 0x7fbc5414c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.833+0000 7fbc53fff640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc540b0cf0 0x7fbc540ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.833+0000 7fbc58cfe640 1 -- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc540b1d70 msgr2=0x7fbc5414c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.833+0000 7fbc58cfe640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc540b1d70 0x7fbc5414c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.833+0000 7fbc58cfe640 1 -- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc540b0cf0 msgr2=0x7fbc540ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.833+0000 7fbc58cfe640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc540b0cf0 0x7fbc540ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.833+0000 7fbc58cfe640 1 -- 172.21.15.16:0/1032161742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc48009400 con 0x7fbc541608a0 2023-12-08T08:39:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.833+0000 7fbc537fe640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc540b1d70 0x7fbc5414c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.833+0000 7fbc58cfe640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc541608a0 0x7fbc5414cc10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00cea0 tx=0x7fbc4c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.833+0000 7fbc50ff9640 1 -- 172.21.15.16:0/1032161742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c003b90 con 0x7fbc541608a0 2023-12-08T08:39:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.833+0000 7fbc50ff9640 1 -- 172.21.15.16:0/1032161742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c003d10 con 0x7fbc541608a0 2023-12-08T08:39:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.833+0000 7fbc5a761640 1 -- 172.21.15.16:0/1032161742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc540c6f80 con 0x7fbc541608a0 2023-12-08T08:39:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.833+0000 7fbc50ff9640 1 -- 172.21.15.16:0/1032161742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4c004260 con 0x7fbc541608a0 2023-12-08T08:39:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.833+0000 7fbc5a761640 1 -- 172.21.15.16:0/1032161742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc540c7540 con 0x7fbc541608a0 2023-12-08T08:39:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.834+0000 7fbc5a761640 1 -- 172.21.15.16:0/1032161742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc540b6ed0 con 0x7fbc541608a0 2023-12-08T08:39:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.834+0000 7fbc50ff9640 1 -- 172.21.15.16:0/1032161742 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc4c0043c0 con 0x7fbc541608a0 2023-12-08T08:39:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.835+0000 7fbc50ff9640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbc4006ae40 0x7fbc4006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.835+0000 7fbc50ff9640 1 -- 172.21.15.16:0/1032161742 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbc4c089f90 con 0x7fbc541608a0 2023-12-08T08:39:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.835+0000 7fbc53fff640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbc4006ae40 0x7fbc4006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.836+0000 7fbc53fff640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbc4006ae40 0x7fbc4006d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbc48036000 tx=0x7fbc4803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:42.839+0000 7fbc50ff9640 1 -- 172.21.15.16:0/1032161742 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc4c0bce90 con 0x7fbc541608a0 2023-12-08T08:39:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.019+0000 7fbc5a761640 1 -- 172.21.15.16:0/1032161742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc54067430 con 0x7fbc541608a0 2023-12-08T08:39:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.020+0000 7fbc50ff9640 1 -- 172.21.15.16:0/1032161742 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16787 (secure 0 0 0) 0x7fbc4c054f90 con 0x7fbc541608a0 2023-12-08T08:39:43.022 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:43.023 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":38,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:41.743005+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:39:39.593361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-12-08T08:39:40.726565+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.022+0000 7fbc5a761640 1 -- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbc4006ae40 msgr2=0x7fbc4006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.022+0000 7fbc5a761640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbc4006ae40 0x7fbc4006d300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbc48036000 tx=0x7fbc4803d040 comp rx=0 tx=0).stop 2023-12-08T08:39:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.022+0000 7fbc5a761640 1 -- 172.21.15.16:0/1032161742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc541608a0 msgr2=0x7fbc5414cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.022+0000 7fbc5a761640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc541608a0 0x7fbc5414cc10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c00cea0 tx=0x7fbc4c00cf90 comp rx=0 tx=0).stop 2023-12-08T08:39:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.022+0000 7fbc5a761640 1 -- 172.21.15.16:0/1032161742 shutdown_connections 2023-12-08T08:39:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.022+0000 7fbc5a761640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbc4006ae40 0x7fbc4006d300 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.022+0000 7fbc5a761640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc541608a0 0x7fbc5414cc10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.022+0000 7fbc5a761640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbc540b1d70 0x7fbc5414c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.022+0000 7fbc5a761640 1 --2- 172.21.15.16:0/1032161742 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbc540b0cf0 0x7fbc540ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.022+0000 7fbc5a761640 1 -- 172.21.15.16:0/1032161742 >> 172.21.15.16:0/1032161742 conn(0x7fbc540d3030 msgr2=0x7fbc540d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.023+0000 7fbc5a761640 1 -- 172.21.15.16:0/1032161742 shutdown_connections 2023-12-08T08:39:43.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.023+0000 7fbc5a761640 1 -- 172.21.15.16:0/1032161742 wait complete. 2023-12-08T08:39:43.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-08T08:39:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.143+0000 7fdcef178640 1 Processor -- start 2023-12-08T08:39:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.143+0000 7fdcef178640 1 -- start start 2023-12-08T08:39:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.144+0000 7fdcef178640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdce80d3b80 0x7fdce80ce690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.144+0000 7fdcef178640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce80b5ec0 con 0x7fdce80cfa80 2023-12-08T08:39:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.144+0000 7fdcef178640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce80d3fe0 con 0x7fdce80b6bf0 2023-12-08T08:39:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.144+0000 7fdcef178640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce80cebd0 con 0x7fdce80d3b80 2023-12-08T08:39:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.144+0000 7fdcecf14640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdce80cfa80 0x7fdce80cfe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51546/0 (socket says 172.21.15.16:51546) 2023-12-08T08:39:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.144+0000 7fdcecf14640 1 -- 172.21.15.16:0/3773304176 learned_addr learned my addr 172.21.15.16:0/3773304176 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.144+0000 7fdce7fff640 1 --2- 172.21.15.16:0/3773304176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdce80d3b80 0x7fdce80ce690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.144+0000 7fdce77fe640 1 -- 172.21.15.16:0/3773304176 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437842498 0 0) 0x7fdce80b5ec0 con 0x7fdce80cfa80 2023-12-08T08:39:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.144+0000 7fdce77fe640 1 -- 172.21.15.16:0/3773304176 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd0003520 con 0x7fdce80cfa80 2023-12-08T08:39:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdce77fe640 1 -- 172.21.15.16:0/3773304176 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 148680709 0 0) 0x7fdcd0003520 con 0x7fdce80cfa80 2023-12-08T08:39:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdce77fe640 1 -- 172.21.15.16:0/3773304176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdce80d3b80 msgr2=0x7fdce80ce690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdce77fe640 1 --2- 172.21.15.16:0/3773304176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdce80d3b80 0x7fdce80ce690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdce77fe640 1 -- 172.21.15.16:0/3773304176 >> v1:172.21.15.112:6789/0 conn(0x7fdce80b6bf0 legacy=0x7fdce80b7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdce77fe640 1 -- 172.21.15.16:0/3773304176 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce80cede0 con 0x7fdce80cfa80 2023-12-08T08:39:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdce77fe640 1 -- 172.21.15.16:0/3773304176 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7fdcd8001670 con 0x7fdce80cfa80 2023-12-08T08:39:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdce77fe640 1 -- 172.21.15.16:0/3773304176 >> v1:172.21.15.16:6789/0 conn(0x7fdce80cfa80 legacy=0x7fdce80cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdce77fe640 1 --2- 172.21.15.16:0/3773304176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd00039a0 0x7fdcd0003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdce77fe640 1 --2- 172.21.15.16:0/3773304176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdce80d3b80 0x7fdcd00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdce77fe640 1 --2- 172.21.15.16:0/3773304176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcd0004d00 0x7fdcd0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdce77fe640 1 -- 172.21.15.16:0/3773304176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce80cebd0 con 0x7fdcd00039a0 2023-12-08T08:39:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdce77fe640 1 -- 172.21.15.16:0/3773304176 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd000ba60 con 0x7fdcd0004d00 2023-12-08T08:39:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdce77fe640 1 -- 172.21.15.16:0/3773304176 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd000bc20 con 0x7fdce80d3b80 2023-12-08T08:39:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdcecf14640 1 --2- 172.21.15.16:0/3773304176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd00039a0 0x7fdcd0003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdced715640 1 --2- 172.21.15.16:0/3773304176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcd0004d00 0x7fdcd0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdce7fff640 1 --2- 172.21.15.16:0/3773304176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdce80d3b80 0x7fdcd00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdcecf14640 1 -- 172.21.15.16:0/3773304176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdce80d3b80 msgr2=0x7fdcd00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdcecf14640 1 --2- 172.21.15.16:0/3773304176 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdce80d3b80 0x7fdcd00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdcecf14640 1 -- 172.21.15.16:0/3773304176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcd0004d00 msgr2=0x7fdcd0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdcecf14640 1 --2- 172.21.15.16:0/3773304176 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdcd0004d00 0x7fdcd0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.145+0000 7fdcecf14640 1 -- 172.21.15.16:0/3773304176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd0003520 con 0x7fdcd00039a0 2023-12-08T08:39:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.146+0000 7fdcecf14640 1 --2- 172.21.15.16:0/3773304176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd00039a0 0x7fdcd0003d80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdcd800c190 tx=0x7fdcd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2cec3c0de26da0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.146+0000 7fdce77fe640 1 -- 172.21.15.16:0/3773304176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8005ad0 con 0x7fdcd00039a0 2023-12-08T08:39:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.146+0000 7fdce77fe640 1 -- 172.21.15.16:0/3773304176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd8005c30 con 0x7fdcd00039a0 2023-12-08T08:39:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.146+0000 7fdce77fe640 1 -- 172.21.15.16:0/3773304176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd802c810 con 0x7fdcd00039a0 2023-12-08T08:39:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.146+0000 7fdcef178640 1 -- 172.21.15.16:0/3773304176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd00039a0 msgr2=0x7fdcd0003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.146+0000 7fdcef178640 1 --2- 172.21.15.16:0/3773304176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd00039a0 0x7fdcd0003d80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdcd800c190 tx=0x7fdcd00057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.146+0000 7fdced715640 1 -- 172.21.15.16:0/3773304176 reap_dead start 2023-12-08T08:39:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.146+0000 7fdcef178640 1 -- 172.21.15.16:0/3773304176 shutdown_connections 2023-12-08T08:39:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.146+0000 7fdcef178640 1 -- 172.21.15.16:0/3773304176 >> 172.21.15.16:0/3773304176 conn(0x7fdce809c340 msgr2=0x7fdce80d3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.147+0000 7fdcef178640 1 -- 172.21.15.16:0/3773304176 shutdown_connections 2023-12-08T08:39:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.147+0000 7fdcef178640 1 -- 172.21.15.16:0/3773304176 wait complete. 2023-12-08T08:39:43.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.148+0000 7fdcef178640 1 Processor -- start 2023-12-08T08:39:43.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.148+0000 7fdcef178640 1 -- start start 2023-12-08T08:39:43.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcef178640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd00039a0 0x7fdce81f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:43.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcef178640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdce80c8330 0x7fdce81f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:43.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcef178640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce815de10 0x7fdce806c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:43.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcef178640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce80c6b60 con 0x7fdcd00039a0 2023-12-08T08:39:43.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcef178640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce81f33d0 con 0x7fdce815de10 2023-12-08T08:39:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcef178640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce81f36b0 con 0x7fdce80c8330 2023-12-08T08:39:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcecf14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd00039a0 0x7fdce81f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcecf14640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd00039a0 0x7fdce81f2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34690/0 (socket says 172.21.15.16:34690) 2023-12-08T08:39:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcecf14640 1 -- 172.21.15.16:0/2866215976 learned_addr learned my addr 172.21.15.16:0/2866215976 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdced715640 1 --2- 172.21.15.16:0/2866215976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce815de10 0x7fdce806c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdce7fff640 1 --2- 172.21.15.16:0/2866215976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdce80c8330 0x7fdce81f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcecf14640 1 -- 172.21.15.16:0/2866215976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdce80c8330 msgr2=0x7fdce81f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcecf14640 1 --2- 172.21.15.16:0/2866215976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdce80c8330 0x7fdce81f01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcecf14640 1 -- 172.21.15.16:0/2866215976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce815de10 msgr2=0x7fdce806c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcecf14640 1 --2- 172.21.15.16:0/2866215976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce815de10 0x7fdce806c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcecf14640 1 -- 172.21.15.16:0/2866215976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce81f3990 con 0x7fdcd00039a0 2023-12-08T08:39:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcecf14640 1 --2- 172.21.15.16:0/2866215976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd00039a0 0x7fdce81f2dc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdcd802c3b0 tx=0x7fdcd8001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdce57fa640 1 -- 172.21.15.16:0/2866215976 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8039070 con 0x7fdcd00039a0 2023-12-08T08:39:43.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdce57fa640 1 -- 172.21.15.16:0/2866215976 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd802cc40 con 0x7fdcd00039a0 2023-12-08T08:39:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcef178640 1 -- 172.21.15.16:0/2866215976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce81f3c70 con 0x7fdcd00039a0 2023-12-08T08:39:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdce57fa640 1 -- 172.21.15.16:0/2866215976 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8007070 con 0x7fdcd00039a0 2023-12-08T08:39:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.149+0000 7fdcef178640 1 -- 172.21.15.16:0/2866215976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce80c87b0 con 0x7fdcd00039a0 2023-12-08T08:39:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.150+0000 7fdcef178640 1 -- 172.21.15.16:0/2866215976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce80d1fc0 con 0x7fdcd00039a0 2023-12-08T08:39:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.151+0000 7fdce57fa640 1 -- 172.21.15.16:0/2866215976 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdcd8032070 con 0x7fdcd00039a0 2023-12-08T08:39:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.151+0000 7fdce57fa640 1 --2- 172.21.15.16:0/2866215976 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdcd406ae70 0x7fdcd406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.151+0000 7fdce57fa640 1 -- 172.21.15.16:0/2866215976 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdcd803c040 con 0x7fdcd00039a0 2023-12-08T08:39:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.152+0000 7fdce7fff640 1 --2- 172.21.15.16:0/2866215976 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdcd406ae70 0x7fdcd406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.152+0000 7fdce7fff640 1 --2- 172.21.15.16:0/2866215976 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdcd406ae70 0x7fdcd406d330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdce80a0f10 tx=0x7fdcdc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.156+0000 7fdce57fa640 1 -- 172.21.15.16:0/2866215976 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdcd8075880 con 0x7fdcd00039a0 2023-12-08T08:39:43.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.373+0000 7fdcef178640 1 -- 172.21.15.16:0/2866215976 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdce8067430 con 0x7fdcd00039a0 2023-12-08T08:39:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.389+0000 7fdce57fa640 1 -- 172.21.15.16:0/2866215976 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdcd8078ea0 con 0x7fdcd00039a0 2023-12-08T08:39:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.393+0000 7fdcef178640 1 -- 172.21.15.16:0/2866215976 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdcd406ae70 msgr2=0x7fdcd406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.393+0000 7fdcef178640 1 --2- 172.21.15.16:0/2866215976 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdcd406ae70 0x7fdcd406d330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdce80a0f10 tx=0x7fdcdc008040 comp rx=0 tx=0).stop 2023-12-08T08:39:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.393+0000 7fdcef178640 1 -- 172.21.15.16:0/2866215976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd00039a0 msgr2=0x7fdce81f2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.393+0000 7fdcef178640 1 --2- 172.21.15.16:0/2866215976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd00039a0 0x7fdce81f2dc0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdcd802c3b0 tx=0x7fdcd8001930 comp rx=0 tx=0).stop 2023-12-08T08:39:43.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.393+0000 7fdcef178640 1 -- 172.21.15.16:0/2866215976 shutdown_connections 2023-12-08T08:39:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.393+0000 7fdcef178640 1 --2- 172.21.15.16:0/2866215976 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fdcd406ae70 0x7fdcd406d330 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.393+0000 7fdcef178640 1 --2- 172.21.15.16:0/2866215976 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fdce815de10 0x7fdce806c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.393+0000 7fdcef178640 1 --2- 172.21.15.16:0/2866215976 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fdce80c8330 0x7fdce81f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.393+0000 7fdcef178640 1 --2- 172.21.15.16:0/2866215976 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcd00039a0 0x7fdce81f2dc0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.393+0000 7fdcef178640 1 -- 172.21.15.16:0/2866215976 >> 172.21.15.16:0/2866215976 conn(0x7fdce809c340 msgr2=0x7fdce806b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.393+0000 7fdcef178640 1 -- 172.21.15.16:0/2866215976 shutdown_connections 2023-12-08T08:39:43.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.393+0000 7fdcef178640 1 -- 172.21.15.16:0/2866215976 wait complete. 2023-12-08T08:39:43.406 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-08T08:39:43.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-08T08:39:43.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.514+0000 7f7311861640 1 Processor -- start 2023-12-08T08:39:43.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.514+0000 7f7311861640 1 -- start start 2023-12-08T08:39:43.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.515+0000 7f7311861640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f730c143980 0x7f730c0cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:43.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.515+0000 7f7311861640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c06d750 0x7f730c0b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:43.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.515+0000 7f7311861640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730c0b3360 con 0x7f730c0cf160 2023-12-08T08:39:43.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.515+0000 7f7311861640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f730c0cf5c0 con 0x7f730c143980 2023-12-08T08:39:43.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.515+0000 7f7311861640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730c0b2de0 con 0x7f730c06d750 2023-12-08T08:39:43.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.515+0000 7f730affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f730c143980 0x7f730c0cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:43.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.515+0000 7f730affd640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f730c143980 0x7f730c0cec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34694/0 (socket says 172.21.15.16:34694) 2023-12-08T08:39:43.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.515+0000 7f730affd640 1 -- 172.21.15.16:0/3343293037 learned_addr learned my addr 172.21.15.16:0/3343293037 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:43.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.515+0000 7f730b7fe640 1 --2- 172.21.15.16:0/3343293037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c06d750 0x7f730c0b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:43.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.515+0000 7f730affd640 1 -- 172.21.15.16:0/3343293037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c06d750 msgr2=0x7f730c0b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:43.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.515+0000 7f730affd640 1 --2- 172.21.15.16:0/3343293037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c06d750 0x7f730c0b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.515+0000 7f730affd640 1 -- 172.21.15.16:0/3343293037 >> v1:172.21.15.112:6789/0 conn(0x7f730c0cf160 legacy=0x7f730c06d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:43.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.515+0000 7f730affd640 1 -- 172.21.15.16:0/3343293037 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730c0cdfd0 con 0x7f730c143980 2023-12-08T08:39:43.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.516+0000 7f730affd640 1 --2- 172.21.15.16:0/3343293037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c143980 0x7f730c0cec20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009870 tx=0x7f72fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3bdbdf56d0ffcf7 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.516+0000 7f7309ffb640 1 -- 172.21.15.16:0/3343293037 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc031640 con 0x7f730c143980 2023-12-08T08:39:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.516+0000 7f7309ffb640 1 -- 172.21.15.16:0/3343293037 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc0317a0 con 0x7f730c143980 2023-12-08T08:39:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.516+0000 7f7309ffb640 1 -- 172.21.15.16:0/3343293037 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc031b30 con 0x7f730c143980 2023-12-08T08:39:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.517+0000 7f7311861640 1 -- 172.21.15.16:0/3343293037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c143980 msgr2=0x7f730c0cec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.517+0000 7f7311861640 1 --2- 172.21.15.16:0/3343293037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c143980 0x7f730c0cec20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009870 tx=0x7f72fc02b8f0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.517+0000 7f7311861640 1 -- 172.21.15.16:0/3343293037 shutdown_connections 2023-12-08T08:39:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.517+0000 7f7311861640 1 --2- 172.21.15.16:0/3343293037 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c06d750 0x7f730c0b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.517+0000 7f7311861640 1 --2- 172.21.15.16:0/3343293037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c143980 0x7f730c0cec20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.517+0000 7f7311861640 1 -- 172.21.15.16:0/3343293037 >> 172.21.15.16:0/3343293037 conn(0x7f730c0b1ae0 msgr2=0x7f730c0b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.517+0000 7f7311861640 1 -- 172.21.15.16:0/3343293037 shutdown_connections 2023-12-08T08:39:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.517+0000 7f7311861640 1 -- 172.21.15.16:0/3343293037 wait complete. 2023-12-08T08:39:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.518+0000 7f7311861640 1 Processor -- start 2023-12-08T08:39:43.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.518+0000 7f7311861640 1 -- start start 2023-12-08T08:39:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f7311861640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c06d750 0x7f730c159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f7311861640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c0cf160 0x7f730c1f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f7311861640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f730c143980 0x7f730c1352e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f7311861640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c0b0a50 con 0x7f730c06d750 2023-12-08T08:39:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f7311861640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c1577e0 con 0x7f730c143980 2023-12-08T08:39:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f7311861640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730c157ac0 con 0x7f730c0cf160 2023-12-08T08:39:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f730affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c06d750 0x7f730c159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f730affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c06d750 0x7f730c159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34708/0 (socket says 172.21.15.16:34708) 2023-12-08T08:39:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f730affd640 1 -- 172.21.15.16:0/4087918740 learned_addr learned my addr 172.21.15.16:0/4087918740 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f730b7fe640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f730c143980 0x7f730c1352e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f730a7fc640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c0cf160 0x7f730c1f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f730affd640 1 -- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c0cf160 msgr2=0x7f730c1f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f730affd640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c0cf160 0x7f730c1f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f730affd640 1 -- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f730c143980 msgr2=0x7f730c1352e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f730affd640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f730c143980 0x7f730c1352e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f730affd640 1 -- 172.21.15.16:0/4087918740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc0094d0 con 0x7f730c06d750 2023-12-08T08:39:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f730b7fe640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f730c143980 0x7f730c1352e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f730a7fc640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c0cf160 0x7f730c1f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f730affd640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c06d750 0x7f730c159550 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009870 tx=0x7f72fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f72f3fff640 1 -- 172.21.15.16:0/4087918740 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc03ac60 con 0x7f730c06d750 2023-12-08T08:39:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f72f3fff640 1 -- 172.21.15.16:0/4087918740 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc03ade0 con 0x7f730c06d750 2023-12-08T08:39:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.519+0000 7f7311861640 1 -- 172.21.15.16:0/4087918740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730c157da0 con 0x7f730c06d750 2023-12-08T08:39:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.520+0000 7f72f3fff640 1 -- 172.21.15.16:0/4087918740 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc044630 con 0x7f730c06d750 2023-12-08T08:39:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.520+0000 7f7311861640 1 -- 172.21.15.16:0/4087918740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730c159a90 con 0x7f730c06d750 2023-12-08T08:39:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.520+0000 7f7311861640 1 -- 172.21.15.16:0/4087918740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730c067430 con 0x7f730c06d750 2023-12-08T08:39:43.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.521+0000 7f72f3fff640 1 -- 172.21.15.16:0/4087918740 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f72fc02f050 con 0x7f730c06d750 2023-12-08T08:39:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.521+0000 7f72f3fff640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f72f806ae40 0x7f72f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.521+0000 7f72f3fff640 1 -- 172.21.15.16:0/4087918740 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f72fc0ab870 con 0x7f730c06d750 2023-12-08T08:39:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.522+0000 7f730a7fc640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f72f806ae40 0x7f72f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.522+0000 7f730a7fc640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f72f806ae40 0x7f72f806d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f730c0aedb0 tx=0x7f7300009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:43.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.526+0000 7f72f3fff640 1 -- 172.21.15.16:0/4087918740 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f72fc0767e0 con 0x7f730c06d750 2023-12-08T08:39:43.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:43.738+0000 7f7311861640 1 -- 172.21.15.16:0/4087918740 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f730c071e70 con 0x7f730c06d750 2023-12-08T08:39:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.395+0000 7f72f3fff640 1 -- 172.21.15.16:0/4087918740 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f72fc079e00 con 0x7f730c06d750 2023-12-08T08:39:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.398+0000 7f7311861640 1 -- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f72f806ae40 msgr2=0x7f72f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.398+0000 7f7311861640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f72f806ae40 0x7f72f806d300 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f730c0aedb0 tx=0x7f7300009040 comp rx=0 tx=0).stop 2023-12-08T08:39:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.398+0000 7f7311861640 1 -- 172.21.15.16:0/4087918740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c06d750 msgr2=0x7f730c159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.398+0000 7f7311861640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c06d750 0x7f730c159550 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f72fc009870 tx=0x7f72fc0046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.398+0000 7f7311861640 1 -- 172.21.15.16:0/4087918740 shutdown_connections 2023-12-08T08:39:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.398+0000 7f7311861640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f72f806ae40 0x7f72f806d300 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.398+0000 7f7311861640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f730c143980 0x7f730c1352e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.398+0000 7f7311861640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f730c0cf160 0x7f730c1f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.398+0000 7f7311861640 1 --2- 172.21.15.16:0/4087918740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f730c06d750 0x7f730c159550 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.398+0000 7f7311861640 1 -- 172.21.15.16:0/4087918740 >> 172.21.15.16:0/4087918740 conn(0x7f730c0b1ae0 msgr2=0x7f730c0b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:44.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.398+0000 7f7311861640 1 -- 172.21.15.16:0/4087918740 shutdown_connections 2023-12-08T08:39:44.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.398+0000 7f7311861640 1 -- 172.21.15.16:0/4087918740 wait complete. 2023-12-08T08:39:44.413 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-08T08:39:44.413 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-08T08:39:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.525+0000 7fbedd57c640 1 Processor -- start 2023-12-08T08:39:44.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.525+0000 7fbedd57c640 1 -- start start 2023-12-08T08:39:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.526+0000 7fbedd57c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbed8140a10 0x7fbed80cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.526+0000 7fbedd57c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed8070db0 con 0x7fbed8071e50 2023-12-08T08:39:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.526+0000 7fbedd57c640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed8071090 con 0x7fbed80b2640 2023-12-08T08:39:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.526+0000 7fbedd57c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8070ba0 con 0x7fbed8140a10 2023-12-08T08:39:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.526+0000 7fbed67fc640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbed8071e50 0x7fbed80b1f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51556/0 (socket says 172.21.15.16:51556) 2023-12-08T08:39:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.526+0000 7fbed67fc640 1 -- 172.21.15.16:0/1682351833 learned_addr learned my addr 172.21.15.16:0/1682351833 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.526+0000 7fbed5ffb640 1 -- 172.21.15.16:0/1682351833 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729194854 0 0) 0x7fbed8070db0 con 0x7fbed8071e50 2023-12-08T08:39:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.526+0000 7fbed5ffb640 1 -- 172.21.15.16:0/1682351833 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbec4003520 con 0x7fbed8071e50 2023-12-08T08:39:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.526+0000 7fbed6ffd640 1 --2- 172.21.15.16:0/1682351833 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbed8140a10 0x7fbed80cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed5ffb640 1 -- 172.21.15.16:0/1682351833 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 89367587 0 0) 0x7fbec4003520 con 0x7fbed8071e50 2023-12-08T08:39:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed5ffb640 1 -- 172.21.15.16:0/1682351833 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbed8140a10 msgr2=0x7fbed80cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed5ffb640 1 --2- 172.21.15.16:0/1682351833 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbed8140a10 0x7fbed80cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed5ffb640 1 -- 172.21.15.16:0/1682351833 >> v1:172.21.15.112:6789/0 conn(0x7fbed80b2640 legacy=0x7fbed806c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed5ffb640 1 -- 172.21.15.16:0/1682351833 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed8071370 con 0x7fbed8071e50 2023-12-08T08:39:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed5ffb640 1 -- 172.21.15.16:0/1682351833 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3092512649 0 0) 0x7fbec0001670 con 0x7fbed8071e50 2023-12-08T08:39:44.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed5ffb640 1 -- 172.21.15.16:0/1682351833 >> v1:172.21.15.16:6789/0 conn(0x7fbed8071e50 legacy=0x7fbed80b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed5ffb640 1 --2- 172.21.15.16:0/1682351833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbec40039a0 0x7fbec4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed5ffb640 1 --2- 172.21.15.16:0/1682351833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec40047c0 0x7fbec4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed5ffb640 1 --2- 172.21.15.16:0/1682351833 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbed8140a10 0x7fbec4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed5ffb640 1 -- 172.21.15.16:0/1682351833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8070ba0 con 0x7fbec40047c0 2023-12-08T08:39:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed5ffb640 1 -- 172.21.15.16:0/1682351833 --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec400bac0 con 0x7fbec40039a0 2023-12-08T08:39:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed5ffb640 1 -- 172.21.15.16:0/1682351833 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec400bc80 con 0x7fbed8140a10 2023-12-08T08:39:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed67fc640 1 --2- 172.21.15.16:0/1682351833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec40047c0 0x7fbec4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed6ffd640 1 --2- 172.21.15.16:0/1682351833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbec40039a0 0x7fbec4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed77fe640 1 --2- 172.21.15.16:0/1682351833 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbed8140a10 0x7fbec4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed67fc640 1 -- 172.21.15.16:0/1682351833 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbed8140a10 msgr2=0x7fbec4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed67fc640 1 --2- 172.21.15.16:0/1682351833 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbed8140a10 0x7fbec4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed67fc640 1 -- 172.21.15.16:0/1682351833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbec40039a0 msgr2=0x7fbec4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed67fc640 1 --2- 172.21.15.16:0/1682351833 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbec40039a0 0x7fbec4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.527+0000 7fbed67fc640 1 -- 172.21.15.16:0/1682351833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec4003520 con 0x7fbec40047c0 2023-12-08T08:39:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.528+0000 7fbed67fc640 1 --2- 172.21.15.16:0/1682351833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec40047c0 0x7fbec4004c40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbed815dc50 tx=0x7fbec40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762f0a583be6088e server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.528+0000 7fbed5ffb640 1 -- 172.21.15.16:0/1682351833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0005ad0 con 0x7fbec40047c0 2023-12-08T08:39:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.528+0000 7fbed5ffb640 1 -- 172.21.15.16:0/1682351833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec0005c30 con 0x7fbec40047c0 2023-12-08T08:39:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.528+0000 7fbed5ffb640 1 -- 172.21.15.16:0/1682351833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec002c830 con 0x7fbec40047c0 2023-12-08T08:39:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.529+0000 7fbedd57c640 1 -- 172.21.15.16:0/1682351833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec40047c0 msgr2=0x7fbec4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:44.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.529+0000 7fbedd57c640 1 --2- 172.21.15.16:0/1682351833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbec40047c0 0x7fbec4004c40 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fbed815dc50 tx=0x7fbec40057f0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.529+0000 7fbed77fe640 1 -- 172.21.15.16:0/1682351833 reap_dead start 2023-12-08T08:39:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.529+0000 7fbedd57c640 1 -- 172.21.15.16:0/1682351833 shutdown_connections 2023-12-08T08:39:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.529+0000 7fbedd57c640 1 -- 172.21.15.16:0/1682351833 >> 172.21.15.16:0/1682351833 conn(0x7fbed80b09f0 msgr2=0x7fbed80b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.529+0000 7fbedd57c640 1 -- 172.21.15.16:0/1682351833 shutdown_connections 2023-12-08T08:39:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.529+0000 7fbedd57c640 1 -- 172.21.15.16:0/1682351833 wait complete. 2023-12-08T08:39:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.530+0000 7fbedd57c640 1 Processor -- start 2023-12-08T08:39:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.530+0000 7fbedd57c640 1 -- start start 2023-12-08T08:39:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbedd57c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbec40047c0 0x7fbed813db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbedd57c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed8142870 0x7fbed81df320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbedd57c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbed81f10f0 0x7fbed81f15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbedd57c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8142d00 con 0x7fbed8142870 2023-12-08T08:39:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbedd57c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed8142fe0 con 0x7fbed81f10f0 2023-12-08T08:39:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbedd57c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed81432c0 con 0x7fbec40047c0 2023-12-08T08:39:44.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbed67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed8142870 0x7fbed81df320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbed67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed8142870 0x7fbed81df320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34724/0 (socket says 172.21.15.16:34724) 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbed67fc640 1 -- 172.21.15.16:0/1138450829 learned_addr learned my addr 172.21.15.16:0/1138450829 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbed6ffd640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbec40047c0 0x7fbed813db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbed77fe640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbed81f10f0 0x7fbed81f15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbed67fc640 1 -- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbec40047c0 msgr2=0x7fbed813db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbed67fc640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbec40047c0 0x7fbed813db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbed67fc640 1 -- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbed81f10f0 msgr2=0x7fbed81f15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbed67fc640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbed81f10f0 0x7fbed81f15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbed67fc640 1 -- 172.21.15.16:0/1138450829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed81435a0 con 0x7fbed8142870 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.531+0000 7fbed77fe640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbed81f10f0 0x7fbed81f15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.532+0000 7fbed67fc640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed8142870 0x7fbed81df320 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbec0001930 tx=0x7fbed80b3cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.532+0000 7fbecf7fe640 1 -- 172.21.15.16:0/1138450829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0039070 con 0x7fbed8142870 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.532+0000 7fbecf7fe640 1 -- 172.21.15.16:0/1138450829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec002cc40 con 0x7fbed8142870 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.532+0000 7fbecf7fe640 1 -- 172.21.15.16:0/1138450829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec003c040 con 0x7fbed8142870 2023-12-08T08:39:44.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.532+0000 7fbedd57c640 1 -- 172.21.15.16:0/1138450829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed81df860 con 0x7fbed8142870 2023-12-08T08:39:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.532+0000 7fbedd57c640 1 -- 172.21.15.16:0/1138450829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed81f1cc0 con 0x7fbed8142870 2023-12-08T08:39:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.533+0000 7fbedd57c640 1 -- 172.21.15.16:0/1138450829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed81dfa50 con 0x7fbed8142870 2023-12-08T08:39:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.533+0000 7fbecf7fe640 1 -- 172.21.15.16:0/1138450829 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbec0032070 con 0x7fbed8142870 2023-12-08T08:39:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.534+0000 7fbecf7fe640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbeb806ae40 0x7fbeb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.534+0000 7fbecf7fe640 1 -- 172.21.15.16:0/1138450829 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fbec00aaa20 con 0x7fbed8142870 2023-12-08T08:39:44.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.534+0000 7fbed6ffd640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbeb806ae40 0x7fbeb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.534+0000 7fbed6ffd640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbeb806ae40 0x7fbeb806d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009bc0 tx=0x7fbed0009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:44.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.538+0000 7fbecf7fe640 1 -- 172.21.15.16:0/1138450829 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbec0075a40 con 0x7fbed8142870 2023-12-08T08:39:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.752+0000 7fbedd57c640 1 -- 172.21.15.16:0/1138450829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbed8067430 con 0x7fbed8142870 2023-12-08T08:39:44.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.752+0000 7fbecf7fe640 1 -- 172.21.15.16:0/1138450829 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4916 (secure 0 0 0) 0x7fbec0079060 con 0x7fbed8142870 2023-12-08T08:39:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:44.755 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1455612594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1455612594},{"type":"v1","addr":"172.21.15.112:6837","nonce":1455612594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4904,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4205380866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4205380866},{"type":"v1","addr":"172.21.15.16:6834","nonce":4205380866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/516241878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":516241878},{"type":"v1","addr":"172.21.15.16:6835","nonce":516241878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-08T08:39:41.743317+0000","modified":"2023-12-08T08:39:43.739526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5170},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5170":{"gid":5170,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/1132982118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1132982118},{"type":"v1","addr":"172.21.15.112:6836","nonce":1132982118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-08T08:39:44.755 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2023-12-08T08:39:44.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.754+0000 7fbedd57c640 1 -- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbeb806ae40 msgr2=0x7fbeb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.754+0000 7fbedd57c640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbeb806ae40 0x7fbeb806d300 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009bc0 tx=0x7fbed0009190 comp rx=0 tx=0).stop 2023-12-08T08:39:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.754+0000 7fbedd57c640 1 -- 172.21.15.16:0/1138450829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed8142870 msgr2=0x7fbed81df320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.754+0000 7fbedd57c640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed8142870 0x7fbed81df320 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbec0001930 tx=0x7fbed80b3cf0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.754+0000 7fbedd57c640 1 -- 172.21.15.16:0/1138450829 shutdown_connections 2023-12-08T08:39:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.754+0000 7fbedd57c640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fbeb806ae40 0x7fbeb806d300 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.754+0000 7fbedd57c640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fbed81f10f0 0x7fbed81f15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.754+0000 7fbedd57c640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed8142870 0x7fbed81df320 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.754+0000 7fbedd57c640 1 --2- 172.21.15.16:0/1138450829 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fbec40047c0 0x7fbed813db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.754+0000 7fbedd57c640 1 -- 172.21.15.16:0/1138450829 >> 172.21.15.16:0/1138450829 conn(0x7fbed80b09f0 msgr2=0x7fbed813f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.754+0000 7fbedd57c640 1 -- 172.21.15.16:0/1138450829 shutdown_connections 2023-12-08T08:39:44.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.754+0000 7fbedd57c640 1 -- 172.21.15.16:0/1138450829 wait complete. 2023-12-08T08:39:44.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-08T08:39:44.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.871+0000 7f1f4a2cf640 1 Processor -- start 2023-12-08T08:39:44.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.872+0000 7f1f4a2cf640 1 -- start start 2023-12-08T08:39:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.872+0000 7f1f4a2cf640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f441608a0 0x7f1f440d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.872+0000 7f1f4a2cf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f440b1d70 0x7f1f440d1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.872+0000 7f1f4a2cf640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f1f440afce0 0x7f1f440b0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.872+0000 7f1f4a2cf640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f440d0620 con 0x7f1f441608a0 2023-12-08T08:39:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.872+0000 7f1f4a2cf640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f440d20b0 con 0x7f1f440afce0 2023-12-08T08:39:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.872+0000 7f1f4a2cf640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440d1dd0 con 0x7f1f440b1d70 2023-12-08T08:39:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.872+0000 7f1f43fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f441608a0 0x7f1f440d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.872+0000 7f1f43fff640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f441608a0 0x7f1f440d0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34740/0 (socket says 172.21.15.16:34740) 2023-12-08T08:39:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.872+0000 7f1f43fff640 1 -- 172.21.15.16:0/381947641 learned_addr learned my addr 172.21.15.16:0/381947641 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.873+0000 7f1f437fe640 1 --2- 172.21.15.16:0/381947641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f440b1d70 0x7f1f440d1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.873+0000 7f1f4886c640 1 --2- 172.21.15.16:0/381947641 >> v2:172.21.15.112:3300/0 conn(0x7f1f440afce0 0x7f1f440b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.873+0000 7f1f43fff640 1 -- 172.21.15.16:0/381947641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f440b1d70 msgr2=0x7f1f440d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.873+0000 7f1f43fff640 1 --2- 172.21.15.16:0/381947641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f440b1d70 0x7f1f440d1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.873+0000 7f1f43fff640 1 -- 172.21.15.16:0/381947641 >> v2:172.21.15.112:3300/0 conn(0x7f1f440afce0 msgr2=0x7f1f440b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:44.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.873+0000 7f1f43fff640 1 --2- 172.21.15.16:0/381947641 >> v2:172.21.15.112:3300/0 conn(0x7f1f440afce0 0x7f1f440b0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.873+0000 7f1f43fff640 1 -- 172.21.15.16:0/381947641 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f440cff60 con 0x7f1f441608a0 2023-12-08T08:39:44.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.873+0000 7f1f4886c640 1 --2- 172.21.15.16:0/381947641 >> v2:172.21.15.112:3300/0 conn(0x7f1f440afce0 0x7f1f440b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:44.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.873+0000 7f1f43fff640 1 --2- 172.21.15.16:0/381947641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f441608a0 0x7f1f440d0f30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1f440b27c0 tx=0x7f1f3402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7136da214b7bee8f server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:44.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.874+0000 7f1f42ffd640 1 -- 172.21.15.16:0/381947641 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f340317e0 con 0x7f1f441608a0 2023-12-08T08:39:44.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.874+0000 7f1f42ffd640 1 -- 172.21.15.16:0/381947641 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f34031940 con 0x7f1f441608a0 2023-12-08T08:39:44.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.874+0000 7f1f42ffd640 1 -- 172.21.15.16:0/381947641 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34031cd0 con 0x7f1f441608a0 2023-12-08T08:39:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.874+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/381947641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f441608a0 msgr2=0x7f1f440d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.874+0000 7f1f4a2cf640 1 --2- 172.21.15.16:0/381947641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f441608a0 0x7f1f440d0f30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1f440b27c0 tx=0x7f1f3402b880 comp rx=0 tx=0).stop 2023-12-08T08:39:44.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.874+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/381947641 shutdown_connections 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.874+0000 7f1f4a2cf640 1 --2- 172.21.15.16:0/381947641 >> v2:172.21.15.112:3300/0 conn(0x7f1f440afce0 0x7f1f440b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.874+0000 7f1f4a2cf640 1 --2- 172.21.15.16:0/381947641 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f440b1d70 0x7f1f440d1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.874+0000 7f1f4a2cf640 1 --2- 172.21.15.16:0/381947641 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f441608a0 0x7f1f440d0f30 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.874+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/381947641 >> 172.21.15.16:0/381947641 conn(0x7f1f440d3030 msgr2=0x7f1f440d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.874+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/381947641 shutdown_connections 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.874+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/381947641 wait complete. 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.875+0000 7f1f4a2cf640 1 Processor -- start 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.876+0000 7f1f4a2cf640 1 -- start start 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.876+0000 7f1f4a2cf640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f440afce0 0x7f1f4415c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.876+0000 7f1f4a2cf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f440b1d70 0x7f1f440c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.876+0000 7f1f4a2cf640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f441608a0 0x7f1f440b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.876+0000 7f1f4a2cf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440c7ce0 con 0x7f1f440b1d70 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.876+0000 7f1f4a2cf640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440c7fc0 con 0x7f1f441608a0 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.876+0000 7f1f4a2cf640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f440c82a0 con 0x7f1f440afce0 2023-12-08T08:39:44.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.876+0000 7f1f437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f440b1d70 0x7f1f440c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.876+0000 7f1f437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f440b1d70 0x7f1f440c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34744/0 (socket says 172.21.15.16:34744) 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.876+0000 7f1f437fe640 1 -- 172.21.15.16:0/1205621536 learned_addr learned my addr 172.21.15.16:0/1205621536 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.877+0000 7f1f4886c640 1 --2- 172.21.15.16:0/1205621536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f441608a0 0x7f1f440b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.877+0000 7f1f43fff640 1 --2- 172.21.15.16:0/1205621536 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f440afce0 0x7f1f4415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.877+0000 7f1f437fe640 1 -- 172.21.15.16:0/1205621536 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f440afce0 msgr2=0x7f1f4415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.877+0000 7f1f437fe640 1 --2- 172.21.15.16:0/1205621536 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f440afce0 0x7f1f4415c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.877+0000 7f1f437fe640 1 -- 172.21.15.16:0/1205621536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f441608a0 msgr2=0x7f1f440b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.877+0000 7f1f437fe640 1 --2- 172.21.15.16:0/1205621536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f441608a0 0x7f1f440b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.877+0000 7f1f437fe640 1 -- 172.21.15.16:0/1205621536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f340094d0 con 0x7f1f440b1d70 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.877+0000 7f1f437fe640 1 --2- 172.21.15.16:0/1205621536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f440b1d70 0x7f1f440c94a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00d5f0 tx=0x7f1f2c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.877+0000 7f1f40ff9640 1 -- 172.21.15.16:0/1205621536 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c0125f0 con 0x7f1f440b1d70 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.877+0000 7f1f40ff9640 1 -- 172.21.15.16:0/1205621536 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c012750 con 0x7f1f440b1d70 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.877+0000 7f1f40ff9640 1 -- 172.21.15.16:0/1205621536 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c012b00 con 0x7f1f440b1d70 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.877+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/1205621536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f440cba40 con 0x7f1f440b1d70 2023-12-08T08:39:44.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.877+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/1205621536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f440cb710 con 0x7f1f440b1d70 2023-12-08T08:39:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.878+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/1205621536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f44067430 con 0x7f1f440b1d70 2023-12-08T08:39:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.878+0000 7f1f40ff9640 1 -- 172.21.15.16:0/1205621536 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f2c012c60 con 0x7f1f440b1d70 2023-12-08T08:39:44.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.879+0000 7f1f40ff9640 1 --2- 172.21.15.16:0/1205621536 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1f2406ae40 0x7f1f2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:44.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.879+0000 7f1f40ff9640 1 -- 172.21.15.16:0/1205621536 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1f2c089c40 con 0x7f1f440b1d70 2023-12-08T08:39:44.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.879+0000 7f1f43fff640 1 --2- 172.21.15.16:0/1205621536 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1f2406ae40 0x7f1f2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:44.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.880+0000 7f1f43fff640 1 --2- 172.21.15.16:0/1205621536 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1f2406ae40 0x7f1f2406d300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f34036000 tx=0x7f1f3403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:44.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:44.883+0000 7f1f40ff9640 1 -- 172.21.15.16:0/1205621536 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f2c054bb0 con 0x7f1f440b1d70 2023-12-08T08:39:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.061+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/1205621536 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f440d35b0 con 0x7f1f440b1d70 2023-12-08T08:39:45.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.062+0000 7f1f40ff9640 1 -- 172.21.15.16:0/1205621536 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16774 (secure 0 0 0) 0x7f1f2c0581d0 con 0x7f1f440b1d70 2023-12-08T08:39:45.064 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:45.064 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","created":"2023-12-08T08:37:54.860669+0000","modified":"2023-12-08T08:39:43.386270+0000","last_up_change":"2023-12-08T08:38:05.653655+0000","last_in_change":"2023-12-08T08:37:57.072680+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-08T08:38:07.031785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-12-08T08:39:39.593361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-12-08T08:39:40.726565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1debed5b-b2a4-4603-84ce-36ea491b7332","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6818","nonce":3573990929}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6822","nonce":3573990929}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6830","nonce":3573990929}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3573990929},{"type":"v1","addr":"172.21.15.16:6826","nonce":3573990929}]},"public_addr":"172.21.15.16:6818/3573990929","cluster_addr":"172.21.15.16:6822/3573990929","heartbeat_back_addr":"172.21.15.16:6830/3573990929","heartbeat_front_addr":"172.21.15.16:6826/3573990929","state":["exists","up"]},{"osd":1,"uuid":"33af8d04-7310-43c1-ac4b-205f09214b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6817","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6819","nonce":1015436151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6821","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6823","nonce":1015436151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6829","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6831","nonce":1015436151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6825","nonce":1015436151},{"type":"v1","addr":"172.21.15.16:6827","nonce":1015436151}]},"public_addr":"172.21.15.16:6819/1015436151","cluster_addr":"172.21.15.16:6823/1015436151","heartbeat_back_addr":"172.21.15.16:6831/1015436151","heartbeat_front_addr":"172.21.15.16:6827/1015436151","state":["exists","up"]},{"osd":2,"uuid":"838af08b-fb26-4744-81ce-929407379290","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6801","nonce":3962287192}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6803","nonce":3962287192}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6807","nonce":3962287192}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":3962287192},{"type":"v1","addr":"172.21.15.16:6805","nonce":3962287192}]},"public_addr":"172.21.15.16:6801/3962287192","cluster_addr":"172.21.15.16:6803/3962287192","heartbeat_back_addr":"172.21.15.16:6807/3962287192","heartbeat_front_addr":"172.21.15.16:6805/3962287192","state":["exists","up"]},{"osd":3,"uuid":"137a9f13-952c-4bb3-b7fc-88a86457f7bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6809","nonce":3215779870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6811","nonce":3215779870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6815","nonce":3215779870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3215779870},{"type":"v1","addr":"172.21.15.16:6813","nonce":3215779870}]},"public_addr":"172.21.15.16:6809/3215779870","cluster_addr":"172.21.15.16:6811/3215779870","heartbeat_back_addr":"172.21.15.16:6815/3215779870","heartbeat_front_addr":"172.21.15.16:6813/3215779870","state":["exists","up"]},{"osd":4,"uuid":"f2d870f3-d72c-4943-bf09-a181ac9a70ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6824","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6825","nonce":898596008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6826","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6827","nonce":898596008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6830","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6831","nonce":898596008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6828","nonce":898596008},{"type":"v1","addr":"172.21.15.112:6829","nonce":898596008}]},"public_addr":"172.21.15.112:6825/898596008","cluster_addr":"172.21.15.112:6827/898596008","heartbeat_back_addr":"172.21.15.112:6831/898596008","heartbeat_front_addr":"172.21.15.112:6829/898596008","state":["exists","up"]},{"osd":5,"uuid":"1229fda8-dfc6-48d3-99c1-3c8b8338aacf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6816","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6817","nonce":1387317691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6818","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6819","nonce":1387317691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6822","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6823","nonce":1387317691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6820","nonce":1387317691},{"type":"v1","addr":"172.21.15.112:6821","nonce":1387317691}]},"public_addr":"172.21.15.112:6817/1387317691","cluster_addr":"172.21.15.112:6819/1387317691","heartbeat_back_addr":"172.21.15.112:6823/1387317691","heartbeat_front_addr":"172.21.15.112:6821/1387317691","state":["exists","up"]},{"osd":6,"uuid":"916cbfdb-5ef0-452c-b1cd-fa2d59f7fddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6800","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6801","nonce":1165220586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6802","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6803","nonce":1165220586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6806","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6807","nonce":1165220586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6804","nonce":1165220586},{"type":"v1","addr":"172.21.15.112:6805","nonce":1165220586}]},"public_addr":"172.21.15.112:6801/1165220586","cluster_addr":"172.21.15.112:6803/1165220586","heartbeat_back_addr":"172.21.15.112:6807/1165220586","heartbeat_front_addr":"172.21.15.112:6805/1165220586","state":["exists","up"]},{"osd":7,"uuid":"a4052663-452c-4808-8fb5-11069b32cf37","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6808","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6809","nonce":1116392641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6810","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6811","nonce":1116392641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6814","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6815","nonce":1116392641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6812","nonce":1116392641},{"type":"v1","addr":"172.21.15.112:6813","nonce":1116392641}]},"public_addr":"172.21.15.112:6809/1116392641","cluster_addr":"172.21.15.112:6811/1116392641","heartbeat_back_addr":"172.21.15.112:6815/1116392641","heartbeat_front_addr":"172.21.15.112:6813/1116392641","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.258885+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.197953+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.178180+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.132124+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.250275+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.194003+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.224922+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-08T08:38:02.179032+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-08T08:39:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.064+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/1205621536 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1f2406ae40 msgr2=0x7f1f2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.064+0000 7f1f4a2cf640 1 --2- 172.21.15.16:0/1205621536 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1f2406ae40 0x7f1f2406d300 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f34036000 tx=0x7f1f3403d040 comp rx=0 tx=0).stop 2023-12-08T08:39:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.064+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/1205621536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f440b1d70 msgr2=0x7f1f440c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.064+0000 7f1f4a2cf640 1 --2- 172.21.15.16:0/1205621536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f440b1d70 0x7f1f440c94a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00d5f0 tx=0x7f1f2c00dab0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.064+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/1205621536 shutdown_connections 2023-12-08T08:39:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.064+0000 7f1f4a2cf640 1 --2- 172.21.15.16:0/1205621536 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f1f2406ae40 0x7f1f2406d300 secure :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f34036000 tx=0x7f1f3403d040 comp rx=0 tx=0).stop 2023-12-08T08:39:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.064+0000 7f1f4a2cf640 1 --2- 172.21.15.16:0/1205621536 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f1f441608a0 0x7f1f440b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.064+0000 7f1f4a2cf640 1 --2- 172.21.15.16:0/1205621536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f440b1d70 0x7f1f440c94a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.064+0000 7f1f4a2cf640 1 --2- 172.21.15.16:0/1205621536 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f1f440afce0 0x7f1f4415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.064+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/1205621536 >> 172.21.15.16:0/1205621536 conn(0x7f1f440d3030 msgr2=0x7f1f440b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.064+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/1205621536 shutdown_connections 2023-12-08T08:39:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.064+0000 7f1f4a2cf640 1 -- 172.21.15.16:0/1205621536 wait complete. 2023-12-08T08:39:45.077 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-08T08:39:45.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-08T08:39:45.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.181+0000 7ff53b961640 1 Processor -- start 2023-12-08T08:39:45.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.181+0000 7ff53b961640 1 -- start start 2023-12-08T08:39:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.182+0000 7ff53b961640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5340b6ac0 0x7ff5340cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.182+0000 7ff53b961640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff5340ce190 0x7ff5340cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.182+0000 7ff53b961640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5340d0070 con 0x7ff5340cd660 2023-12-08T08:39:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.182+0000 7ff53b961640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff534160cb0 con 0x7ff5340ce190 2023-12-08T08:39:45.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.182+0000 7ff53b961640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340d02b0 con 0x7ff5340b6ac0 2023-12-08T08:39:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.182+0000 7ff5396fd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5340b6ac0 0x7ff5340cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.183+0000 7ff539efe640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7ff5340ce190 0x7ff5340cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.183+0000 7ff5396fd640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5340b6ac0 0x7ff5340cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:42120/0 (socket says 172.21.15.16:42120) 2023-12-08T08:39:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.183+0000 7ff5396fd640 1 -- 172.21.15.16:0/2436826624 learned_addr learned my addr 172.21.15.16:0/2436826624 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.183+0000 7ff5396fd640 1 -- 172.21.15.16:0/2436826624 >> v2:172.21.15.112:3300/0 conn(0x7ff5340ce190 msgr2=0x7ff5340cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.183+0000 7ff5396fd640 1 --2- 172.21.15.16:0/2436826624 >> v2:172.21.15.112:3300/0 conn(0x7ff5340ce190 0x7ff5340cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.183+0000 7ff5396fd640 1 -- 172.21.15.16:0/2436826624 >> v1:172.21.15.112:6789/0 conn(0x7ff5340cd660 legacy=0x7ff5340cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.183+0000 7ff533fff640 1 -- 172.21.15.16:0/2436826624 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792514173 0 0) 0x7ff528001110 con 0x7ff5340cd660 2023-12-08T08:39:45.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.183+0000 7ff5396fd640 1 -- 172.21.15.16:0/2436826624 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5340cb430 con 0x7ff5340b6ac0 2023-12-08T08:39:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.184+0000 7ff5396fd640 1 --2- 172.21.15.16:0/2436826624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5340b6ac0 0x7ff5340cfb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009870 tx=0x7ff52c02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3ce0f79abb631d server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.184+0000 7ff533fff640 1 -- 172.21.15.16:0/2436826624 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0317c0 con 0x7ff5340b6ac0 2023-12-08T08:39:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.184+0000 7ff533fff640 1 -- 172.21.15.16:0/2436826624 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c031920 con 0x7ff5340b6ac0 2023-12-08T08:39:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.184+0000 7ff533fff640 1 -- 172.21.15.16:0/2436826624 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c031cb0 con 0x7ff5340b6ac0 2023-12-08T08:39:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.184+0000 7ff53b961640 1 -- 172.21.15.16:0/2436826624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5340b6ac0 msgr2=0x7ff5340cfb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.184+0000 7ff53b961640 1 --2- 172.21.15.16:0/2436826624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5340b6ac0 0x7ff5340cfb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009870 tx=0x7ff52c02b860 comp rx=0 tx=0).stop 2023-12-08T08:39:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.184+0000 7ff53b961640 1 -- 172.21.15.16:0/2436826624 shutdown_connections 2023-12-08T08:39:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.184+0000 7ff53b961640 1 --2- 172.21.15.16:0/2436826624 >> v2:172.21.15.112:3300/0 conn(0x7ff5340ce190 0x7ff5340cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.184+0000 7ff53b961640 1 --2- 172.21.15.16:0/2436826624 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5340b6ac0 0x7ff5340cfb30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.184+0000 7ff53b961640 1 -- 172.21.15.16:0/2436826624 >> 172.21.15.16:0/2436826624 conn(0x7ff5340d0de0 msgr2=0x7ff5340d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:45.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.185+0000 7ff53b961640 1 -- 172.21.15.16:0/2436826624 shutdown_connections 2023-12-08T08:39:45.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.185+0000 7ff53b961640 1 -- 172.21.15.16:0/2436826624 wait complete. 2023-12-08T08:39:45.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.186+0000 7ff53b961640 1 Processor -- start 2023-12-08T08:39:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.186+0000 7ff53b961640 1 -- start start 2023-12-08T08:39:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.186+0000 7ff53b961640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5340b6ac0 0x7ff53415e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.186+0000 7ff53b961640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5340cd660 0x7ff53409c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.186+0000 7ff53b961640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5340ce190 0x7ff5340b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.186+0000 7ff53b961640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340aff70 con 0x7ff5340b6ac0 2023-12-08T08:39:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.186+0000 7ff53b961640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53415ecb0 con 0x7ff5340ce190 2023-12-08T08:39:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.186+0000 7ff53b961640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53409bb00 con 0x7ff5340cd660 2023-12-08T08:39:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.186+0000 7ff5396fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5340b6ac0 0x7ff53415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff5396fd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5340b6ac0 0x7ff53415e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34758/0 (socket says 172.21.15.16:34758) 2023-12-08T08:39:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff5396fd640 1 -- 172.21.15.16:0/977896077 learned_addr learned my addr 172.21.15.16:0/977896077 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff538efc640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5340cd660 0x7ff53409c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff5396fd640 1 -- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5340cd660 msgr2=0x7ff53409c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff5396fd640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5340cd660 0x7ff53409c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff5396fd640 1 -- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5340ce190 msgr2=0x7ff5340b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff539efe640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5340ce190 0x7ff5340b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff5396fd640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5340ce190 0x7ff5340b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff5396fd640 1 -- 172.21.15.16:0/977896077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c0094d0 con 0x7ff5340b6ac0 2023-12-08T08:39:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff538efc640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5340cd660 0x7ff53409c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff5396fd640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5340b6ac0 0x7ff53415e770 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff52c000c00 tx=0x7ff52c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff531ffb640 1 -- 172.21.15.16:0/977896077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c032450 con 0x7ff5340b6ac0 2023-12-08T08:39:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff531ffb640 1 -- 172.21.15.16:0/977896077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c0325b0 con 0x7ff5340b6ac0 2023-12-08T08:39:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff531ffb640 1 -- 172.21.15.16:0/977896077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c032960 con 0x7ff5340b6ac0 2023-12-08T08:39:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff53b961640 1 -- 172.21.15.16:0/977896077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5340b0ad0 con 0x7ff5340b6ac0 2023-12-08T08:39:45.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.187+0000 7ff53b961640 1 -- 172.21.15.16:0/977896077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5340b0480 con 0x7ff5340b6ac0 2023-12-08T08:39:45.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.188+0000 7ff53b961640 1 -- 172.21.15.16:0/977896077 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff534067430 con 0x7ff5340b6ac0 2023-12-08T08:39:45.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.189+0000 7ff531ffb640 1 -- 172.21.15.16:0/977896077 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff52c043070 con 0x7ff5340b6ac0 2023-12-08T08:39:45.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.189+0000 7ff531ffb640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff52006ae40 0x7ff52006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.189+0000 7ff531ffb640 1 -- 172.21.15.16:0/977896077 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7ff52c0ab240 con 0x7ff5340b6ac0 2023-12-08T08:39:45.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.189+0000 7ff538efc640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff52006ae40 0x7ff52006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.190+0000 7ff538efc640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff52006ae40 0x7ff52006d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff5341f65b0 tx=0x7ff528009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:45.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.194+0000 7ff531ffb640 1 -- 172.21.15.16:0/977896077 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff52c076110 con 0x7ff5340b6ac0 2023-12-08T08:39:45.360 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-12-08T08:39:45.358+0000 7f52e8b12640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-08T08:39:45.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.366+0000 7ff53b961640 1 -- 172.21.15.16:0/977896077 --> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff53406ee20 con 0x7ff52006ae40 2023-12-08T08:39:45.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.374+0000 7ff531ffb640 1 -- 172.21.15.16:0/977896077 <== mgr.4111 v2:172.21.15.112:6832/804067392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277533 (secure 0 0 0) 0x7ff53406ee20 con 0x7ff52006ae40 2023-12-08T08:39:45.378 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:45.380 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-12-08T08:39:45.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.376+0000 7ff53b961640 1 -- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff52006ae40 msgr2=0x7ff52006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.376+0000 7ff53b961640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff52006ae40 0x7ff52006d300 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff5341f65b0 tx=0x7ff528009040 comp rx=0 tx=0).stop 2023-12-08T08:39:45.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.376+0000 7ff53b961640 1 -- 172.21.15.16:0/977896077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5340b6ac0 msgr2=0x7ff53415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.376+0000 7ff53b961640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5340b6ac0 0x7ff53415e770 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff52c000c00 tx=0x7ff52c0046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.377+0000 7ff53b961640 1 -- 172.21.15.16:0/977896077 shutdown_connections 2023-12-08T08:39:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.377+0000 7ff53b961640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7ff52006ae40 0x7ff52006d300 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.377+0000 7ff53b961640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7ff5340ce190 0x7ff5340b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.377+0000 7ff53b961640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7ff5340cd660 0x7ff53409c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.377+0000 7ff53b961640 1 --2- 172.21.15.16:0/977896077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff5340b6ac0 0x7ff53415e770 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.377+0000 7ff53b961640 1 -- 172.21.15.16:0/977896077 >> 172.21.15.16:0/977896077 conn(0x7ff5340d0de0 msgr2=0x7ff53406efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.377+0000 7ff53b961640 1 -- 172.21.15.16:0/977896077 shutdown_connections 2023-12-08T08:39:45.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.377+0000 7ff53b961640 1 -- 172.21.15.16:0/977896077 wait complete. 2023-12-08T08:39:45.389 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2023-12-08T08:39:44.996436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233016,"kb_used_data":4356,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741704,"statfs":{"total":773094113280,"available":772855504896,"internally_reserved":0,"allocated":4460544,"data_stored":2554093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.629075"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372184+0000","last_change":"2023-12-08T08:39:40.372184+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372184+0000","last_undegraded":"2023-12-08T08:39:40.372184+0000","last_fullsized":"2023-12-08T08:39:40.372184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372016+0000","last_change":"2023-12-08T08:39:40.372016+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372016+0000","last_undegraded":"2023-12-08T08:39:40.372016+0000","last_fullsized":"2023-12-08T08:39:40.372016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375234+0000","last_change":"2023-12-08T08:39:40.375234+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375234+0000","last_undegraded":"2023-12-08T08:39:40.375234+0000","last_fullsized":"2023-12-08T08:39:40.375234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369509+0000","last_change":"2023-12-08T08:39:40.369509+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369509+0000","last_undegraded":"2023-12-08T08:39:40.369509+0000","last_fullsized":"2023-12-08T08:39:40.369509+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371677+0000","last_change":"2023-12-08T08:39:40.371677+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371677+0000","last_undegraded":"2023-12-08T08:39:40.371677+0000","last_fullsized":"2023-12-08T08:39:40.371677+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373183+0000","last_change":"2023-12-08T08:39:40.373183+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373183+0000","last_undegraded":"2023-12-08T08:39:40.373183+0000","last_fullsized":"2023-12-08T08:39:40.373183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375447+0000","last_change":"2023-12-08T08:39:40.375447+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375447+0000","last_undegraded":"2023-12-08T08:39:40.375447+0000","last_fullsized":"2023-12-08T08:39:40.375447+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369561+0000","last_change":"2023-12-08T08:39:40.369561+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369561+0000","last_undegraded":"2023-12-08T08:39:40.369561+0000","last_fullsized":"2023-12-08T08:39:40.369561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372358+0000","last_change":"2023-12-08T08:39:40.372358+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372358+0000","last_undegraded":"2023-12-08T08:39:40.372358+0000","last_fullsized":"2023-12-08T08:39:40.372358+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371732+0000","last_change":"2023-12-08T08:39:40.371732+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371732+0000","last_undegraded":"2023-12-08T08:39:40.371732+0000","last_fullsized":"2023-12-08T08:39:40.371732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371718+0000","last_change":"2023-12-08T08:39:40.371718+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371718+0000","last_undegraded":"2023-12-08T08:39:40.371718+0000","last_fullsized":"2023-12-08T08:39:40.371718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371956+0000","last_change":"2023-12-08T08:39:40.371956+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371956+0000","last_undegraded":"2023-12-08T08:39:40.371956+0000","last_fullsized":"2023-12-08T08:39:40.371956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372089+0000","last_change":"2023-12-08T08:39:40.372089+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372089+0000","last_undegraded":"2023-12-08T08:39:40.372089+0000","last_fullsized":"2023-12-08T08:39:40.372089+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375171+0000","last_change":"2023-12-08T08:39:40.375171+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375171+0000","last_undegraded":"2023-12-08T08:39:40.375171+0000","last_fullsized":"2023-12-08T08:39:40.375171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371915+0000","last_change":"2023-12-08T08:39:40.371915+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371915+0000","last_undegraded":"2023-12-08T08:39:40.371915+0000","last_fullsized":"2023-12-08T08:39:40.371915+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373260+0000","last_change":"2023-12-08T08:39:40.373260+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373260+0000","last_undegraded":"2023-12-08T08:39:40.373260+0000","last_fullsized":"2023-12-08T08:39:40.373260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372660+0000","last_change":"2023-12-08T08:39:40.372660+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372660+0000","last_undegraded":"2023-12-08T08:39:40.372660+0000","last_fullsized":"2023-12-08T08:39:40.372660+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369563+0000","last_change":"2023-12-08T08:39:40.369563+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369563+0000","last_undegraded":"2023-12-08T08:39:40.369563+0000","last_fullsized":"2023-12-08T08:39:40.369563+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.370714+0000","last_change":"2023-12-08T08:39:40.370714+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.370714+0000","last_undegraded":"2023-12-08T08:39:40.370714+0000","last_fullsized":"2023-12-08T08:39:40.370714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375212+0000","last_change":"2023-12-08T08:39:40.375212+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375212+0000","last_undegraded":"2023-12-08T08:39:40.375212+0000","last_fullsized":"2023-12-08T08:39:40.375212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.370162+0000","last_change":"2023-12-08T08:39:40.370162+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.370162+0000","last_undegraded":"2023-12-08T08:39:40.370162+0000","last_fullsized":"2023-12-08T08:39:40.370162+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.370100+0000","last_change":"2023-12-08T08:39:40.370100+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.370100+0000","last_undegraded":"2023-12-08T08:39:40.370100+0000","last_fullsized":"2023-12-08T08:39:40.370100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373451+0000","last_change":"2023-12-08T08:39:40.373451+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373451+0000","last_undegraded":"2023-12-08T08:39:40.373451+0000","last_fullsized":"2023-12-08T08:39:40.373451+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369727+0000","last_change":"2023-12-08T08:39:40.369727+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369727+0000","last_undegraded":"2023-12-08T08:39:40.369727+0000","last_fullsized":"2023-12-08T08:39:40.369727+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372229+0000","last_change":"2023-12-08T08:39:40.372229+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372229+0000","last_undegraded":"2023-12-08T08:39:40.372229+0000","last_fullsized":"2023-12-08T08:39:40.372229+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369727+0000","last_change":"2023-12-08T08:39:40.369727+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369727+0000","last_undegraded":"2023-12-08T08:39:40.369727+0000","last_fullsized":"2023-12-08T08:39:40.369727+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372041+0000","last_change":"2023-12-08T08:39:40.372041+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372041+0000","last_undegraded":"2023-12-08T08:39:40.372041+0000","last_fullsized":"2023-12-08T08:39:40.372041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372086+0000","last_change":"2023-12-08T08:39:40.372086+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372086+0000","last_undegraded":"2023-12-08T08:39:40.372086+0000","last_fullsized":"2023-12-08T08:39:40.372086+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371953+0000","last_change":"2023-12-08T08:39:40.371953+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371953+0000","last_undegraded":"2023-12-08T08:39:40.371953+0000","last_fullsized":"2023-12-08T08:39:40.371953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369539+0000","last_change":"2023-12-08T08:39:40.369539+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369539+0000","last_undegraded":"2023-12-08T08:39:40.369539+0000","last_fullsized":"2023-12-08T08:39:40.369539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371795+0000","last_change":"2023-12-08T08:39:40.371795+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371795+0000","last_undegraded":"2023-12-08T08:39:40.371795+0000","last_fullsized":"2023-12-08T08:39:40.371795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373495+0000","last_change":"2023-12-08T08:39:40.373495+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373495+0000","last_undegraded":"2023-12-08T08:39:40.373495+0000","last_fullsized":"2023-12-08T08:39:40.373495+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-08T08:39:35.915241+0000","last_change":"2023-12-08T08:38:08.276815+0000","last_active":"2023-12-08T08:39:35.915241+0000","last_peered":"2023-12-08T08:39:35.915241+0000","last_clean":"2023-12-08T08:39:35.915241+0000","last_became_active":"2023-12-08T08:38:08.275828+0000","last_became_peered":"2023-12-08T08:38:08.275828+0000","last_unstale":"2023-12-08T08:39:35.915241+0000","last_undegraded":"2023-12-08T08:39:35.915241+0000","last_fullsized":"2023-12-08T08:39:35.915241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_clean_scrub_stamp":"2023-12-08T08:38:07.104197+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:35:34.851149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":764339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.629}]},{"osd":5,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.39500000000000002}]},{"osd":6,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.438},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":167834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.69099999999999995}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.61699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":173971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64000000000000001}]},{"osd":1,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.5},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.443},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.624},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.58299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":167834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.498},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.46400000000000002}]},{"osd":1,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42499999999999999}]},{"osd":5,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.441},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.39200000000000002}]},{"osd":6,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55900000000000005}]},{"osd":7,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":401408,"data_stored":167834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.30199999999999999}]},{"osd":1,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.24299999999999999}]},{"osd":2,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.317},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.32500000000000001}]},{"osd":4,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.628}]},{"osd":6,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.621},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.70599999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":173971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.34799999999999998}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.377},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.36399999999999999}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.63},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.55500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":173971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.313},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.373}]},{"osd":2,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.28000000000000003}]},{"osd":4,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.503},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.49199999999999999}]},{"osd":7,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":764339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.33300000000000002}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.316},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.46400000000000002}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.436}]},{"osd":6,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.38900000000000001}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.372}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-08T08:39:45.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-08T08:39:45.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.534+0000 7f0f52f78640 1 Processor -- start 2023-12-08T08:39:45.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.534+0000 7f0f52f78640 1 -- start start 2023-12-08T08:39:45.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f52f78640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f4c0ce3e0 0x7f0f4c0b1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f52f78640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f4c071e50 0x7f0f4c0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f52f78640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4c0b29b0 con 0x7f0f4c140a10 2023-12-08T08:39:45.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f52f78640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4c070ba0 con 0x7f0f4c071e50 2023-12-08T08:39:45.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f52f78640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c070db0 con 0x7f0f4c0ce3e0 2023-12-08T08:39:45.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f51515640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f4c071e50 0x7f0f4c0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f50d14640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0f4c140a10 0x7f0f4c0cdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51570/0 (socket says 172.21.15.16:51570) 2023-12-08T08:39:45.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f50d14640 1 -- 172.21.15.16:0/590863408 learned_addr learned my addr 172.21.15.16:0/590863408 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:45.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f4bfff640 1 --2- 172.21.15.16:0/590863408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f4c0ce3e0 0x7f0f4c0b1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f51515640 1 -- 172.21.15.16:0/590863408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f4c0ce3e0 msgr2=0x7f0f4c0b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:45.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f51515640 1 --2- 172.21.15.16:0/590863408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f4c0ce3e0 0x7f0f4c0b1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f51515640 1 -- 172.21.15.16:0/590863408 >> v1:172.21.15.16:6789/0 conn(0x7f0f4c140a10 legacy=0x7f0f4c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f4b7fe640 1 -- 172.21.15.16:0/590863408 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636736442 0 0) 0x7f0f4c0b29b0 con 0x7f0f4c140a10 2023-12-08T08:39:45.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f51515640 1 -- 172.21.15.16:0/590863408 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c071090 con 0x7f0f4c071e50 2023-12-08T08:39:45.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.535+0000 7f0f4bfff640 1 --2- 172.21.15.16:0/590863408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f4c0ce3e0 0x7f0f4c0b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.536+0000 7f0f51515640 1 --2- 172.21.15.16:0/590863408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c071e50 0x7f0f4c0b2470 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009d90 tx=0x7f0f4402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d25105c64177661 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.536+0000 7f0f4b7fe640 1 -- 172.21.15.16:0/590863408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4402cce0 con 0x7f0f4c071e50 2023-12-08T08:39:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.536+0000 7f0f4b7fe640 1 -- 172.21.15.16:0/590863408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4402ce40 con 0x7f0f4c071e50 2023-12-08T08:39:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.536+0000 7f0f4b7fe640 1 -- 172.21.15.16:0/590863408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44034660 con 0x7f0f4c071e50 2023-12-08T08:39:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.537+0000 7f0f52f78640 1 -- 172.21.15.16:0/590863408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c071e50 msgr2=0x7f0f4c0b2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.537+0000 7f0f52f78640 1 --2- 172.21.15.16:0/590863408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c071e50 0x7f0f4c0b2470 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009d90 tx=0x7f0f4402bf30 comp rx=0 tx=0).stop 2023-12-08T08:39:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.537+0000 7f0f52f78640 1 -- 172.21.15.16:0/590863408 shutdown_connections 2023-12-08T08:39:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.537+0000 7f0f52f78640 1 --2- 172.21.15.16:0/590863408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c071e50 0x7f0f4c0b2470 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.537+0000 7f0f52f78640 1 --2- 172.21.15.16:0/590863408 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f4c0ce3e0 0x7f0f4c0b1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.537+0000 7f0f52f78640 1 -- 172.21.15.16:0/590863408 >> 172.21.15.16:0/590863408 conn(0x7f0f4c0b09f0 msgr2=0x7f0f4c0b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:45.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.537+0000 7f0f52f78640 1 -- 172.21.15.16:0/590863408 shutdown_connections 2023-12-08T08:39:45.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.537+0000 7f0f52f78640 1 -- 172.21.15.16:0/590863408 wait complete. 2023-12-08T08:39:45.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.538+0000 7f0f52f78640 1 Processor -- start 2023-12-08T08:39:45.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.538+0000 7f0f52f78640 1 -- start start 2023-12-08T08:39:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.539+0000 7f0f52f78640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f4c071e50 0x7f0f4c142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.539+0000 7f0f52f78640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0ce3e0 0x7f0f4c142790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.539+0000 7f0f52f78640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c140a10 0x7f0f4c1f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.539+0000 7f0f52f78640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c13df70 con 0x7f0f4c140a10 2023-12-08T08:39:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.539+0000 7f0f52f78640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c0ace50 con 0x7f0f4c0ce3e0 2023-12-08T08:39:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.539+0000 7f0f52f78640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4c142d80 con 0x7f0f4c071e50 2023-12-08T08:39:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.539+0000 7f0f51515640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c140a10 0x7f0f4c1f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.539+0000 7f0f51515640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c140a10 0x7f0f4c1f33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34764/0 (socket says 172.21.15.16:34764) 2023-12-08T08:39:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.539+0000 7f0f51515640 1 -- 172.21.15.16:0/516313130 learned_addr learned my addr 172.21.15.16:0/516313130 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.539+0000 7f0f50d14640 1 --2- 172.21.15.16:0/516313130 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f4c071e50 0x7f0f4c142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.539+0000 7f0f4bfff640 1 --2- 172.21.15.16:0/516313130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0ce3e0 0x7f0f4c142790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.540+0000 7f0f51515640 1 -- 172.21.15.16:0/516313130 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f4c071e50 msgr2=0x7f0f4c142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.540+0000 7f0f51515640 1 --2- 172.21.15.16:0/516313130 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f4c071e50 0x7f0f4c142250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.540+0000 7f0f51515640 1 -- 172.21.15.16:0/516313130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0ce3e0 msgr2=0x7f0f4c142790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.540+0000 7f0f51515640 1 --2- 172.21.15.16:0/516313130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0ce3e0 0x7f0f4c142790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.540+0000 7f0f51515640 1 -- 172.21.15.16:0/516313130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f44009a70 con 0x7f0f4c140a10 2023-12-08T08:39:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.540+0000 7f0f51515640 1 --2- 172.21.15.16:0/516313130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c140a10 0x7f0f4c1f33e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0f44037040 tx=0x7f0f44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.540+0000 7f0f497fa640 1 -- 172.21.15.16:0/516313130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4403d600 con 0x7f0f4c140a10 2023-12-08T08:39:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.540+0000 7f0f497fa640 1 -- 172.21.15.16:0/516313130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4403d760 con 0x7f0f4c140a10 2023-12-08T08:39:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.540+0000 7f0f52f78640 1 -- 172.21.15.16:0/516313130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4c143060 con 0x7f0f4c140a10 2023-12-08T08:39:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.540+0000 7f0f497fa640 1 -- 172.21.15.16:0/516313130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4403dad0 con 0x7f0f4c140a10 2023-12-08T08:39:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.540+0000 7f0f52f78640 1 -- 172.21.15.16:0/516313130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4c143620 con 0x7f0f4c140a10 2023-12-08T08:39:45.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.541+0000 7f0f52f78640 1 -- 172.21.15.16:0/516313130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f4c067430 con 0x7f0f4c140a10 2023-12-08T08:39:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.541+0000 7f0f497fa640 1 -- 172.21.15.16:0/516313130 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0f4403b040 con 0x7f0f4c140a10 2023-12-08T08:39:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.542+0000 7f0f497fa640 1 --2- 172.21.15.16:0/516313130 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0f3806ae70 0x7f0f3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.542+0000 7f0f497fa640 1 -- 172.21.15.16:0/516313130 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0f440ab640 con 0x7f0f4c140a10 2023-12-08T08:39:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.542+0000 7f0f50d14640 1 --2- 172.21.15.16:0/516313130 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0f3806ae70 0x7f0f3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.543+0000 7f0f50d14640 1 --2- 172.21.15.16:0/516313130 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0f3806ae70 0x7f0f3806d330 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c007fd0 tx=0x7f0f3c007de0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.546+0000 7f0f497fa640 1 -- 172.21.15.16:0/516313130 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f440765c0 con 0x7f0f4c140a10 2023-12-08T08:39:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.718+0000 7f0f52f78640 1 -- 172.21.15.16:0/516313130 --> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f4c071930 con 0x7f0f3806ae70 2023-12-08T08:39:45.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.725+0000 7f0f497fa640 1 -- 172.21.15.16:0/516313130 <== mgr.4111 v2:172.21.15.112:6832/804067392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277533 (secure 0 0 0) 0x7f0f3c004c70 con 0x7f0f3806ae70 2023-12-08T08:39:45.730 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-12-08T08:39:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.728+0000 7f0f52f78640 1 -- 172.21.15.16:0/516313130 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0f3806ae70 msgr2=0x7f0f3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.728+0000 7f0f52f78640 1 --2- 172.21.15.16:0/516313130 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0f3806ae70 0x7f0f3806d330 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c007fd0 tx=0x7f0f3c007de0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.728+0000 7f0f52f78640 1 -- 172.21.15.16:0/516313130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c140a10 msgr2=0x7f0f4c1f33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.728+0000 7f0f52f78640 1 --2- 172.21.15.16:0/516313130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c140a10 0x7f0f4c1f33e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0f44037040 tx=0x7f0f44004a40 comp rx=0 tx=0).stop 2023-12-08T08:39:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.729+0000 7f0f52f78640 1 -- 172.21.15.16:0/516313130 shutdown_connections 2023-12-08T08:39:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.729+0000 7f0f52f78640 1 --2- 172.21.15.16:0/516313130 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f0f3806ae70 0x7f0f3806d330 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.729+0000 7f0f52f78640 1 --2- 172.21.15.16:0/516313130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f4c140a10 0x7f0f4c1f33e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.729+0000 7f0f52f78640 1 --2- 172.21.15.16:0/516313130 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f0f4c0ce3e0 0x7f0f4c142790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.729+0000 7f0f52f78640 1 --2- 172.21.15.16:0/516313130 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f0f4c071e50 0x7f0f4c142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.729+0000 7f0f52f78640 1 -- 172.21.15.16:0/516313130 >> 172.21.15.16:0/516313130 conn(0x7f0f4c0b09f0 msgr2=0x7f0f4c159c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:45.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.729+0000 7f0f52f78640 1 -- 172.21.15.16:0/516313130 shutdown_connections 2023-12-08T08:39:45.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.729+0000 7f0f52f78640 1 -- 172.21.15.16:0/516313130 wait complete. 2023-12-08T08:39:45.741 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2023-12-08T08:39:44.996436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233016,"kb_used_data":4356,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741704,"statfs":{"total":773094113280,"available":772855504896,"internally_reserved":0,"allocated":4460544,"data_stored":2554093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.629075"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372184+0000","last_change":"2023-12-08T08:39:40.372184+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372184+0000","last_undegraded":"2023-12-08T08:39:40.372184+0000","last_fullsized":"2023-12-08T08:39:40.372184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372016+0000","last_change":"2023-12-08T08:39:40.372016+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372016+0000","last_undegraded":"2023-12-08T08:39:40.372016+0000","last_fullsized":"2023-12-08T08:39:40.372016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375234+0000","last_change":"2023-12-08T08:39:40.375234+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375234+0000","last_undegraded":"2023-12-08T08:39:40.375234+0000","last_fullsized":"2023-12-08T08:39:40.375234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369509+0000","last_change":"2023-12-08T08:39:40.369509+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369509+0000","last_undegraded":"2023-12-08T08:39:40.369509+0000","last_fullsized":"2023-12-08T08:39:40.369509+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371677+0000","last_change":"2023-12-08T08:39:40.371677+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371677+0000","last_undegraded":"2023-12-08T08:39:40.371677+0000","last_fullsized":"2023-12-08T08:39:40.371677+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373183+0000","last_change":"2023-12-08T08:39:40.373183+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373183+0000","last_undegraded":"2023-12-08T08:39:40.373183+0000","last_fullsized":"2023-12-08T08:39:40.373183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375447+0000","last_change":"2023-12-08T08:39:40.375447+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375447+0000","last_undegraded":"2023-12-08T08:39:40.375447+0000","last_fullsized":"2023-12-08T08:39:40.375447+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369561+0000","last_change":"2023-12-08T08:39:40.369561+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369561+0000","last_undegraded":"2023-12-08T08:39:40.369561+0000","last_fullsized":"2023-12-08T08:39:40.369561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372358+0000","last_change":"2023-12-08T08:39:40.372358+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372358+0000","last_undegraded":"2023-12-08T08:39:40.372358+0000","last_fullsized":"2023-12-08T08:39:40.372358+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371732+0000","last_change":"2023-12-08T08:39:40.371732+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371732+0000","last_undegraded":"2023-12-08T08:39:40.371732+0000","last_fullsized":"2023-12-08T08:39:40.371732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371718+0000","last_change":"2023-12-08T08:39:40.371718+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371718+0000","last_undegraded":"2023-12-08T08:39:40.371718+0000","last_fullsized":"2023-12-08T08:39:40.371718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371956+0000","last_change":"2023-12-08T08:39:40.371956+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371956+0000","last_undegraded":"2023-12-08T08:39:40.371956+0000","last_fullsized":"2023-12-08T08:39:40.371956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372089+0000","last_change":"2023-12-08T08:39:40.372089+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372089+0000","last_undegraded":"2023-12-08T08:39:40.372089+0000","last_fullsized":"2023-12-08T08:39:40.372089+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375171+0000","last_change":"2023-12-08T08:39:40.375171+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375171+0000","last_undegraded":"2023-12-08T08:39:40.375171+0000","last_fullsized":"2023-12-08T08:39:40.375171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371915+0000","last_change":"2023-12-08T08:39:40.371915+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371915+0000","last_undegraded":"2023-12-08T08:39:40.371915+0000","last_fullsized":"2023-12-08T08:39:40.371915+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373260+0000","last_change":"2023-12-08T08:39:40.373260+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373260+0000","last_undegraded":"2023-12-08T08:39:40.373260+0000","last_fullsized":"2023-12-08T08:39:40.373260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372660+0000","last_change":"2023-12-08T08:39:40.372660+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372660+0000","last_undegraded":"2023-12-08T08:39:40.372660+0000","last_fullsized":"2023-12-08T08:39:40.372660+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369563+0000","last_change":"2023-12-08T08:39:40.369563+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369563+0000","last_undegraded":"2023-12-08T08:39:40.369563+0000","last_fullsized":"2023-12-08T08:39:40.369563+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.370714+0000","last_change":"2023-12-08T08:39:40.370714+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.370714+0000","last_undegraded":"2023-12-08T08:39:40.370714+0000","last_fullsized":"2023-12-08T08:39:40.370714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375212+0000","last_change":"2023-12-08T08:39:40.375212+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375212+0000","last_undegraded":"2023-12-08T08:39:40.375212+0000","last_fullsized":"2023-12-08T08:39:40.375212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.370162+0000","last_change":"2023-12-08T08:39:40.370162+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.370162+0000","last_undegraded":"2023-12-08T08:39:40.370162+0000","last_fullsized":"2023-12-08T08:39:40.370162+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.370100+0000","last_change":"2023-12-08T08:39:40.370100+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.370100+0000","last_undegraded":"2023-12-08T08:39:40.370100+0000","last_fullsized":"2023-12-08T08:39:40.370100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373451+0000","last_change":"2023-12-08T08:39:40.373451+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373451+0000","last_undegraded":"2023-12-08T08:39:40.373451+0000","last_fullsized":"2023-12-08T08:39:40.373451+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369727+0000","last_change":"2023-12-08T08:39:40.369727+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369727+0000","last_undegraded":"2023-12-08T08:39:40.369727+0000","last_fullsized":"2023-12-08T08:39:40.369727+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372229+0000","last_change":"2023-12-08T08:39:40.372229+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372229+0000","last_undegraded":"2023-12-08T08:39:40.372229+0000","last_fullsized":"2023-12-08T08:39:40.372229+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369727+0000","last_change":"2023-12-08T08:39:40.369727+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369727+0000","last_undegraded":"2023-12-08T08:39:40.369727+0000","last_fullsized":"2023-12-08T08:39:40.369727+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372041+0000","last_change":"2023-12-08T08:39:40.372041+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372041+0000","last_undegraded":"2023-12-08T08:39:40.372041+0000","last_fullsized":"2023-12-08T08:39:40.372041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372086+0000","last_change":"2023-12-08T08:39:40.372086+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372086+0000","last_undegraded":"2023-12-08T08:39:40.372086+0000","last_fullsized":"2023-12-08T08:39:40.372086+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371953+0000","last_change":"2023-12-08T08:39:40.371953+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371953+0000","last_undegraded":"2023-12-08T08:39:40.371953+0000","last_fullsized":"2023-12-08T08:39:40.371953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369539+0000","last_change":"2023-12-08T08:39:40.369539+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369539+0000","last_undegraded":"2023-12-08T08:39:40.369539+0000","last_fullsized":"2023-12-08T08:39:40.369539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371795+0000","last_change":"2023-12-08T08:39:40.371795+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371795+0000","last_undegraded":"2023-12-08T08:39:40.371795+0000","last_fullsized":"2023-12-08T08:39:40.371795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373495+0000","last_change":"2023-12-08T08:39:40.373495+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373495+0000","last_undegraded":"2023-12-08T08:39:40.373495+0000","last_fullsized":"2023-12-08T08:39:40.373495+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-08T08:39:35.915241+0000","last_change":"2023-12-08T08:38:08.276815+0000","last_active":"2023-12-08T08:39:35.915241+0000","last_peered":"2023-12-08T08:39:35.915241+0000","last_clean":"2023-12-08T08:39:35.915241+0000","last_became_active":"2023-12-08T08:38:08.275828+0000","last_became_peered":"2023-12-08T08:38:08.275828+0000","last_unstale":"2023-12-08T08:39:35.915241+0000","last_undegraded":"2023-12-08T08:39:35.915241+0000","last_fullsized":"2023-12-08T08:39:35.915241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_clean_scrub_stamp":"2023-12-08T08:38:07.104197+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:35:34.851149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":764339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.629}]},{"osd":5,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.39500000000000002}]},{"osd":6,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.438},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":167834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.69099999999999995}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.61699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":173971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64000000000000001}]},{"osd":1,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.5},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.443},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.624},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.58299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":167834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.498},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.46400000000000002}]},{"osd":1,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42499999999999999}]},{"osd":5,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.441},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.39200000000000002}]},{"osd":6,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55900000000000005}]},{"osd":7,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":401408,"data_stored":167834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.30199999999999999}]},{"osd":1,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.24299999999999999}]},{"osd":2,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.317},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.32500000000000001}]},{"osd":4,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.628}]},{"osd":6,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.621},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.70599999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":173971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.34799999999999998}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.377},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.36399999999999999}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.63},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.55500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":173971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.313},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.373}]},{"osd":2,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.28000000000000003}]},{"osd":4,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.503},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.49199999999999999}]},{"osd":7,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":764339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.33300000000000002}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.316},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.46400000000000002}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.436}]},{"osd":6,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.38900000000000001}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.372}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-08T08:39:45.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-08T08:39:45.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.887+0000 7f7823409640 1 Processor -- start 2023-12-08T08:39:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.887+0000 7f7823409640 1 -- start start 2023-12-08T08:39:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.888+0000 7f7823409640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f781c0ce270 0x7f781c06bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.888+0000 7f7823409640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f781c06c480 0x7f781c15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.888+0000 7f7823409640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f781c0ce6f0 con 0x7f781c0ccd70 2023-12-08T08:39:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.888+0000 7f7823409640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f781c0d0a60 con 0x7f781c0ce270 2023-12-08T08:39:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.888+0000 7f7823409640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781c06c910 con 0x7f781c06c480 2023-12-08T08:39:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.888+0000 7f78209a4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f781c0ce270 0x7f781c06bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.888+0000 7f78209a4640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f781c0ce270 0x7f781c06bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34766/0 (socket says 172.21.15.16:34766) 2023-12-08T08:39:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.888+0000 7f78209a4640 1 -- 172.21.15.16:0/2958734071 learned_addr learned my addr 172.21.15.16:0/2958734071 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.888+0000 7f78219a6640 1 --2- 172.21.15.16:0/2958734071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f781c06c480 0x7f781c15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.888+0000 7f78209a4640 1 -- 172.21.15.16:0/2958734071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f781c06c480 msgr2=0x7f781c15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:45.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.888+0000 7f78209a4640 1 --2- 172.21.15.16:0/2958734071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f781c06c480 0x7f781c15ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.888+0000 7f78209a4640 1 -- 172.21.15.16:0/2958734071 >> v1:172.21.15.112:6789/0 conn(0x7f781c0ccd70 legacy=0x7f781c0cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.888+0000 7f78209a4640 1 -- 172.21.15.16:0/2958734071 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c0cf9c0 con 0x7f781c0ce270 2023-12-08T08:39:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.889+0000 7f78209a4640 1 --2- 172.21.15.16:0/2958734071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f781c0ce270 0x7f781c06bf40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f78100098a0 tx=0x7f781002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8597a8633903aa31 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.889+0000 7f781bfff640 1 -- 172.21.15.16:0/2958734071 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810030840 con 0x7f781c0ce270 2023-12-08T08:39:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.889+0000 7f781bfff640 1 -- 172.21.15.16:0/2958734071 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78100309a0 con 0x7f781c0ce270 2023-12-08T08:39:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.889+0000 7f781bfff640 1 -- 172.21.15.16:0/2958734071 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810030d30 con 0x7f781c0ce270 2023-12-08T08:39:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.890+0000 7f7823409640 1 -- 172.21.15.16:0/2958734071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f781c0ce270 msgr2=0x7f781c06bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.890+0000 7f7823409640 1 --2- 172.21.15.16:0/2958734071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f781c0ce270 0x7f781c06bf40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f78100098a0 tx=0x7f781002b980 comp rx=0 tx=0).stop 2023-12-08T08:39:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.890+0000 7f7823409640 1 -- 172.21.15.16:0/2958734071 shutdown_connections 2023-12-08T08:39:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.890+0000 7f7823409640 1 --2- 172.21.15.16:0/2958734071 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f781c06c480 0x7f781c15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.890+0000 7f7823409640 1 --2- 172.21.15.16:0/2958734071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f781c0ce270 0x7f781c06bf40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.890+0000 7f7823409640 1 -- 172.21.15.16:0/2958734071 >> 172.21.15.16:0/2958734071 conn(0x7f781c0d0de0 msgr2=0x7f781c0d11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.890+0000 7f7823409640 1 -- 172.21.15.16:0/2958734071 shutdown_connections 2023-12-08T08:39:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.890+0000 7f7823409640 1 -- 172.21.15.16:0/2958734071 wait complete. 2023-12-08T08:39:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.891+0000 7f7823409640 1 Processor -- start 2023-12-08T08:39:45.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.891+0000 7f7823409640 1 -- start start 2023-12-08T08:39:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f7823409640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f781c06c480 0x7f781c0b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f7823409640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f781c0ccd70 0x7f781c0b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f7823409640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f781c0ce270 0x7f781c0b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f7823409640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781c0b65c0 con 0x7f781c0ccd70 2023-12-08T08:39:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f7823409640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781c0b67b0 con 0x7f781c0ce270 2023-12-08T08:39:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f7823409640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781c0b6a90 con 0x7f781c06c480 2023-12-08T08:39:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f78209a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f781c0ccd70 0x7f781c0b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f78209a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f781c0ccd70 0x7f781c0b6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34772/0 (socket says 172.21.15.16:34772) 2023-12-08T08:39:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f78209a4640 1 -- 172.21.15.16:0/4115133956 learned_addr learned my addr 172.21.15.16:0/4115133956 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f78211a5640 1 --2- 172.21.15.16:0/4115133956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f781c06c480 0x7f781c0b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f78219a6640 1 --2- 172.21.15.16:0/4115133956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f781c0ce270 0x7f781c0b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f78209a4640 1 -- 172.21.15.16:0/4115133956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f781c06c480 msgr2=0x7f781c0b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f78209a4640 1 --2- 172.21.15.16:0/4115133956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f781c06c480 0x7f781c0b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f78209a4640 1 -- 172.21.15.16:0/4115133956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f781c0ce270 msgr2=0x7f781c0b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f78209a4640 1 --2- 172.21.15.16:0/4115133956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f781c0ce270 0x7f781c0b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.892+0000 7f78209a4640 1 -- 172.21.15.16:0/4115133956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78100094d0 con 0x7f781c0ccd70 2023-12-08T08:39:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.893+0000 7f78209a4640 1 --2- 172.21.15.16:0/4115133956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f781c0ccd70 0x7f781c0b6080 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7810000c00 tx=0x7f7810004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.893+0000 7f7819ffb640 1 -- 172.21.15.16:0/4115133956 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810039a90 con 0x7f781c0ccd70 2023-12-08T08:39:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.893+0000 7f7819ffb640 1 -- 172.21.15.16:0/4115133956 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7810039c10 con 0x7f781c0ccd70 2023-12-08T08:39:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.893+0000 7f7823409640 1 -- 172.21.15.16:0/4115133956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781c09ac20 con 0x7f781c0ccd70 2023-12-08T08:39:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.893+0000 7f7819ffb640 1 -- 172.21.15.16:0/4115133956 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810043410 con 0x7f781c0ccd70 2023-12-08T08:39:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.893+0000 7f7823409640 1 -- 172.21.15.16:0/4115133956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781c09b1e0 con 0x7f781c0ccd70 2023-12-08T08:39:45.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.894+0000 7f7823409640 1 -- 172.21.15.16:0/4115133956 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f781c09b3d0 con 0x7f781c0ccd70 2023-12-08T08:39:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.894+0000 7f7819ffb640 1 -- 172.21.15.16:0/4115133956 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f781004c3b0 con 0x7f781c0ccd70 2023-12-08T08:39:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.895+0000 7f7819ffb640 1 --2- 172.21.15.16:0/4115133956 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f780806ae40 0x7f780806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.895+0000 7f7819ffb640 1 -- 172.21.15.16:0/4115133956 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f78100abaa0 con 0x7f781c0ccd70 2023-12-08T08:39:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.895+0000 7f78211a5640 1 --2- 172.21.15.16:0/4115133956 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f780806ae40 0x7f780806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.896+0000 7f78211a5640 1 --2- 172.21.15.16:0/4115133956 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f780806ae40 0x7f780806d300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f780c0012e0 tx=0x7f780c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:45.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:45.899+0000 7f7819ffb640 1 -- 172.21.15.16:0/4115133956 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7810076b50 con 0x7f781c0ccd70 2023-12-08T08:39:46.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.072+0000 7f7823409640 1 -- 172.21.15.16:0/4115133956 --> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f781c0d0260 con 0x7f780806ae40 2023-12-08T08:39:46.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.079+0000 7f7819ffb640 1 -- 172.21.15.16:0/4115133956 <== mgr.4111 v2:172.21.15.112:6832/804067392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277533 (secure 0 0 0) 0x7f780c00fe00 con 0x7f780806ae40 2023-12-08T08:39:46.084 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:46.085 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-12-08T08:39:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.082+0000 7f7823409640 1 -- 172.21.15.16:0/4115133956 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f780806ae40 msgr2=0x7f780806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:46.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.082+0000 7f7823409640 1 --2- 172.21.15.16:0/4115133956 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f780806ae40 0x7f780806d300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f780c0012e0 tx=0x7f780c009040 comp rx=0 tx=0).stop 2023-12-08T08:39:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.082+0000 7f7823409640 1 -- 172.21.15.16:0/4115133956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f781c0ccd70 msgr2=0x7f781c0b6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.082+0000 7f7823409640 1 --2- 172.21.15.16:0/4115133956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f781c0ccd70 0x7f781c0b6080 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7810000c00 tx=0x7f7810004730 comp rx=0 tx=0).stop 2023-12-08T08:39:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.082+0000 7f7823409640 1 -- 172.21.15.16:0/4115133956 shutdown_connections 2023-12-08T08:39:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.082+0000 7f7823409640 1 --2- 172.21.15.16:0/4115133956 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f780806ae40 0x7f780806d300 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.082+0000 7f7823409640 1 --2- 172.21.15.16:0/4115133956 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f781c0ce270 0x7f781c0b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.082+0000 7f7823409640 1 --2- 172.21.15.16:0/4115133956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f781c0ccd70 0x7f781c0b6080 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.082+0000 7f7823409640 1 --2- 172.21.15.16:0/4115133956 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f781c06c480 0x7f781c0b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.082+0000 7f7823409640 1 -- 172.21.15.16:0/4115133956 >> 172.21.15.16:0/4115133956 conn(0x7f781c0d0de0 msgr2=0x7f781c0b0480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.082+0000 7f7823409640 1 -- 172.21.15.16:0/4115133956 shutdown_connections 2023-12-08T08:39:46.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.082+0000 7f7823409640 1 -- 172.21.15.16:0/4115133956 wait complete. 2023-12-08T08:39:46.095 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2023-12-08T08:39:44.996436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233016,"kb_used_data":4356,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741704,"statfs":{"total":773094113280,"available":772855504896,"internally_reserved":0,"allocated":4460544,"data_stored":2554093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.629075"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372184+0000","last_change":"2023-12-08T08:39:40.372184+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372184+0000","last_undegraded":"2023-12-08T08:39:40.372184+0000","last_fullsized":"2023-12-08T08:39:40.372184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372016+0000","last_change":"2023-12-08T08:39:40.372016+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372016+0000","last_undegraded":"2023-12-08T08:39:40.372016+0000","last_fullsized":"2023-12-08T08:39:40.372016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375234+0000","last_change":"2023-12-08T08:39:40.375234+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375234+0000","last_undegraded":"2023-12-08T08:39:40.375234+0000","last_fullsized":"2023-12-08T08:39:40.375234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369509+0000","last_change":"2023-12-08T08:39:40.369509+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369509+0000","last_undegraded":"2023-12-08T08:39:40.369509+0000","last_fullsized":"2023-12-08T08:39:40.369509+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371677+0000","last_change":"2023-12-08T08:39:40.371677+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371677+0000","last_undegraded":"2023-12-08T08:39:40.371677+0000","last_fullsized":"2023-12-08T08:39:40.371677+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373183+0000","last_change":"2023-12-08T08:39:40.373183+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373183+0000","last_undegraded":"2023-12-08T08:39:40.373183+0000","last_fullsized":"2023-12-08T08:39:40.373183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375447+0000","last_change":"2023-12-08T08:39:40.375447+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375447+0000","last_undegraded":"2023-12-08T08:39:40.375447+0000","last_fullsized":"2023-12-08T08:39:40.375447+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369561+0000","last_change":"2023-12-08T08:39:40.369561+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369561+0000","last_undegraded":"2023-12-08T08:39:40.369561+0000","last_fullsized":"2023-12-08T08:39:40.369561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372358+0000","last_change":"2023-12-08T08:39:40.372358+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372358+0000","last_undegraded":"2023-12-08T08:39:40.372358+0000","last_fullsized":"2023-12-08T08:39:40.372358+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371732+0000","last_change":"2023-12-08T08:39:40.371732+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371732+0000","last_undegraded":"2023-12-08T08:39:40.371732+0000","last_fullsized":"2023-12-08T08:39:40.371732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371718+0000","last_change":"2023-12-08T08:39:40.371718+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371718+0000","last_undegraded":"2023-12-08T08:39:40.371718+0000","last_fullsized":"2023-12-08T08:39:40.371718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371956+0000","last_change":"2023-12-08T08:39:40.371956+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371956+0000","last_undegraded":"2023-12-08T08:39:40.371956+0000","last_fullsized":"2023-12-08T08:39:40.371956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372089+0000","last_change":"2023-12-08T08:39:40.372089+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372089+0000","last_undegraded":"2023-12-08T08:39:40.372089+0000","last_fullsized":"2023-12-08T08:39:40.372089+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375171+0000","last_change":"2023-12-08T08:39:40.375171+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375171+0000","last_undegraded":"2023-12-08T08:39:40.375171+0000","last_fullsized":"2023-12-08T08:39:40.375171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371915+0000","last_change":"2023-12-08T08:39:40.371915+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371915+0000","last_undegraded":"2023-12-08T08:39:40.371915+0000","last_fullsized":"2023-12-08T08:39:40.371915+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373260+0000","last_change":"2023-12-08T08:39:40.373260+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373260+0000","last_undegraded":"2023-12-08T08:39:40.373260+0000","last_fullsized":"2023-12-08T08:39:40.373260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372660+0000","last_change":"2023-12-08T08:39:40.372660+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372660+0000","last_undegraded":"2023-12-08T08:39:40.372660+0000","last_fullsized":"2023-12-08T08:39:40.372660+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369563+0000","last_change":"2023-12-08T08:39:40.369563+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369563+0000","last_undegraded":"2023-12-08T08:39:40.369563+0000","last_fullsized":"2023-12-08T08:39:40.369563+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.370714+0000","last_change":"2023-12-08T08:39:40.370714+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.370714+0000","last_undegraded":"2023-12-08T08:39:40.370714+0000","last_fullsized":"2023-12-08T08:39:40.370714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375212+0000","last_change":"2023-12-08T08:39:40.375212+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375212+0000","last_undegraded":"2023-12-08T08:39:40.375212+0000","last_fullsized":"2023-12-08T08:39:40.375212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.370162+0000","last_change":"2023-12-08T08:39:40.370162+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.370162+0000","last_undegraded":"2023-12-08T08:39:40.370162+0000","last_fullsized":"2023-12-08T08:39:40.370162+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.370100+0000","last_change":"2023-12-08T08:39:40.370100+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.370100+0000","last_undegraded":"2023-12-08T08:39:40.370100+0000","last_fullsized":"2023-12-08T08:39:40.370100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373451+0000","last_change":"2023-12-08T08:39:40.373451+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373451+0000","last_undegraded":"2023-12-08T08:39:40.373451+0000","last_fullsized":"2023-12-08T08:39:40.373451+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369727+0000","last_change":"2023-12-08T08:39:40.369727+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369727+0000","last_undegraded":"2023-12-08T08:39:40.369727+0000","last_fullsized":"2023-12-08T08:39:40.369727+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372229+0000","last_change":"2023-12-08T08:39:40.372229+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372229+0000","last_undegraded":"2023-12-08T08:39:40.372229+0000","last_fullsized":"2023-12-08T08:39:40.372229+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369727+0000","last_change":"2023-12-08T08:39:40.369727+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369727+0000","last_undegraded":"2023-12-08T08:39:40.369727+0000","last_fullsized":"2023-12-08T08:39:40.369727+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372041+0000","last_change":"2023-12-08T08:39:40.372041+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372041+0000","last_undegraded":"2023-12-08T08:39:40.372041+0000","last_fullsized":"2023-12-08T08:39:40.372041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372086+0000","last_change":"2023-12-08T08:39:40.372086+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372086+0000","last_undegraded":"2023-12-08T08:39:40.372086+0000","last_fullsized":"2023-12-08T08:39:40.372086+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371953+0000","last_change":"2023-12-08T08:39:40.371953+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371953+0000","last_undegraded":"2023-12-08T08:39:40.371953+0000","last_fullsized":"2023-12-08T08:39:40.371953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369539+0000","last_change":"2023-12-08T08:39:40.369539+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369539+0000","last_undegraded":"2023-12-08T08:39:40.369539+0000","last_fullsized":"2023-12-08T08:39:40.369539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371795+0000","last_change":"2023-12-08T08:39:40.371795+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371795+0000","last_undegraded":"2023-12-08T08:39:40.371795+0000","last_fullsized":"2023-12-08T08:39:40.371795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373495+0000","last_change":"2023-12-08T08:39:40.373495+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373495+0000","last_undegraded":"2023-12-08T08:39:40.373495+0000","last_fullsized":"2023-12-08T08:39:40.373495+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-08T08:39:35.915241+0000","last_change":"2023-12-08T08:38:08.276815+0000","last_active":"2023-12-08T08:39:35.915241+0000","last_peered":"2023-12-08T08:39:35.915241+0000","last_clean":"2023-12-08T08:39:35.915241+0000","last_became_active":"2023-12-08T08:38:08.275828+0000","last_became_peered":"2023-12-08T08:38:08.275828+0000","last_unstale":"2023-12-08T08:39:35.915241+0000","last_undegraded":"2023-12-08T08:39:35.915241+0000","last_fullsized":"2023-12-08T08:39:35.915241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_clean_scrub_stamp":"2023-12-08T08:38:07.104197+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:35:34.851149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":764339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.629}]},{"osd":5,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.39500000000000002}]},{"osd":6,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.438},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":167834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.69099999999999995}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.61699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":173971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64000000000000001}]},{"osd":1,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.5},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.443},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.624},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.58299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":167834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.498},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.46400000000000002}]},{"osd":1,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42499999999999999}]},{"osd":5,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.441},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.39200000000000002}]},{"osd":6,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55900000000000005}]},{"osd":7,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":401408,"data_stored":167834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.30199999999999999}]},{"osd":1,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.24299999999999999}]},{"osd":2,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.317},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.32500000000000001}]},{"osd":4,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.628}]},{"osd":6,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.621},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.70599999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":173971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.34799999999999998}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.377},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.36399999999999999}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.63},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.55500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":173971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.313},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.373}]},{"osd":2,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.28000000000000003}]},{"osd":4,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.503},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.49199999999999999}]},{"osd":7,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":764339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.33300000000000002}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.316},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.46400000000000002}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.436}]},{"osd":6,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.38900000000000001}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.372}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-08T08:39:46.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2023-12-08T08:39:46.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2023-12-08T08:39:46.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2023-12-08T08:39:46.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2023-12-08T08:39:46.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2023-12-08T08:39:46.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2023-12-08T08:39:46.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2023-12-08T08:39:46.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2023-12-08T08:39:46.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2023-12-08T08:39:46.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2023-12-08T08:39:46.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2023-12-08T08:39:46.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2023-12-08T08:39:46.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2023-12-08T08:39:46.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2023-12-08T08:39:46.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2023-12-08T08:39:46.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2023-12-08T08:39:46.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2023-12-08T08:39:46.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2023-12-08T08:39:46.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2023-12-08T08:39:46.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2023-12-08T08:39:46.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2023-12-08T08:39:46.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2023-12-08T08:39:46.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2023-12-08T08:39:46.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2023-12-08T08:39:46.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2023-12-08T08:39:46.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2023-12-08T08:39:46.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2023-12-08T08:39:46.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2023-12-08T08:39:46.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2023-12-08T08:39:46.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2023-12-08T08:39:46.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2023-12-08T08:39:46.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2023-12-08T08:39:46.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2023-12-08T08:39:46.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-12-08T08:39:46.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.372184+0000', 'last_change': '2023-12-08T08:39:40.372184+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.372184+0000', 'last_undegraded': '2023-12-08T08:39:40.372184+0000', 'last_fullsized': '2023-12-08T08:39:40.372184+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-08T08:39:46.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2023-12-08T08:39:46.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-12-08T08:39:46.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.372016+0000', 'last_change': '2023-12-08T08:39:40.372016+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.372016+0000', 'last_undegraded': '2023-12-08T08:39:40.372016+0000', 'last_fullsized': '2023-12-08T08:39:40.372016+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-08T08:39:46.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-12-08T08:39:46.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-12-08T08:39:46.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.375234+0000', 'last_change': '2023-12-08T08:39:40.375234+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.375234+0000', 'last_undegraded': '2023-12-08T08:39:40.375234+0000', 'last_fullsized': '2023-12-08T08:39:40.375234+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-08T08:39:46.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2023-12-08T08:39:46.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-12-08T08:39:46.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.369509+0000', 'last_change': '2023-12-08T08:39:40.369509+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.369509+0000', 'last_undegraded': '2023-12-08T08:39:40.369509+0000', 'last_fullsized': '2023-12-08T08:39:40.369509+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-08T08:39:46.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2023-12-08T08:39:46.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-12-08T08:39:46.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.371677+0000', 'last_change': '2023-12-08T08:39:40.371677+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.371677+0000', 'last_undegraded': '2023-12-08T08:39:40.371677+0000', 'last_fullsized': '2023-12-08T08:39:40.371677+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-08T08:39:46.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2023-12-08T08:39:46.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-12-08T08:39:46.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.373183+0000', 'last_change': '2023-12-08T08:39:40.373183+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.373183+0000', 'last_undegraded': '2023-12-08T08:39:40.373183+0000', 'last_fullsized': '2023-12-08T08:39:40.373183+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-08T08:39:46.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-12-08T08:39:46.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-12-08T08:39:46.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.375447+0000', 'last_change': '2023-12-08T08:39:40.375447+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.375447+0000', 'last_undegraded': '2023-12-08T08:39:40.375447+0000', 'last_fullsized': '2023-12-08T08:39:40.375447+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-08T08:39:46.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2023-12-08T08:39:46.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-12-08T08:39:46.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.369561+0000', 'last_change': '2023-12-08T08:39:40.369561+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.369561+0000', 'last_undegraded': '2023-12-08T08:39:40.369561+0000', 'last_fullsized': '2023-12-08T08:39:40.369561+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-08T08:39:46.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2023-12-08T08:39:46.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-12-08T08:39:46.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.372358+0000', 'last_change': '2023-12-08T08:39:40.372358+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.372358+0000', 'last_undegraded': '2023-12-08T08:39:40.372358+0000', 'last_fullsized': '2023-12-08T08:39:40.372358+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-08T08:39:46.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2023-12-08T08:39:46.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-12-08T08:39:46.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.371732+0000', 'last_change': '2023-12-08T08:39:40.371732+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.371732+0000', 'last_undegraded': '2023-12-08T08:39:40.371732+0000', 'last_fullsized': '2023-12-08T08:39:40.371732+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-08T08:39:46.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2023-12-08T08:39:46.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-12-08T08:39:46.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.371718+0000', 'last_change': '2023-12-08T08:39:40.371718+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.371718+0000', 'last_undegraded': '2023-12-08T08:39:40.371718+0000', 'last_fullsized': '2023-12-08T08:39:40.371718+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-08T08:39:46.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-12-08T08:39:46.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-12-08T08:39:46.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.371956+0000', 'last_change': '2023-12-08T08:39:40.371956+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.371956+0000', 'last_undegraded': '2023-12-08T08:39:40.371956+0000', 'last_fullsized': '2023-12-08T08:39:40.371956+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-08T08:39:46.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2023-12-08T08:39:46.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.109 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-12-08T08:39:46.109 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.372089+0000', 'last_change': '2023-12-08T08:39:40.372089+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.372089+0000', 'last_undegraded': '2023-12-08T08:39:40.372089+0000', 'last_fullsized': '2023-12-08T08:39:40.372089+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-08T08:39:46.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2023-12-08T08:39:46.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-12-08T08:39:46.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.375171+0000', 'last_change': '2023-12-08T08:39:40.375171+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.375171+0000', 'last_undegraded': '2023-12-08T08:39:40.375171+0000', 'last_fullsized': '2023-12-08T08:39:40.375171+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-08T08:39:46.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2023-12-08T08:39:46.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-12-08T08:39:46.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.371915+0000', 'last_change': '2023-12-08T08:39:40.371915+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.371915+0000', 'last_undegraded': '2023-12-08T08:39:40.371915+0000', 'last_fullsized': '2023-12-08T08:39:40.371915+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-08T08:39:46.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2023-12-08T08:39:46.110 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.110 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-12-08T08:39:46.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.373260+0000', 'last_change': '2023-12-08T08:39:40.373260+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.373260+0000', 'last_undegraded': '2023-12-08T08:39:40.373260+0000', 'last_fullsized': '2023-12-08T08:39:40.373260+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-08T08:39:46.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2023-12-08T08:39:46.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-12-08T08:39:46.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.372660+0000', 'last_change': '2023-12-08T08:39:40.372660+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.372660+0000', 'last_undegraded': '2023-12-08T08:39:40.372660+0000', 'last_fullsized': '2023-12-08T08:39:40.372660+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-08T08:39:46.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2023-12-08T08:39:46.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-12-08T08:39:46.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.369563+0000', 'last_change': '2023-12-08T08:39:40.369563+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.369563+0000', 'last_undegraded': '2023-12-08T08:39:40.369563+0000', 'last_fullsized': '2023-12-08T08:39:40.369563+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-08T08:39:46.111 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-12-08T08:39:46.111 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-12-08T08:39:46.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.370714+0000', 'last_change': '2023-12-08T08:39:40.370714+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.370714+0000', 'last_undegraded': '2023-12-08T08:39:40.370714+0000', 'last_fullsized': '2023-12-08T08:39:40.370714+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-08T08:39:46.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2023-12-08T08:39:46.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-12-08T08:39:46.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.375212+0000', 'last_change': '2023-12-08T08:39:40.375212+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.375212+0000', 'last_undegraded': '2023-12-08T08:39:40.375212+0000', 'last_fullsized': '2023-12-08T08:39:40.375212+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-08T08:39:46.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2023-12-08T08:39:46.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-12-08T08:39:46.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.370162+0000', 'last_change': '2023-12-08T08:39:40.370162+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.370162+0000', 'last_undegraded': '2023-12-08T08:39:40.370162+0000', 'last_fullsized': '2023-12-08T08:39:40.370162+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-08T08:39:46.112 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2023-12-08T08:39:46.112 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-12-08T08:39:46.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.370100+0000', 'last_change': '2023-12-08T08:39:40.370100+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.370100+0000', 'last_undegraded': '2023-12-08T08:39:40.370100+0000', 'last_fullsized': '2023-12-08T08:39:40.370100+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-08T08:39:46.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2023-12-08T08:39:46.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-12-08T08:39:46.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.373451+0000', 'last_change': '2023-12-08T08:39:40.373451+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.373451+0000', 'last_undegraded': '2023-12-08T08:39:40.373451+0000', 'last_fullsized': '2023-12-08T08:39:40.373451+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-08T08:39:46.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2023-12-08T08:39:46.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-12-08T08:39:46.113 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.369727+0000', 'last_change': '2023-12-08T08:39:40.369727+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.369727+0000', 'last_undegraded': '2023-12-08T08:39:40.369727+0000', 'last_fullsized': '2023-12-08T08:39:40.369727+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-08T08:39:46.113 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2023-12-08T08:39:46.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-12-08T08:39:46.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.372229+0000', 'last_change': '2023-12-08T08:39:40.372229+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.372229+0000', 'last_undegraded': '2023-12-08T08:39:40.372229+0000', 'last_fullsized': '2023-12-08T08:39:40.372229+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-08T08:39:46.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2023-12-08T08:39:46.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-12-08T08:39:46.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.369727+0000', 'last_change': '2023-12-08T08:39:40.369727+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.369727+0000', 'last_undegraded': '2023-12-08T08:39:40.369727+0000', 'last_fullsized': '2023-12-08T08:39:40.369727+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-08T08:39:46.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2023-12-08T08:39:46.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-12-08T08:39:46.114 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.372041+0000', 'last_change': '2023-12-08T08:39:40.372041+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.372041+0000', 'last_undegraded': '2023-12-08T08:39:40.372041+0000', 'last_fullsized': '2023-12-08T08:39:40.372041+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-08T08:39:46.114 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2023-12-08T08:39:46.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-12-08T08:39:46.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.372086+0000', 'last_change': '2023-12-08T08:39:40.372086+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.372086+0000', 'last_undegraded': '2023-12-08T08:39:40.372086+0000', 'last_fullsized': '2023-12-08T08:39:40.372086+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-08T08:39:46.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2023-12-08T08:39:46.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-12-08T08:39:46.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.371953+0000', 'last_change': '2023-12-08T08:39:40.371953+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.371953+0000', 'last_undegraded': '2023-12-08T08:39:40.371953+0000', 'last_fullsized': '2023-12-08T08:39:40.371953+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-08T08:39:46.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-12-08T08:39:46.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-12-08T08:39:46.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.369539+0000', 'last_change': '2023-12-08T08:39:40.369539+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.369539+0000', 'last_undegraded': '2023-12-08T08:39:40.369539+0000', 'last_fullsized': '2023-12-08T08:39:40.369539+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-08T08:39:46.115 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2023-12-08T08:39:46.115 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-12-08T08:39:46.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.371795+0000', 'last_change': '2023-12-08T08:39:40.371795+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.371795+0000', 'last_undegraded': '2023-12-08T08:39:40.371795+0000', 'last_fullsized': '2023-12-08T08:39:40.371795+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-08T08:39:46.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2023-12-08T08:39:46.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-08T08:39:41.366630+0000', 'last_change': '2023-12-08T08:39:41.366630+0000', 'last_active': '2023-12-08T08:39:41.366630+0000', 'last_peered': '2023-12-08T08:39:41.366630+0000', 'last_clean': '2023-12-08T08:39:41.366630+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:41.366630+0000', 'last_undegraded': '2023-12-08T08:39:41.366630+0000', 'last_fullsized': '2023-12-08T08:39:41.366630+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:41.366630+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-08T08:39:46.116 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-12-08T08:39:46.116 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2023-12-08T08:39:40.373495+0000', 'last_change': '2023-12-08T08:39:40.373495+0000', 'last_active': '2023-12-08T08:39:40.361115+0000', 'last_peered': '2023-12-08T08:39:40.361115+0000', 'last_clean': '2023-12-08T08:39:40.361115+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-08T08:39:40.373495+0000', 'last_undegraded': '2023-12-08T08:39:40.373495+0000', 'last_fullsized': '2023-12-08T08:39:40.373495+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'last_clean_scrub_stamp': '2023-12-08T08:39:40.361115+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-08T08:39:46.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-08T08:39:46.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.239+0000 7f554fbf5640 1 Processor -- start 2023-12-08T08:39:46.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.240+0000 7f554fbf5640 1 -- start start 2023-12-08T08:39:46.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.240+0000 7f554fbf5640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5548157750 0x7f554815e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:46.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.240+0000 7f554fbf5640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f55480b29f0 0x7f5548161530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:46.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.240+0000 7f554fbf5640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f554815e7d0 con 0x7f55480b2540 2023-12-08T08:39:46.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.240+0000 7f554fbf5640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55480b2e60 con 0x7f5548157750 2023-12-08T08:39:46.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.240+0000 7f554fbf5640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554806cd10 con 0x7f55480b29f0 2023-12-08T08:39:46.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.241+0000 7f554d991640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5548157750 0x7f554815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.241+0000 7f554d190640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f55480b2540 0x7f5548160e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51586/0 (socket says 172.21.15.16:51586) 2023-12-08T08:39:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.241+0000 7f554d190640 1 -- 172.21.15.16:0/4279611438 learned_addr learned my addr 172.21.15.16:0/4279611438 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.241+0000 7f554e192640 1 --2- 172.21.15.16:0/4279611438 >> v2:172.21.15.112:3300/0 conn(0x7f55480b29f0 0x7f5548161530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.241+0000 7f554d991640 1 -- 172.21.15.16:0/4279611438 >> v2:172.21.15.112:3300/0 conn(0x7f55480b29f0 msgr2=0x7f5548161530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.241+0000 7f554d991640 1 --2- 172.21.15.16:0/4279611438 >> v2:172.21.15.112:3300/0 conn(0x7f55480b29f0 0x7f5548161530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.241+0000 7f554d991640 1 -- 172.21.15.16:0/4279611438 >> v1:172.21.15.16:6789/0 conn(0x7f55480b2540 legacy=0x7f5548160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.241+0000 7f554d991640 1 -- 172.21.15.16:0/4279611438 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554806cff0 con 0x7f5548157750 2023-12-08T08:39:46.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.241+0000 7f554c98f640 1 -- 172.21.15.16:0/4279611438 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725372861 0 0) 0x7f554815e7d0 con 0x7f55480b2540 2023-12-08T08:39:46.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.242+0000 7f554d991640 1 --2- 172.21.15.16:0/4279611438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5548157750 0x7f554815e290 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f55480b22a0 tx=0x7f554402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d5734937b5a89 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.242+0000 7f554c98f640 1 -- 172.21.15.16:0/4279611438 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544031830 con 0x7f5548157750 2023-12-08T08:39:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.242+0000 7f554c98f640 1 -- 172.21.15.16:0/4279611438 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5544031990 con 0x7f5548157750 2023-12-08T08:39:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.242+0000 7f554c98f640 1 -- 172.21.15.16:0/4279611438 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544031d20 con 0x7f5548157750 2023-12-08T08:39:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.242+0000 7f554fbf5640 1 -- 172.21.15.16:0/4279611438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5548157750 msgr2=0x7f554815e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.242+0000 7f554fbf5640 1 --2- 172.21.15.16:0/4279611438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5548157750 0x7f554815e290 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f55480b22a0 tx=0x7f554402b8d0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.242+0000 7f554fbf5640 1 -- 172.21.15.16:0/4279611438 shutdown_connections 2023-12-08T08:39:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.242+0000 7f554fbf5640 1 --2- 172.21.15.16:0/4279611438 >> v2:172.21.15.112:3300/0 conn(0x7f55480b29f0 0x7f5548161530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.242+0000 7f554fbf5640 1 --2- 172.21.15.16:0/4279611438 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5548157750 0x7f554815e290 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.242+0000 7f554fbf5640 1 -- 172.21.15.16:0/4279611438 >> 172.21.15.16:0/4279611438 conn(0x7f55480cf490 msgr2=0x7f55480cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.243+0000 7f554fbf5640 1 -- 172.21.15.16:0/4279611438 shutdown_connections 2023-12-08T08:39:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.243+0000 7f554fbf5640 1 -- 172.21.15.16:0/4279611438 wait complete. 2023-12-08T08:39:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.244+0000 7f554fbf5640 1 Processor -- start 2023-12-08T08:39:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.244+0000 7f554fbf5640 1 -- start start 2023-12-08T08:39:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.244+0000 7f554fbf5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55480b2540 0x7f5548155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.244+0000 7f554fbf5640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55480b29f0 0x7f5548153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.244+0000 7f554fbf5640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5548157750 0x7f5548154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.244+0000 7f554fbf5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55481f5bb0 con 0x7f55480b2540 2023-12-08T08:39:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.244+0000 7f554fbf5640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55481546c0 con 0x7f55480b29f0 2023-12-08T08:39:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.244+0000 7f554fbf5640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55481f4f30 con 0x7f5548157750 2023-12-08T08:39:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.244+0000 7f554d991640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55480b2540 0x7f5548155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.245+0000 7f554d991640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55480b2540 0x7f5548155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34790/0 (socket says 172.21.15.16:34790) 2023-12-08T08:39:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.245+0000 7f554d991640 1 -- 172.21.15.16:0/3032838762 learned_addr learned my addr 172.21.15.16:0/3032838762 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.245+0000 7f554d991640 1 -- 172.21.15.16:0/3032838762 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5548157750 msgr2=0x7f5548154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.245+0000 7f554d991640 1 --2- 172.21.15.16:0/3032838762 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5548157750 0x7f5548154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.245+0000 7f554d991640 1 -- 172.21.15.16:0/3032838762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55480b29f0 msgr2=0x7f5548153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:46.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.245+0000 7f554d991640 1 --2- 172.21.15.16:0/3032838762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55480b29f0 0x7f5548153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.245+0000 7f554d991640 1 -- 172.21.15.16:0/3032838762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5544009400 con 0x7f55480b2540 2023-12-08T08:39:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.245+0000 7f554d991640 1 --2- 172.21.15.16:0/3032838762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55480b2540 0x7f5548155b00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5544000c00 tx=0x7f55440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.245+0000 7f553e7fc640 1 -- 172.21.15.16:0/3032838762 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544004230 con 0x7f55480b2540 2023-12-08T08:39:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.245+0000 7f553e7fc640 1 -- 172.21.15.16:0/3032838762 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5544004390 con 0x7f55480b2540 2023-12-08T08:39:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.245+0000 7f554fbf5640 1 -- 172.21.15.16:0/3032838762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55481f5210 con 0x7f55480b2540 2023-12-08T08:39:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.245+0000 7f553e7fc640 1 -- 172.21.15.16:0/3032838762 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554403a870 con 0x7f55480b2540 2023-12-08T08:39:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.245+0000 7f554fbf5640 1 -- 172.21.15.16:0/3032838762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55481f57d0 con 0x7f55480b2540 2023-12-08T08:39:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.246+0000 7f554fbf5640 1 -- 172.21.15.16:0/3032838762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55481f5cf0 con 0x7f55480b2540 2023-12-08T08:39:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.247+0000 7f553e7fc640 1 -- 172.21.15.16:0/3032838762 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f554403a9d0 con 0x7f55480b2540 2023-12-08T08:39:46.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.247+0000 7f553e7fc640 1 --2- 172.21.15.16:0/3032838762 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f552c06ae40 0x7f552c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:46.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.247+0000 7f553e7fc640 1 -- 172.21.15.16:0/3032838762 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f55440ab900 con 0x7f55480b2540 2023-12-08T08:39:46.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.248+0000 7f554d190640 1 --2- 172.21.15.16:0/3032838762 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f552c06ae40 0x7f552c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:46.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.248+0000 7f554d190640 1 --2- 172.21.15.16:0/3032838762 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f552c06ae40 0x7f552c06d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5538007fd0 tx=0x7f5538007960 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:46.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.252+0000 7f553e7fc640 1 -- 172.21.15.16:0/3032838762 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5544076900 con 0x7f55480b2540 2023-12-08T08:39:46.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.496+0000 7f554fbf5640 1 -- 172.21.15.16:0/3032838762 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f554806c940 con 0x7f55480b2540 2023-12-08T08:39:46.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.496+0000 7f553e7fc640 1 -- 172.21.15.16:0/3032838762 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1108 (secure 0 0 0) 0x7f5544035470 con 0x7f55480b2540 2023-12-08T08:39:46.498 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:46.499 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":111,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1702024685,"num_in_osds":8,"osd_in_since":1702024677,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":238608384,"bytes_avail":772855504896,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"d","status":"up:active","gid":5170}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-12-08T08:39:24.991647+0000","services":{}},"progress_events":{}} 2023-12-08T08:39:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.498+0000 7f554fbf5640 1 -- 172.21.15.16:0/3032838762 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f552c06ae40 msgr2=0x7f552c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.498+0000 7f554fbf5640 1 --2- 172.21.15.16:0/3032838762 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f552c06ae40 0x7f552c06d300 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5538007fd0 tx=0x7f5538007960 comp rx=0 tx=0).stop 2023-12-08T08:39:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.498+0000 7f554fbf5640 1 -- 172.21.15.16:0/3032838762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55480b2540 msgr2=0x7f5548155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.498+0000 7f554fbf5640 1 --2- 172.21.15.16:0/3032838762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55480b2540 0x7f5548155b00 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5544000c00 tx=0x7f55440046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.498+0000 7f554fbf5640 1 -- 172.21.15.16:0/3032838762 shutdown_connections 2023-12-08T08:39:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.498+0000 7f554fbf5640 1 --2- 172.21.15.16:0/3032838762 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f552c06ae40 0x7f552c06d300 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.498+0000 7f554fbf5640 1 --2- 172.21.15.16:0/3032838762 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5548157750 0x7f5548154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.498+0000 7f554fbf5640 1 --2- 172.21.15.16:0/3032838762 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f55480b29f0 0x7f5548153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.498+0000 7f554fbf5640 1 --2- 172.21.15.16:0/3032838762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55480b2540 0x7f5548155b00 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.498+0000 7f554fbf5640 1 -- 172.21.15.16:0/3032838762 >> 172.21.15.16:0/3032838762 conn(0x7f55480cf490 msgr2=0x7f55480b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.499+0000 7f554fbf5640 1 -- 172.21.15.16:0/3032838762 shutdown_connections 2023-12-08T08:39:46.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.499+0000 7f554fbf5640 1 -- 172.21.15.16:0/3032838762 wait complete. 2023-12-08T08:39:46.511 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-08T08:39:46.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-08T08:39:46.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.616+0000 7f6ef3190640 1 Processor -- start 2023-12-08T08:39:46.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.616+0000 7f6ef3190640 1 -- start start 2023-12-08T08:39:46.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6ef3190640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f6eec15d580 0x7f6eec0d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:46.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6ef3190640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6eec0d3ad0 0x7f6eec15f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:46.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6ef3190640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6eec15f6c0 0x7f6eec15fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6ef3190640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eec15ee90 con 0x7f6eec15f6c0 2023-12-08T08:39:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6ef3190640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eec0d0fe0 con 0x7f6eec15d580 2023-12-08T08:39:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6ef3190640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0d1280 con 0x7f6eec0d3ad0 2023-12-08T08:39:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6ef172d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6eec15f6c0 0x7f6eec15fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6ef172d640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6eec15f6c0 0x7f6eec15fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41520/0 (socket says 172.21.15.16:41520) 2023-12-08T08:39:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6ef172d640 1 -- 172.21.15.16:0/2113088027 learned_addr learned my addr 172.21.15.16:0/2113088027 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6eebfff640 1 --2- 172.21.15.16:0/2113088027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6eec0d3ad0 0x7f6eec15f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6ef0f2c640 1 --2- 172.21.15.16:0/2113088027 >> v2:172.21.15.112:3300/0 conn(0x7f6eec15d580 0x7f6eec0d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6ef172d640 1 -- 172.21.15.16:0/2113088027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6eec0d3ad0 msgr2=0x7f6eec15f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6ef172d640 1 --2- 172.21.15.16:0/2113088027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6eec0d3ad0 0x7f6eec15f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6ef172d640 1 -- 172.21.15.16:0/2113088027 >> v2:172.21.15.112:3300/0 conn(0x7f6eec15d580 msgr2=0x7f6eec0d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.617+0000 7f6ef172d640 1 --2- 172.21.15.16:0/2113088027 >> v2:172.21.15.112:3300/0 conn(0x7f6eec15d580 0x7f6eec0d1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.618+0000 7f6ef172d640 1 -- 172.21.15.16:0/2113088027 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec0d1560 con 0x7f6eec15f6c0 2023-12-08T08:39:46.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.618+0000 7f6ef172d640 1 --2- 172.21.15.16:0/2113088027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eec15f6c0 0x7f6eec15fb50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0b5ab0 tx=0x7f6ee402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d1c9f370feb645 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:46.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.618+0000 7f6eeb7fe640 1 -- 172.21.15.16:0/2113088027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee402c9e0 con 0x7f6eec15f6c0 2023-12-08T08:39:46.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.618+0000 7f6eeb7fe640 1 -- 172.21.15.16:0/2113088027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee402cb40 con 0x7f6eec15f6c0 2023-12-08T08:39:46.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.618+0000 7f6eeb7fe640 1 -- 172.21.15.16:0/2113088027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee40343f0 con 0x7f6eec15f6c0 2023-12-08T08:39:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.619+0000 7f6ef3190640 1 -- 172.21.15.16:0/2113088027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eec15f6c0 msgr2=0x7f6eec15fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.619+0000 7f6ef3190640 1 --2- 172.21.15.16:0/2113088027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eec15f6c0 0x7f6eec15fb50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0b5ab0 tx=0x7f6ee402be40 comp rx=0 tx=0).stop 2023-12-08T08:39:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.619+0000 7f6ef3190640 1 -- 172.21.15.16:0/2113088027 shutdown_connections 2023-12-08T08:39:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.619+0000 7f6ef3190640 1 --2- 172.21.15.16:0/2113088027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eec15f6c0 0x7f6eec15fb50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.619+0000 7f6ef3190640 1 --2- 172.21.15.16:0/2113088027 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6eec0d3ad0 0x7f6eec15f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.619+0000 7f6ef3190640 1 --2- 172.21.15.16:0/2113088027 >> v2:172.21.15.112:3300/0 conn(0x7f6eec15d580 0x7f6eec0d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.619+0000 7f6ef3190640 1 -- 172.21.15.16:0/2113088027 >> 172.21.15.16:0/2113088027 conn(0x7f6eec0d2fb0 msgr2=0x7f6eec0d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.619+0000 7f6ef3190640 1 -- 172.21.15.16:0/2113088027 shutdown_connections 2023-12-08T08:39:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.619+0000 7f6ef3190640 1 -- 172.21.15.16:0/2113088027 wait complete. 2023-12-08T08:39:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.620+0000 7f6ef3190640 1 Processor -- start 2023-12-08T08:39:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.620+0000 7f6ef3190640 1 -- start start 2023-12-08T08:39:46.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.621+0000 7f6ef3190640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6eec0d3ad0 0x7f6eec0b8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.621+0000 7f6ef3190640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6eec15d580 0x7f6eec1525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.621+0000 7f6ef3190640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eec15f6c0 0x7f6eec0ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.621+0000 7f6ef3190640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0b6b20 con 0x7f6eec15f6c0 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.621+0000 7f6ef3190640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0b7dd0 con 0x7f6eec15d580 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.621+0000 7f6ef3190640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0b6d30 con 0x7f6eec0d3ad0 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.621+0000 7f6ef172d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eec15f6c0 0x7f6eec0ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.621+0000 7f6ef172d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eec15f6c0 0x7f6eec0ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41524/0 (socket says 172.21.15.16:41524) 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.621+0000 7f6ef172d640 1 -- 172.21.15.16:0/3917333555 learned_addr learned my addr 172.21.15.16:0/3917333555 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.621+0000 7f6ef0f2c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6eec0d3ad0 0x7f6eec0b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.621+0000 7f6eebfff640 1 --2- 172.21.15.16:0/3917333555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6eec15d580 0x7f6eec1525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.622+0000 7f6ef172d640 1 -- 172.21.15.16:0/3917333555 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6eec0d3ad0 msgr2=0x7f6eec0b8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.622+0000 7f6ef172d640 1 --2- 172.21.15.16:0/3917333555 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6eec0d3ad0 0x7f6eec0b8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.622+0000 7f6ef172d640 1 -- 172.21.15.16:0/3917333555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6eec15d580 msgr2=0x7f6eec1525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.622+0000 7f6ef172d640 1 --2- 172.21.15.16:0/3917333555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6eec15d580 0x7f6eec1525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.622+0000 7f6ef172d640 1 -- 172.21.15.16:0/3917333555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee4009b30 con 0x7f6eec15f6c0 2023-12-08T08:39:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.622+0000 7f6ef172d640 1 --2- 172.21.15.16:0/3917333555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eec15f6c0 0x7f6eec0ca080 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009f40 tx=0x7f6ee4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:46.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.623+0000 7f6ee97fa640 1 -- 172.21.15.16:0/3917333555 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4009510 con 0x7f6eec15f6c0 2023-12-08T08:39:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.623+0000 7f6ee97fa640 1 -- 172.21.15.16:0/3917333555 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee403e450 con 0x7f6eec15f6c0 2023-12-08T08:39:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.623+0000 7f6ee97fa640 1 -- 172.21.15.16:0/3917333555 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee403e7e0 con 0x7f6eec15f6c0 2023-12-08T08:39:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.623+0000 7f6ef3190640 1 -- 172.21.15.16:0/3917333555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eec0b7010 con 0x7f6eec15f6c0 2023-12-08T08:39:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.623+0000 7f6ef3190640 1 -- 172.21.15.16:0/3917333555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eec0b75d0 con 0x7f6eec15f6c0 2023-12-08T08:39:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.624+0000 7f6ef3190640 1 -- 172.21.15.16:0/3917333555 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eec0d09d0 con 0x7f6eec15f6c0 2023-12-08T08:39:46.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.625+0000 7f6ee97fa640 1 -- 172.21.15.16:0/3917333555 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ee403e940 con 0x7f6eec15f6c0 2023-12-08T08:39:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.625+0000 7f6ee97fa640 1 --2- 172.21.15.16:0/3917333555 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6ed806ae70 0x7f6ed806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.625+0000 7f6ee97fa640 1 -- 172.21.15.16:0/3917333555 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f6ee40ac030 con 0x7f6eec15f6c0 2023-12-08T08:39:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.625+0000 7f6ef0f2c640 1 --2- 172.21.15.16:0/3917333555 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6ed806ae70 0x7f6ed806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:46.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.626+0000 7f6ef0f2c640 1 --2- 172.21.15.16:0/3917333555 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6ed806ae70 0x7f6ed806d330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6edc006fd0 tx=0x7f6edc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:46.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.630+0000 7f6ee97fa640 1 -- 172.21.15.16:0/3917333555 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ee40770e0 con 0x7f6eec15f6c0 2023-12-08T08:39:46.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.805+0000 7f6ef3190640 1 -- 172.21.15.16:0/3917333555 --> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6eec067430 con 0x7f6ed806ae70 2023-12-08T08:39:46.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.812+0000 7f6ee97fa640 1 -- 172.21.15.16:0/3917333555 <== mgr.4111 v2:172.21.15.112:6832/804067392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277533 (secure 0 0 0) 0x7f6eec067430 con 0x7f6ed806ae70 2023-12-08T08:39:46.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:46.818 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-12-08T08:39:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.815+0000 7f6ef3190640 1 -- 172.21.15.16:0/3917333555 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6ed806ae70 msgr2=0x7f6ed806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.815+0000 7f6ef3190640 1 --2- 172.21.15.16:0/3917333555 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6ed806ae70 0x7f6ed806d330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6edc006fd0 tx=0x7f6edc008040 comp rx=0 tx=0).stop 2023-12-08T08:39:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.815+0000 7f6ef3190640 1 -- 172.21.15.16:0/3917333555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eec15f6c0 msgr2=0x7f6eec0ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.815+0000 7f6ef3190640 1 --2- 172.21.15.16:0/3917333555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eec15f6c0 0x7f6eec0ca080 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009f40 tx=0x7f6ee4004a40 comp rx=0 tx=0).stop 2023-12-08T08:39:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.815+0000 7f6ef3190640 1 -- 172.21.15.16:0/3917333555 shutdown_connections 2023-12-08T08:39:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.815+0000 7f6ef3190640 1 --2- 172.21.15.16:0/3917333555 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f6ed806ae70 0x7f6ed806d330 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.815+0000 7f6ef3190640 1 --2- 172.21.15.16:0/3917333555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6eec15f6c0 0x7f6eec0ca080 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.815+0000 7f6ef3190640 1 --2- 172.21.15.16:0/3917333555 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f6eec15d580 0x7f6eec1525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.815+0000 7f6ef3190640 1 --2- 172.21.15.16:0/3917333555 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f6eec0d3ad0 0x7f6eec0b8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.815+0000 7f6ef3190640 1 -- 172.21.15.16:0/3917333555 >> 172.21.15.16:0/3917333555 conn(0x7f6eec0d2fb0 msgr2=0x7f6eec0b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.815+0000 7f6ef3190640 1 -- 172.21.15.16:0/3917333555 shutdown_connections 2023-12-08T08:39:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:46.815+0000 7f6ef3190640 1 -- 172.21.15.16:0/3917333555 wait complete. 2023-12-08T08:39:46.828 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2023-12-08T08:39:44.996436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233016,"kb_used_data":4356,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741704,"statfs":{"total":773094113280,"available":772855504896,"internally_reserved":0,"allocated":4460544,"data_stored":2554093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.629075"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372184+0000","last_change":"2023-12-08T08:39:40.372184+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372184+0000","last_undegraded":"2023-12-08T08:39:40.372184+0000","last_fullsized":"2023-12-08T08:39:40.372184+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372016+0000","last_change":"2023-12-08T08:39:40.372016+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372016+0000","last_undegraded":"2023-12-08T08:39:40.372016+0000","last_fullsized":"2023-12-08T08:39:40.372016+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375234+0000","last_change":"2023-12-08T08:39:40.375234+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375234+0000","last_undegraded":"2023-12-08T08:39:40.375234+0000","last_fullsized":"2023-12-08T08:39:40.375234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369509+0000","last_change":"2023-12-08T08:39:40.369509+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369509+0000","last_undegraded":"2023-12-08T08:39:40.369509+0000","last_fullsized":"2023-12-08T08:39:40.369509+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371677+0000","last_change":"2023-12-08T08:39:40.371677+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371677+0000","last_undegraded":"2023-12-08T08:39:40.371677+0000","last_fullsized":"2023-12-08T08:39:40.371677+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373183+0000","last_change":"2023-12-08T08:39:40.373183+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373183+0000","last_undegraded":"2023-12-08T08:39:40.373183+0000","last_fullsized":"2023-12-08T08:39:40.373183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375447+0000","last_change":"2023-12-08T08:39:40.375447+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375447+0000","last_undegraded":"2023-12-08T08:39:40.375447+0000","last_fullsized":"2023-12-08T08:39:40.375447+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369561+0000","last_change":"2023-12-08T08:39:40.369561+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369561+0000","last_undegraded":"2023-12-08T08:39:40.369561+0000","last_fullsized":"2023-12-08T08:39:40.369561+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372358+0000","last_change":"2023-12-08T08:39:40.372358+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372358+0000","last_undegraded":"2023-12-08T08:39:40.372358+0000","last_fullsized":"2023-12-08T08:39:40.372358+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371732+0000","last_change":"2023-12-08T08:39:40.371732+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371732+0000","last_undegraded":"2023-12-08T08:39:40.371732+0000","last_fullsized":"2023-12-08T08:39:40.371732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371718+0000","last_change":"2023-12-08T08:39:40.371718+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371718+0000","last_undegraded":"2023-12-08T08:39:40.371718+0000","last_fullsized":"2023-12-08T08:39:40.371718+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371956+0000","last_change":"2023-12-08T08:39:40.371956+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371956+0000","last_undegraded":"2023-12-08T08:39:40.371956+0000","last_fullsized":"2023-12-08T08:39:40.371956+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372089+0000","last_change":"2023-12-08T08:39:40.372089+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372089+0000","last_undegraded":"2023-12-08T08:39:40.372089+0000","last_fullsized":"2023-12-08T08:39:40.372089+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375171+0000","last_change":"2023-12-08T08:39:40.375171+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375171+0000","last_undegraded":"2023-12-08T08:39:40.375171+0000","last_fullsized":"2023-12-08T08:39:40.375171+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371915+0000","last_change":"2023-12-08T08:39:40.371915+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371915+0000","last_undegraded":"2023-12-08T08:39:40.371915+0000","last_fullsized":"2023-12-08T08:39:40.371915+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373260+0000","last_change":"2023-12-08T08:39:40.373260+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373260+0000","last_undegraded":"2023-12-08T08:39:40.373260+0000","last_fullsized":"2023-12-08T08:39:40.373260+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372660+0000","last_change":"2023-12-08T08:39:40.372660+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372660+0000","last_undegraded":"2023-12-08T08:39:40.372660+0000","last_fullsized":"2023-12-08T08:39:40.372660+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369563+0000","last_change":"2023-12-08T08:39:40.369563+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369563+0000","last_undegraded":"2023-12-08T08:39:40.369563+0000","last_fullsized":"2023-12-08T08:39:40.369563+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.370714+0000","last_change":"2023-12-08T08:39:40.370714+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.370714+0000","last_undegraded":"2023-12-08T08:39:40.370714+0000","last_fullsized":"2023-12-08T08:39:40.370714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.375212+0000","last_change":"2023-12-08T08:39:40.375212+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.375212+0000","last_undegraded":"2023-12-08T08:39:40.375212+0000","last_fullsized":"2023-12-08T08:39:40.375212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.370162+0000","last_change":"2023-12-08T08:39:40.370162+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.370162+0000","last_undegraded":"2023-12-08T08:39:40.370162+0000","last_fullsized":"2023-12-08T08:39:40.370162+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.370100+0000","last_change":"2023-12-08T08:39:40.370100+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.370100+0000","last_undegraded":"2023-12-08T08:39:40.370100+0000","last_fullsized":"2023-12-08T08:39:40.370100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373451+0000","last_change":"2023-12-08T08:39:40.373451+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373451+0000","last_undegraded":"2023-12-08T08:39:40.373451+0000","last_fullsized":"2023-12-08T08:39:40.373451+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369727+0000","last_change":"2023-12-08T08:39:40.369727+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369727+0000","last_undegraded":"2023-12-08T08:39:40.369727+0000","last_fullsized":"2023-12-08T08:39:40.369727+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372229+0000","last_change":"2023-12-08T08:39:40.372229+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372229+0000","last_undegraded":"2023-12-08T08:39:40.372229+0000","last_fullsized":"2023-12-08T08:39:40.372229+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369727+0000","last_change":"2023-12-08T08:39:40.369727+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369727+0000","last_undegraded":"2023-12-08T08:39:40.369727+0000","last_fullsized":"2023-12-08T08:39:40.369727+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372041+0000","last_change":"2023-12-08T08:39:40.372041+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372041+0000","last_undegraded":"2023-12-08T08:39:40.372041+0000","last_fullsized":"2023-12-08T08:39:40.372041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.372086+0000","last_change":"2023-12-08T08:39:40.372086+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.372086+0000","last_undegraded":"2023-12-08T08:39:40.372086+0000","last_fullsized":"2023-12-08T08:39:40.372086+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371953+0000","last_change":"2023-12-08T08:39:40.371953+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371953+0000","last_undegraded":"2023-12-08T08:39:40.371953+0000","last_fullsized":"2023-12-08T08:39:40.371953+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.369539+0000","last_change":"2023-12-08T08:39:40.369539+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.369539+0000","last_undegraded":"2023-12-08T08:39:40.369539+0000","last_fullsized":"2023-12-08T08:39:40.369539+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.371795+0000","last_change":"2023-12-08T08:39:40.371795+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.371795+0000","last_undegraded":"2023-12-08T08:39:40.371795+0000","last_fullsized":"2023-12-08T08:39:40.371795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-08T08:39:41.366630+0000","last_change":"2023-12-08T08:39:41.366630+0000","last_active":"2023-12-08T08:39:41.366630+0000","last_peered":"2023-12-08T08:39:41.366630+0000","last_clean":"2023-12-08T08:39:41.366630+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:41.366630+0000","last_undegraded":"2023-12-08T08:39:41.366630+0000","last_fullsized":"2023-12-08T08:39:41.366630+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2023-12-08T08:39:40.373495+0000","last_change":"2023-12-08T08:39:40.373495+0000","last_active":"2023-12-08T08:39:40.361115+0000","last_peered":"2023-12-08T08:39:40.361115+0000","last_clean":"2023-12-08T08:39:40.361115+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-08T08:39:40.373495+0000","last_undegraded":"2023-12-08T08:39:40.373495+0000","last_fullsized":"2023-12-08T08:39:40.373495+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-08T08:39:35.915241+0000","last_change":"2023-12-08T08:38:08.276815+0000","last_active":"2023-12-08T08:39:35.915241+0000","last_peered":"2023-12-08T08:39:35.915241+0000","last_clean":"2023-12-08T08:39:35.915241+0000","last_became_active":"2023-12-08T08:38:08.275828+0000","last_became_peered":"2023-12-08T08:38:08.275828+0000","last_unstale":"2023-12-08T08:39:35.915241+0000","last_undegraded":"2023-12-08T08:39:35.915241+0000","last_fullsized":"2023-12-08T08:39:35.915241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_clean_scrub_stamp":"2023-12-08T08:38:07.104197+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:35:34.851149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":764339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.629}]},{"osd":5,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.39500000000000002}]},{"osd":6,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.438},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":167834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.59599999999999997}]},{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55000000000000004}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.69099999999999995}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.61699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":173971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64000000000000001}]},{"osd":1,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.5},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.443},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.624},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.58299999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":167834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.498},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.46400000000000002}]},{"osd":1,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.60299999999999998}]},{"osd":3,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.42499999999999999}]},{"osd":5,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.441},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.39200000000000002}]},{"osd":6,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55900000000000005}]},{"osd":7,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":401408,"data_stored":167834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.30199999999999999}]},{"osd":1,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.24299999999999999}]},{"osd":2,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.317},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.32500000000000001}]},{"osd":4,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":0.60999999999999999}]},{"osd":5,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.628}]},{"osd":6,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.621},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.70599999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":173971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.34799999999999998}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.377},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.36399999999999999}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.56799999999999995}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.63},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.55500000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":173971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.313},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.373}]},{"osd":2,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.28000000000000003}]},{"osd":4,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.503},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.49199999999999999}]},{"osd":7,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54600000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1007616,"data_stored":764339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.33300000000000002}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.316},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.44500000000000001}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.46400000000000002}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.436}]},{"osd":6,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.38900000000000001}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.372}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-08T08:39:49.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-08T08:39:49.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.950+0000 7f5edbf53640 1 Processor -- start 2023-12-08T08:39:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.951+0000 7f5edbf53640 1 -- start start 2023-12-08T08:39:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.951+0000 7f5edbf53640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5ed40b0cf0 0x7f5ed40d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.951+0000 7f5edbf53640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed40d20b0 con 0x7f5ed40b1d70 2023-12-08T08:39:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.951+0000 7f5edbf53640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed40d1b80 con 0x7f5ed41608a0 2023-12-08T08:39:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.951+0000 7f5edbf53640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed40d0620 con 0x7f5ed40b0cf0 2023-12-08T08:39:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.952+0000 7f5ed94ee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5ed40b0cf0 0x7f5ed40d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.952+0000 7f5ed94ee640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5ed40b0cf0 0x7f5ed40d1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41534/0 (socket says 172.21.15.16:41534) 2023-12-08T08:39:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.952+0000 7f5ed94ee640 1 -- 172.21.15.16:0/3201862226 learned_addr learned my addr 172.21.15.16:0/3201862226 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.952+0000 7f5eda4f0640 1 --1- 172.21.15.16:0/3201862226 >> v1:172.21.15.16:6789/0 conn(0x7f5ed40b1d70 0x7f5ed40cdc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35400/0 (socket says 172.21.15.16:35400) 2023-12-08T08:39:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.952+0000 7f5ed94ee640 1 -- 172.21.15.16:0/3201862226 >> v1:172.21.15.112:6789/0 conn(0x7f5ed41608a0 legacy=0x7f5ed40d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.952+0000 7f5ed94ee640 1 -- 172.21.15.16:0/3201862226 >> v1:172.21.15.16:6789/0 conn(0x7f5ed40b1d70 legacy=0x7f5ed40cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.952+0000 7f5ed94ee640 1 -- 172.21.15.16:0/3201862226 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed40d1dd0 con 0x7f5ed40b0cf0 2023-12-08T08:39:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.953+0000 7f5ed94ee640 1 --2- 172.21.15.16:0/3201862226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40b0cf0 0x7f5ed40d1640 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009870 tx=0x7f5ec402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1955b5d43146f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.953+0000 7f5ed8ced640 1 -- 172.21.15.16:0/3201862226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4030850 con 0x7f5ed40b0cf0 2023-12-08T08:39:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.953+0000 7f5ed8ced640 1 -- 172.21.15.16:0/3201862226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec40309b0 con 0x7f5ed40b0cf0 2023-12-08T08:39:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.953+0000 7f5ed8ced640 1 -- 172.21.15.16:0/3201862226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4030d40 con 0x7f5ed40b0cf0 2023-12-08T08:39:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.953+0000 7f5edbf53640 1 -- 172.21.15.16:0/3201862226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40b0cf0 msgr2=0x7f5ed40d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.953+0000 7f5edbf53640 1 --2- 172.21.15.16:0/3201862226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40b0cf0 0x7f5ed40d1640 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009870 tx=0x7f5ec402b990 comp rx=0 tx=0).stop 2023-12-08T08:39:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.953+0000 7f5edbf53640 1 -- 172.21.15.16:0/3201862226 shutdown_connections 2023-12-08T08:39:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.953+0000 7f5edbf53640 1 --2- 172.21.15.16:0/3201862226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed40b0cf0 0x7f5ed40d1640 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.953+0000 7f5edbf53640 1 -- 172.21.15.16:0/3201862226 >> 172.21.15.16:0/3201862226 conn(0x7f5ed40d3030 msgr2=0x7f5ed40d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.953+0000 7f5edbf53640 1 -- 172.21.15.16:0/3201862226 shutdown_connections 2023-12-08T08:39:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.954+0000 7f5edbf53640 1 -- 172.21.15.16:0/3201862226 wait complete. 2023-12-08T08:39:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.955+0000 7f5edbf53640 1 Processor -- start 2023-12-08T08:39:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.955+0000 7f5edbf53640 1 -- start start 2023-12-08T08:39:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.955+0000 7f5edbf53640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed40b0cf0 0x7f5ed40cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5edbf53640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ed40b1d70 0x7f5ed40c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5edbf53640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed41608a0 0x7f5ed41f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5edbf53640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed40ca100 con 0x7f5ed41608a0 2023-12-08T08:39:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5edbf53640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed41f6d60 con 0x7f5ed40b0cf0 2023-12-08T08:39:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5edbf53640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed41f7040 con 0x7f5ed40b1d70 2023-12-08T08:39:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5eda4f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed41608a0 0x7f5ed41f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5eda4f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed41608a0 0x7f5ed41f62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41544/0 (socket says 172.21.15.16:41544) 2023-12-08T08:39:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5eda4f0640 1 -- 172.21.15.16:0/3889417471 learned_addr learned my addr 172.21.15.16:0/3889417471 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5ed94ee640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ed40b1d70 0x7f5ed40c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5ed9cef640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed40b0cf0 0x7f5ed40cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5eda4f0640 1 -- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ed40b1d70 msgr2=0x7f5ed40c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5eda4f0640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ed40b1d70 0x7f5ed40c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5eda4f0640 1 -- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed40b0cf0 msgr2=0x7f5ed40cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5eda4f0640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed40b0cf0 0x7f5ed40cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5eda4f0640 1 -- 172.21.15.16:0/3889417471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec40094d0 con 0x7f5ed41608a0 2023-12-08T08:39:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5ed9cef640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed40b0cf0 0x7f5ed40cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5eda4f0640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed41608a0 0x7f5ed41f62e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00c780 tx=0x7f5ecc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5eca7fc640 1 -- 172.21.15.16:0/3889417471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc00ce90 con 0x7f5ed41608a0 2023-12-08T08:39:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5eca7fc640 1 -- 172.21.15.16:0/3889417471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ecc00a640 con 0x7f5ed41608a0 2023-12-08T08:39:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5edbf53640 1 -- 172.21.15.16:0/3889417471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed40ca350 con 0x7f5ed41608a0 2023-12-08T08:39:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5eca7fc640 1 -- 172.21.15.16:0/3889417471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc00a9f0 con 0x7f5ed41608a0 2023-12-08T08:39:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.956+0000 7f5edbf53640 1 -- 172.21.15.16:0/3889417471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed40b8770 con 0x7f5ed41608a0 2023-12-08T08:39:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.957+0000 7f5edbf53640 1 -- 172.21.15.16:0/3889417471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed4067430 con 0x7f5ed41608a0 2023-12-08T08:39:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.958+0000 7f5eca7fc640 1 -- 172.21.15.16:0/3889417471 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ecc00ab50 con 0x7f5ed41608a0 2023-12-08T08:39:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.958+0000 7f5eca7fc640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5eb806ae40 0x7f5eb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.959+0000 7f5eca7fc640 1 -- 172.21.15.16:0/3889417471 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5ecc088dd0 con 0x7f5ed41608a0 2023-12-08T08:39:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.959+0000 7f5ed9cef640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5eb806ae40 0x7f5eb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.959+0000 7f5ed9cef640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5eb806ae40 0x7f5eb806d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0007fd0 tx=0x7f5ed0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:49.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:49.963+0000 7f5eca7fc640 1 -- 172.21.15.16:0/3889417471 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ecc053e80 con 0x7f5ed41608a0 2023-12-08T08:39:50.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.139+0000 7f5edbf53640 1 -- 172.21.15.16:0/3889417471 --> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ed40cd3f0 con 0x7f5eb806ae40 2023-12-08T08:39:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.147+0000 7f5eca7fc640 1 -- 172.21.15.16:0/3889417471 <== mgr.4111 v2:172.21.15.112:6832/804067392 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289532 (secure 0 0 0) 0x7f5ed40cd3f0 con 0x7f5eb806ae40 2023-12-08T08:39:50.153 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:50.154 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-12-08T08:39:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.150+0000 7f5edbf53640 1 -- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5eb806ae40 msgr2=0x7f5eb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.150+0000 7f5edbf53640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5eb806ae40 0x7f5eb806d300 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0007fd0 tx=0x7f5ed0009040 comp rx=0 tx=0).stop 2023-12-08T08:39:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.150+0000 7f5edbf53640 1 -- 172.21.15.16:0/3889417471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed41608a0 msgr2=0x7f5ed41f62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.150+0000 7f5edbf53640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed41608a0 0x7f5ed41f62e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00c780 tx=0x7f5ecc00cc40 comp rx=0 tx=0).stop 2023-12-08T08:39:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.150+0000 7f5edbf53640 1 -- 172.21.15.16:0/3889417471 shutdown_connections 2023-12-08T08:39:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.150+0000 7f5edbf53640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f5eb806ae40 0x7f5eb806d300 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.150+0000 7f5edbf53640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ed41608a0 0x7f5ed41f62e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.150+0000 7f5edbf53640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f5ed40b1d70 0x7f5ed40c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.150+0000 7f5edbf53640 1 --2- 172.21.15.16:0/3889417471 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f5ed40b0cf0 0x7f5ed40cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.150+0000 7f5edbf53640 1 -- 172.21.15.16:0/3889417471 >> 172.21.15.16:0/3889417471 conn(0x7f5ed40d3030 msgr2=0x7f5ed40d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.150+0000 7f5edbf53640 1 -- 172.21.15.16:0/3889417471 shutdown_connections 2023-12-08T08:39:50.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.150+0000 7f5edbf53640 1 -- 172.21.15.16:0/3889417471 wait complete. 2023-12-08T08:39:50.164 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2023-12-08T08:39:48.997412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233408,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741312,"statfs":{"total":773094113280,"available":772855103488,"internally_reserved":0,"allocated":4857856,"data_stored":2739804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.625112"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.392628+0000","last_change":"2023-12-08T08:39:42.389296+0000","last_active":"2023-12-08T08:39:43.392628+0000","last_peered":"2023-12-08T08:39:43.392628+0000","last_clean":"2023-12-08T08:39:43.392628+0000","last_became_active":"2023-12-08T08:39:42.389035+0000","last_became_peered":"2023-12-08T08:39:42.389035+0000","last_unstale":"2023-12-08T08:39:43.392628+0000","last_undegraded":"2023-12-08T08:39:43.392628+0000","last_fullsized":"2023-12-08T08:39:43.392628+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:35:34.758731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.392188+0000","last_change":"2023-12-08T08:39:42.389917+0000","last_active":"2023-12-08T08:39:43.392188+0000","last_peered":"2023-12-08T08:39:43.392188+0000","last_clean":"2023-12-08T08:39:43.392188+0000","last_became_active":"2023-12-08T08:39:42.389663+0000","last_became_peered":"2023-12-08T08:39:42.389663+0000","last_unstale":"2023-12-08T08:39:43.392188+0000","last_undegraded":"2023-12-08T08:39:43.392188+0000","last_fullsized":"2023-12-08T08:39:43.392188+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:58:29.935974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.158841+0000","last_change":"2023-12-08T08:39:42.404217+0000","last_active":"2023-12-08T08:39:44.158841+0000","last_peered":"2023-12-08T08:39:44.158841+0000","last_clean":"2023-12-08T08:39:44.158841+0000","last_became_active":"2023-12-08T08:39:42.403925+0000","last_became_peered":"2023-12-08T08:39:42.403925+0000","last_unstale":"2023-12-08T08:39:44.158841+0000","last_undegraded":"2023-12-08T08:39:44.158841+0000","last_fullsized":"2023-12-08T08:39:44.158841+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T20:07:40.881713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.411431+0000","last_change":"2023-12-08T08:39:42.395814+0000","last_active":"2023-12-08T08:39:43.411431+0000","last_peered":"2023-12-08T08:39:43.411431+0000","last_clean":"2023-12-08T08:39:43.411431+0000","last_became_active":"2023-12-08T08:39:42.395533+0000","last_became_peered":"2023-12-08T08:39:42.395533+0000","last_unstale":"2023-12-08T08:39:43.411431+0000","last_undegraded":"2023-12-08T08:39:43.411431+0000","last_fullsized":"2023-12-08T08:39:43.411431+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T14:38:00.535181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.411393+0000","last_change":"2023-12-08T08:39:42.398588+0000","last_active":"2023-12-08T08:39:43.411393+0000","last_peered":"2023-12-08T08:39:43.411393+0000","last_clean":"2023-12-08T08:39:43.411393+0000","last_became_active":"2023-12-08T08:39:42.396796+0000","last_became_peered":"2023-12-08T08:39:42.396796+0000","last_unstale":"2023-12-08T08:39:43.411393+0000","last_undegraded":"2023-12-08T08:39:43.411393+0000","last_fullsized":"2023-12-08T08:39:43.411393+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:47:00.910523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.471971+0000","last_change":"2023-12-08T08:39:42.392439+0000","last_active":"2023-12-08T08:39:44.471971+0000","last_peered":"2023-12-08T08:39:44.471971+0000","last_clean":"2023-12-08T08:39:44.471971+0000","last_became_active":"2023-12-08T08:39:42.392160+0000","last_became_peered":"2023-12-08T08:39:42.392160+0000","last_unstale":"2023-12-08T08:39:44.471971+0000","last_undegraded":"2023-12-08T08:39:44.471971+0000","last_fullsized":"2023-12-08T08:39:44.471971+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:37:13.047884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472854+0000","last_change":"2023-12-08T08:39:42.394397+0000","last_active":"2023-12-08T08:39:44.472854+0000","last_peered":"2023-12-08T08:39:44.472854+0000","last_clean":"2023-12-08T08:39:44.472854+0000","last_became_active":"2023-12-08T08:39:42.394152+0000","last_became_peered":"2023-12-08T08:39:42.394152+0000","last_unstale":"2023-12-08T08:39:44.472854+0000","last_undegraded":"2023-12-08T08:39:44.472854+0000","last_fullsized":"2023-12-08T08:39:44.472854+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:58:29.935974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.392030+0000","last_change":"2023-12-08T08:39:42.401498+0000","last_active":"2023-12-08T08:39:43.392030+0000","last_peered":"2023-12-08T08:39:43.392030+0000","last_clean":"2023-12-08T08:39:43.392030+0000","last_became_active":"2023-12-08T08:39:42.401227+0000","last_became_peered":"2023-12-08T08:39:42.401227+0000","last_unstale":"2023-12-08T08:39:43.392030+0000","last_undegraded":"2023-12-08T08:39:43.392030+0000","last_fullsized":"2023-12-08T08:39:43.392030+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:30:45.444389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.158783+0000","last_change":"2023-12-08T08:39:42.402966+0000","last_active":"2023-12-08T08:39:44.158783+0000","last_peered":"2023-12-08T08:39:44.158783+0000","last_clean":"2023-12-08T08:39:44.158783+0000","last_became_active":"2023-12-08T08:39:42.402660+0000","last_became_peered":"2023-12-08T08:39:42.402660+0000","last_unstale":"2023-12-08T08:39:44.158783+0000","last_undegraded":"2023-12-08T08:39:44.158783+0000","last_fullsized":"2023-12-08T08:39:44.158783+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:11:31.054767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472999+0000","last_change":"2023-12-08T08:39:42.398875+0000","last_active":"2023-12-08T08:39:44.472999+0000","last_peered":"2023-12-08T08:39:44.472999+0000","last_clean":"2023-12-08T08:39:44.472999+0000","last_became_active":"2023-12-08T08:39:42.398632+0000","last_became_peered":"2023-12-08T08:39:42.398632+0000","last_unstale":"2023-12-08T08:39:44.472999+0000","last_undegraded":"2023-12-08T08:39:44.472999+0000","last_fullsized":"2023-12-08T08:39:44.472999+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:24:07.734572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472486+0000","last_change":"2023-12-08T08:39:42.397295+0000","last_active":"2023-12-08T08:39:44.472486+0000","last_peered":"2023-12-08T08:39:44.472486+0000","last_clean":"2023-12-08T08:39:44.472486+0000","last_became_active":"2023-12-08T08:39:42.397026+0000","last_became_peered":"2023-12-08T08:39:42.397026+0000","last_unstale":"2023-12-08T08:39:44.472486+0000","last_undegraded":"2023-12-08T08:39:44.472486+0000","last_fullsized":"2023-12-08T08:39:44.472486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:48:49.036284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.158630+0000","last_change":"2023-12-08T08:39:42.386063+0000","last_active":"2023-12-08T08:39:44.158630+0000","last_peered":"2023-12-08T08:39:44.158630+0000","last_clean":"2023-12-08T08:39:44.158630+0000","last_became_active":"2023-12-08T08:39:42.385592+0000","last_became_peered":"2023-12-08T08:39:42.385592+0000","last_unstale":"2023-12-08T08:39:44.158630+0000","last_undegraded":"2023-12-08T08:39:44.158630+0000","last_fullsized":"2023-12-08T08:39:44.158630+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:55:31.370134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472801+0000","last_change":"2023-12-08T08:39:42.398134+0000","last_active":"2023-12-08T08:39:44.472801+0000","last_peered":"2023-12-08T08:39:44.472801+0000","last_clean":"2023-12-08T08:39:44.472801+0000","last_became_active":"2023-12-08T08:39:42.397877+0000","last_became_peered":"2023-12-08T08:39:42.397877+0000","last_unstale":"2023-12-08T08:39:44.472801+0000","last_undegraded":"2023-12-08T08:39:44.472801+0000","last_fullsized":"2023-12-08T08:39:44.472801+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:10:31.013022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.397803+0000","last_change":"2023-12-08T08:39:42.402307+0000","last_active":"2023-12-08T08:39:43.397803+0000","last_peered":"2023-12-08T08:39:43.397803+0000","last_clean":"2023-12-08T08:39:43.397803+0000","last_became_active":"2023-12-08T08:39:42.399934+0000","last_became_peered":"2023-12-08T08:39:42.399934+0000","last_unstale":"2023-12-08T08:39:43.397803+0000","last_undegraded":"2023-12-08T08:39:43.397803+0000","last_fullsized":"2023-12-08T08:39:43.397803+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:37:13.047884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472971+0000","last_change":"2023-12-08T08:39:42.388408+0000","last_active":"2023-12-08T08:39:44.472971+0000","last_peered":"2023-12-08T08:39:44.472971+0000","last_clean":"2023-12-08T08:39:44.472971+0000","last_became_active":"2023-12-08T08:39:42.388040+0000","last_became_peered":"2023-12-08T08:39:42.388040+0000","last_unstale":"2023-12-08T08:39:44.472971+0000","last_undegraded":"2023-12-08T08:39:44.472971+0000","last_fullsized":"2023-12-08T08:39:44.472971+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:18:47.663728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.163107+0000","last_change":"2023-12-08T08:39:42.404779+0000","last_active":"2023-12-08T08:39:44.163107+0000","last_peered":"2023-12-08T08:39:44.163107+0000","last_clean":"2023-12-08T08:39:44.163107+0000","last_became_active":"2023-12-08T08:39:42.404417+0000","last_became_peered":"2023-12-08T08:39:42.404417+0000","last_unstale":"2023-12-08T08:39:44.163107+0000","last_undegraded":"2023-12-08T08:39:44.163107+0000","last_fullsized":"2023-12-08T08:39:44.163107+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:29:10.077076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.163619+0000","last_change":"2023-12-08T08:39:42.404331+0000","last_active":"2023-12-08T08:39:44.163619+0000","last_peered":"2023-12-08T08:39:44.163619+0000","last_clean":"2023-12-08T08:39:44.163619+0000","last_became_active":"2023-12-08T08:39:42.404070+0000","last_became_peered":"2023-12-08T08:39:42.404070+0000","last_unstale":"2023-12-08T08:39:44.163619+0000","last_undegraded":"2023-12-08T08:39:44.163619+0000","last_fullsized":"2023-12-08T08:39:44.163619+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:30:45.444389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472735+0000","last_change":"2023-12-08T08:39:42.400677+0000","last_active":"2023-12-08T08:39:44.472735+0000","last_peered":"2023-12-08T08:39:44.472735+0000","last_clean":"2023-12-08T08:39:44.472735+0000","last_became_active":"2023-12-08T08:39:42.400396+0000","last_became_peered":"2023-12-08T08:39:42.400396+0000","last_unstale":"2023-12-08T08:39:44.472735+0000","last_undegraded":"2023-12-08T08:39:44.472735+0000","last_fullsized":"2023-12-08T08:39:44.472735+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:40:37.259965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.411289+0000","last_change":"2023-12-08T08:39:42.391259+0000","last_active":"2023-12-08T08:39:43.411289+0000","last_peered":"2023-12-08T08:39:43.411289+0000","last_clean":"2023-12-08T08:39:43.411289+0000","last_became_active":"2023-12-08T08:39:42.390839+0000","last_became_peered":"2023-12-08T08:39:42.390839+0000","last_unstale":"2023-12-08T08:39:43.411289+0000","last_undegraded":"2023-12-08T08:39:43.411289+0000","last_fullsized":"2023-12-08T08:39:43.411289+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:58:09.293121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472400+0000","last_change":"2023-12-08T08:39:42.398696+0000","last_active":"2023-12-08T08:39:44.472400+0000","last_peered":"2023-12-08T08:39:44.472400+0000","last_clean":"2023-12-08T08:39:44.472400+0000","last_became_active":"2023-12-08T08:39:42.398258+0000","last_became_peered":"2023-12-08T08:39:42.398258+0000","last_unstale":"2023-12-08T08:39:44.472400+0000","last_undegraded":"2023-12-08T08:39:44.472400+0000","last_fullsized":"2023-12-08T08:39:44.472400+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:25:53.940616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.473071+0000","last_change":"2023-12-08T08:39:42.401753+0000","last_active":"2023-12-08T08:39:44.473071+0000","last_peered":"2023-12-08T08:39:44.473071+0000","last_clean":"2023-12-08T08:39:44.473071+0000","last_became_active":"2023-12-08T08:39:42.401088+0000","last_became_peered":"2023-12-08T08:39:42.401088+0000","last_unstale":"2023-12-08T08:39:44.473071+0000","last_undegraded":"2023-12-08T08:39:44.473071+0000","last_fullsized":"2023-12-08T08:39:44.473071+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:37:15.590199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.398087+0000","last_change":"2023-12-08T08:39:42.401455+0000","last_active":"2023-12-08T08:39:43.398087+0000","last_peered":"2023-12-08T08:39:43.398087+0000","last_clean":"2023-12-08T08:39:43.398087+0000","last_became_active":"2023-12-08T08:39:42.396370+0000","last_became_peered":"2023-12-08T08:39:42.396370+0000","last_unstale":"2023-12-08T08:39:43.398087+0000","last_undegraded":"2023-12-08T08:39:43.398087+0000","last_fullsized":"2023-12-08T08:39:43.398087+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:12:35.472868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.163153+0000","last_change":"2023-12-08T08:39:42.403052+0000","last_active":"2023-12-08T08:39:44.163153+0000","last_peered":"2023-12-08T08:39:44.163153+0000","last_clean":"2023-12-08T08:39:44.163153+0000","last_became_active":"2023-12-08T08:39:42.402804+0000","last_became_peered":"2023-12-08T08:39:42.402804+0000","last_unstale":"2023-12-08T08:39:44.163153+0000","last_undegraded":"2023-12-08T08:39:44.163153+0000","last_fullsized":"2023-12-08T08:39:44.163153+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:11:31.054767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.411334+0000","last_change":"2023-12-08T08:39:42.398198+0000","last_active":"2023-12-08T08:39:43.411334+0000","last_peered":"2023-12-08T08:39:43.411334+0000","last_clean":"2023-12-08T08:39:43.411334+0000","last_became_active":"2023-12-08T08:39:42.397910+0000","last_became_peered":"2023-12-08T08:39:42.397910+0000","last_unstale":"2023-12-08T08:39:43.411334+0000","last_undegraded":"2023-12-08T08:39:43.411334+0000","last_fullsized":"2023-12-08T08:39:43.411334+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:49:25.363788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472666+0000","last_change":"2023-12-08T08:39:42.394966+0000","last_active":"2023-12-08T08:39:44.472666+0000","last_peered":"2023-12-08T08:39:44.472666+0000","last_clean":"2023-12-08T08:39:44.472666+0000","last_became_active":"2023-12-08T08:39:42.394711+0000","last_became_peered":"2023-12-08T08:39:42.394711+0000","last_unstale":"2023-12-08T08:39:44.472666+0000","last_undegraded":"2023-12-08T08:39:44.472666+0000","last_fullsized":"2023-12-08T08:39:44.472666+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:14:37.077962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472048+0000","last_change":"2023-12-08T08:39:42.389556+0000","last_active":"2023-12-08T08:39:44.472048+0000","last_peered":"2023-12-08T08:39:44.472048+0000","last_clean":"2023-12-08T08:39:44.472048+0000","last_became_active":"2023-12-08T08:39:42.389269+0000","last_became_peered":"2023-12-08T08:39:42.389269+0000","last_unstale":"2023-12-08T08:39:44.472048+0000","last_undegraded":"2023-12-08T08:39:44.472048+0000","last_fullsized":"2023-12-08T08:39:44.472048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:12:35.472868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472866+0000","last_change":"2023-12-08T08:39:42.387457+0000","last_active":"2023-12-08T08:39:44.472866+0000","last_peered":"2023-12-08T08:39:44.472866+0000","last_clean":"2023-12-08T08:39:44.472866+0000","last_became_active":"2023-12-08T08:39:42.387179+0000","last_became_peered":"2023-12-08T08:39:42.387179+0000","last_unstale":"2023-12-08T08:39:44.472866+0000","last_undegraded":"2023-12-08T08:39:44.472866+0000","last_fullsized":"2023-12-08T08:39:44.472866+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:52:50.758297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472760+0000","last_change":"2023-12-08T08:39:42.400213+0000","last_active":"2023-12-08T08:39:44.472760+0000","last_peered":"2023-12-08T08:39:44.472760+0000","last_clean":"2023-12-08T08:39:44.472760+0000","last_became_active":"2023-12-08T08:39:42.399590+0000","last_became_peered":"2023-12-08T08:39:42.399590+0000","last_unstale":"2023-12-08T08:39:44.472760+0000","last_undegraded":"2023-12-08T08:39:44.472760+0000","last_fullsized":"2023-12-08T08:39:44.472760+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:30:45.444389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472633+0000","last_change":"2023-12-08T08:39:42.393099+0000","last_active":"2023-12-08T08:39:44.472633+0000","last_peered":"2023-12-08T08:39:44.472633+0000","last_clean":"2023-12-08T08:39:44.472633+0000","last_became_active":"2023-12-08T08:39:42.392816+0000","last_became_peered":"2023-12-08T08:39:42.392816+0000","last_unstale":"2023-12-08T08:39:44.472633+0000","last_undegraded":"2023-12-08T08:39:44.472633+0000","last_fullsized":"2023-12-08T08:39:44.472633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:44:59.044021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472268+0000","last_change":"2023-12-08T08:39:42.393729+0000","last_active":"2023-12-08T08:39:44.472268+0000","last_peered":"2023-12-08T08:39:44.472268+0000","last_clean":"2023-12-08T08:39:44.472268+0000","last_became_active":"2023-12-08T08:39:42.393464+0000","last_became_peered":"2023-12-08T08:39:42.393464+0000","last_unstale":"2023-12-08T08:39:44.472268+0000","last_undegraded":"2023-12-08T08:39:44.472268+0000","last_fullsized":"2023-12-08T08:39:44.472268+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:30:22.264159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.163339+0000","last_change":"2023-12-08T08:39:42.405416+0000","last_active":"2023-12-08T08:39:44.163339+0000","last_peered":"2023-12-08T08:39:44.163339+0000","last_clean":"2023-12-08T08:39:44.163339+0000","last_became_active":"2023-12-08T08:39:42.404165+0000","last_became_peered":"2023-12-08T08:39:42.404165+0000","last_unstale":"2023-12-08T08:39:44.163339+0000","last_undegraded":"2023-12-08T08:39:44.163339+0000","last_fullsized":"2023-12-08T08:39:44.163339+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:19:43.814763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.471879+0000","last_change":"2023-12-08T08:39:42.387958+0000","last_active":"2023-12-08T08:39:44.471879+0000","last_peered":"2023-12-08T08:39:44.471879+0000","last_clean":"2023-12-08T08:39:44.471879+0000","last_became_active":"2023-12-08T08:39:42.387685+0000","last_became_peered":"2023-12-08T08:39:42.387685+0000","last_unstale":"2023-12-08T08:39:44.471879+0000","last_undegraded":"2023-12-08T08:39:44.471879+0000","last_fullsized":"2023-12-08T08:39:44.471879+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:57:08.623518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.163431+0000","last_change":"2023-12-08T08:39:42.404028+0000","last_active":"2023-12-08T08:39:44.163431+0000","last_peered":"2023-12-08T08:39:44.163431+0000","last_clean":"2023-12-08T08:39:44.163431+0000","last_became_active":"2023-12-08T08:39:42.403762+0000","last_became_peered":"2023-12-08T08:39:42.403762+0000","last_unstale":"2023-12-08T08:39:44.163431+0000","last_undegraded":"2023-12-08T08:39:44.163431+0000","last_fullsized":"2023-12-08T08:39:44.163431+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:49:40.395063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.404938+0000","last_change":"2023-12-08T08:39:41.389188+0000","last_active":"2023-12-08T08:39:42.404938+0000","last_peered":"2023-12-08T08:39:42.404938+0000","last_clean":"2023-12-08T08:39:42.404938+0000","last_became_active":"2023-12-08T08:39:41.388896+0000","last_became_peered":"2023-12-08T08:39:41.388896+0000","last_unstale":"2023-12-08T08:39:42.404938+0000","last_undegraded":"2023-12-08T08:39:42.404938+0000","last_fullsized":"2023-12-08T08:39:42.404938+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:12:28.617790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472144+0000","last_change":"2023-12-08T08:39:42.405112+0000","last_active":"2023-12-08T08:39:44.472144+0000","last_peered":"2023-12-08T08:39:44.472144+0000","last_clean":"2023-12-08T08:39:44.472144+0000","last_became_active":"2023-12-08T08:39:42.404816+0000","last_became_peered":"2023-12-08T08:39:42.404816+0000","last_unstale":"2023-12-08T08:39:44.472144+0000","last_undegraded":"2023-12-08T08:39:44.472144+0000","last_fullsized":"2023-12-08T08:39:44.472144+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:55:31.370134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.409214+0000","last_change":"2023-12-08T08:39:41.393046+0000","last_active":"2023-12-08T08:39:42.409214+0000","last_peered":"2023-12-08T08:39:42.409214+0000","last_clean":"2023-12-08T08:39:42.409214+0000","last_became_active":"2023-12-08T08:39:41.392774+0000","last_became_peered":"2023-12-08T08:39:41.392774+0000","last_unstale":"2023-12-08T08:39:42.409214+0000","last_undegraded":"2023-12-08T08:39:42.409214+0000","last_fullsized":"2023-12-08T08:39:42.409214+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:12:28.617790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.398087+0000","last_change":"2023-12-08T08:39:42.394801+0000","last_active":"2023-12-08T08:39:43.398087+0000","last_peered":"2023-12-08T08:39:43.398087+0000","last_clean":"2023-12-08T08:39:43.398087+0000","last_became_active":"2023-12-08T08:39:42.394514+0000","last_became_peered":"2023-12-08T08:39:42.394514+0000","last_unstale":"2023-12-08T08:39:43.398087+0000","last_undegraded":"2023-12-08T08:39:43.398087+0000","last_fullsized":"2023-12-08T08:39:43.398087+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:55:31.370134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.408500+0000","last_change":"2023-12-08T08:39:41.386532+0000","last_active":"2023-12-08T08:39:42.408500+0000","last_peered":"2023-12-08T08:39:42.408500+0000","last_clean":"2023-12-08T08:39:42.408500+0000","last_became_active":"2023-12-08T08:39:41.386229+0000","last_became_peered":"2023-12-08T08:39:41.386229+0000","last_unstale":"2023-12-08T08:39:42.408500+0000","last_undegraded":"2023-12-08T08:39:42.408500+0000","last_fullsized":"2023-12-08T08:39:42.408500+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:40:36.254450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.392491+0000","last_change":"2023-12-08T08:39:42.402161+0000","last_active":"2023-12-08T08:39:43.392491+0000","last_peered":"2023-12-08T08:39:43.392491+0000","last_clean":"2023-12-08T08:39:43.392491+0000","last_became_active":"2023-12-08T08:39:42.401900+0000","last_became_peered":"2023-12-08T08:39:42.401900+0000","last_unstale":"2023-12-08T08:39:43.392491+0000","last_undegraded":"2023-12-08T08:39:43.392491+0000","last_fullsized":"2023-12-08T08:39:43.392491+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T20:18:33.489640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.396239+0000","last_change":"2023-12-08T08:39:41.394756+0000","last_active":"2023-12-08T08:39:42.396239+0000","last_peered":"2023-12-08T08:39:42.396239+0000","last_clean":"2023-12-08T08:39:42.396239+0000","last_became_active":"2023-12-08T08:39:41.394456+0000","last_became_peered":"2023-12-08T08:39:41.394456+0000","last_unstale":"2023-12-08T08:39:42.396239+0000","last_undegraded":"2023-12-08T08:39:42.396239+0000","last_fullsized":"2023-12-08T08:39:42.396239+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:40:36.254450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.158443+0000","last_change":"2023-12-08T08:39:42.393892+0000","last_active":"2023-12-08T08:39:44.158443+0000","last_peered":"2023-12-08T08:39:44.158443+0000","last_clean":"2023-12-08T08:39:44.158443+0000","last_became_active":"2023-12-08T08:39:42.393619+0000","last_became_peered":"2023-12-08T08:39:42.393619+0000","last_unstale":"2023-12-08T08:39:44.158443+0000","last_undegraded":"2023-12-08T08:39:44.158443+0000","last_fullsized":"2023-12-08T08:39:44.158443+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:37:13.047884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.402587+0000","last_change":"2023-12-08T08:39:41.393021+0000","last_active":"2023-12-08T08:39:42.402587+0000","last_peered":"2023-12-08T08:39:42.402587+0000","last_clean":"2023-12-08T08:39:42.402587+0000","last_became_active":"2023-12-08T08:39:41.392703+0000","last_became_peered":"2023-12-08T08:39:41.392703+0000","last_unstale":"2023-12-08T08:39:42.402587+0000","last_undegraded":"2023-12-08T08:39:42.402587+0000","last_fullsized":"2023-12-08T08:39:42.402587+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:40:58.873848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.473008+0000","last_change":"2023-12-08T08:39:42.400754+0000","last_active":"2023-12-08T08:39:44.473008+0000","last_peered":"2023-12-08T08:39:44.473008+0000","last_clean":"2023-12-08T08:39:44.473008+0000","last_became_active":"2023-12-08T08:39:42.400472+0000","last_became_peered":"2023-12-08T08:39:42.400472+0000","last_unstale":"2023-12-08T08:39:44.473008+0000","last_undegraded":"2023-12-08T08:39:44.473008+0000","last_fullsized":"2023-12-08T08:39:44.473008+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:11:13.945747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.407507+0000","last_change":"2023-12-08T08:39:41.393095+0000","last_active":"2023-12-08T08:39:42.407507+0000","last_peered":"2023-12-08T08:39:42.407507+0000","last_clean":"2023-12-08T08:39:42.407507+0000","last_became_active":"2023-12-08T08:39:41.392794+0000","last_became_peered":"2023-12-08T08:39:41.392794+0000","last_unstale":"2023-12-08T08:39:42.407507+0000","last_undegraded":"2023-12-08T08:39:42.407507+0000","last_fullsized":"2023-12-08T08:39:42.407507+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:30:21.258643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.398244+0000","last_change":"2023-12-08T08:39:42.402503+0000","last_active":"2023-12-08T08:39:43.398244+0000","last_peered":"2023-12-08T08:39:43.398244+0000","last_clean":"2023-12-08T08:39:43.398244+0000","last_became_active":"2023-12-08T08:39:42.402050+0000","last_became_peered":"2023-12-08T08:39:42.402050+0000","last_unstale":"2023-12-08T08:39:43.398244+0000","last_undegraded":"2023-12-08T08:39:43.398244+0000","last_fullsized":"2023-12-08T08:39:43.398244+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:11:31.054767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.379730+0000","last_change":"2023-12-08T08:39:41.388680+0000","last_active":"2023-12-08T08:39:42.379730+0000","last_peered":"2023-12-08T08:39:42.379730+0000","last_clean":"2023-12-08T08:39:42.379730+0000","last_became_active":"2023-12-08T08:39:41.388400+0000","last_became_peered":"2023-12-08T08:39:41.388400+0000","last_unstale":"2023-12-08T08:39:42.379730+0000","last_undegraded":"2023-12-08T08:39:42.379730+0000","last_fullsized":"2023-12-08T08:39:42.379730+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:12:28.617790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.392200+0000","last_change":"2023-12-08T08:39:42.406727+0000","last_active":"2023-12-08T08:39:43.392200+0000","last_peered":"2023-12-08T08:39:43.392200+0000","last_clean":"2023-12-08T08:39:43.392200+0000","last_became_active":"2023-12-08T08:39:42.406236+0000","last_became_peered":"2023-12-08T08:39:42.406236+0000","last_unstale":"2023-12-08T08:39:43.392200+0000","last_undegraded":"2023-12-08T08:39:43.392200+0000","last_fullsized":"2023-12-08T08:39:43.392200+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:18:47.663728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.382874+0000","last_change":"2023-12-08T08:39:41.391299+0000","last_active":"2023-12-08T08:39:42.382874+0000","last_peered":"2023-12-08T08:39:42.382874+0000","last_clean":"2023-12-08T08:39:42.382874+0000","last_became_active":"2023-12-08T08:39:41.391010+0000","last_became_peered":"2023-12-08T08:39:41.391010+0000","last_unstale":"2023-12-08T08:39:42.382874+0000","last_undegraded":"2023-12-08T08:39:42.382874+0000","last_fullsized":"2023-12-08T08:39:42.382874+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:14:36.072447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.411504+0000","last_change":"2023-12-08T08:39:42.395177+0000","last_active":"2023-12-08T08:39:43.411504+0000","last_peered":"2023-12-08T08:39:43.411504+0000","last_clean":"2023-12-08T08:39:43.411504+0000","last_became_active":"2023-12-08T08:39:42.394909+0000","last_became_peered":"2023-12-08T08:39:42.394909+0000","last_unstale":"2023-12-08T08:39:43.411504+0000","last_undegraded":"2023-12-08T08:39:43.411504+0000","last_fullsized":"2023-12-08T08:39:43.411504+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:12:29.623306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.384116+0000","last_change":"2023-12-08T08:39:41.391711+0000","last_active":"2023-12-08T08:39:42.384116+0000","last_peered":"2023-12-08T08:39:42.384116+0000","last_clean":"2023-12-08T08:39:42.384116+0000","last_became_active":"2023-12-08T08:39:41.391286+0000","last_became_peered":"2023-12-08T08:39:41.391286+0000","last_unstale":"2023-12-08T08:39:42.384116+0000","last_undegraded":"2023-12-08T08:39:42.384116+0000","last_fullsized":"2023-12-08T08:39:42.384116+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:48:48.030769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.163517+0000","last_change":"2023-12-08T08:39:42.402379+0000","last_active":"2023-12-08T08:39:44.163517+0000","last_peered":"2023-12-08T08:39:44.163517+0000","last_clean":"2023-12-08T08:39:44.163517+0000","last_became_active":"2023-12-08T08:39:42.401414+0000","last_became_peered":"2023-12-08T08:39:42.401414+0000","last_unstale":"2023-12-08T08:39:44.163517+0000","last_undegraded":"2023-12-08T08:39:44.163517+0000","last_fullsized":"2023-12-08T08:39:44.163517+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:48:49.036284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.384177+0000","last_change":"2023-12-08T08:39:41.390127+0000","last_active":"2023-12-08T08:39:42.384177+0000","last_peered":"2023-12-08T08:39:42.384177+0000","last_clean":"2023-12-08T08:39:42.384177+0000","last_became_active":"2023-12-08T08:39:41.389825+0000","last_became_peered":"2023-12-08T08:39:41.389825+0000","last_unstale":"2023-12-08T08:39:42.384177+0000","last_undegraded":"2023-12-08T08:39:42.384177+0000","last_fullsized":"2023-12-08T08:39:42.384177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:55:30.364619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.158834+0000","last_change":"2023-12-08T08:39:42.398662+0000","last_active":"2023-12-08T08:39:44.158834+0000","last_peered":"2023-12-08T08:39:44.158834+0000","last_clean":"2023-12-08T08:39:44.158834+0000","last_became_active":"2023-12-08T08:39:42.398357+0000","last_became_peered":"2023-12-08T08:39:42.398357+0000","last_unstale":"2023-12-08T08:39:44.158834+0000","last_undegraded":"2023-12-08T08:39:44.158834+0000","last_fullsized":"2023-12-08T08:39:44.158834+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:46.427503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.402814+0000","last_change":"2023-12-08T08:39:41.390974+0000","last_active":"2023-12-08T08:39:42.402814+0000","last_peered":"2023-12-08T08:39:42.402814+0000","last_clean":"2023-12-08T08:39:42.402814+0000","last_became_active":"2023-12-08T08:39:41.390522+0000","last_became_peered":"2023-12-08T08:39:41.390522+0000","last_unstale":"2023-12-08T08:39:42.402814+0000","last_undegraded":"2023-12-08T08:39:42.402814+0000","last_fullsized":"2023-12-08T08:39:42.402814+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T20:07:39.876198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.398167+0000","last_change":"2023-12-08T08:39:42.395894+0000","last_active":"2023-12-08T08:39:43.398167+0000","last_peered":"2023-12-08T08:39:43.398167+0000","last_clean":"2023-12-08T08:39:43.398167+0000","last_became_active":"2023-12-08T08:39:42.394146+0000","last_became_peered":"2023-12-08T08:39:42.394146+0000","last_unstale":"2023-12-08T08:39:43.398167+0000","last_undegraded":"2023-12-08T08:39:43.398167+0000","last_fullsized":"2023-12-08T08:39:43.398167+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:29:10.077076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.411143+0000","last_change":"2023-12-08T08:39:41.390718+0000","last_active":"2023-12-08T08:39:42.411143+0000","last_peered":"2023-12-08T08:39:42.411143+0000","last_clean":"2023-12-08T08:39:42.411143+0000","last_became_active":"2023-12-08T08:39:41.390427+0000","last_became_peered":"2023-12-08T08:39:41.390427+0000","last_unstale":"2023-12-08T08:39:42.411143+0000","last_undegraded":"2023-12-08T08:39:42.411143+0000","last_fullsized":"2023-12-08T08:39:42.411143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:19:42.809248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.158621+0000","last_change":"2023-12-08T08:39:42.403354+0000","last_active":"2023-12-08T08:39:44.158621+0000","last_peered":"2023-12-08T08:39:44.158621+0000","last_clean":"2023-12-08T08:39:44.158621+0000","last_became_active":"2023-12-08T08:39:42.403084+0000","last_became_peered":"2023-12-08T08:39:42.403084+0000","last_unstale":"2023-12-08T08:39:44.158621+0000","last_undegraded":"2023-12-08T08:39:44.158621+0000","last_fullsized":"2023-12-08T08:39:44.158621+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:40:59.879364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.405143+0000","last_change":"2023-12-08T08:39:41.383829+0000","last_active":"2023-12-08T08:39:42.405143+0000","last_peered":"2023-12-08T08:39:42.405143+0000","last_clean":"2023-12-08T08:39:42.405143+0000","last_became_active":"2023-12-08T08:39:41.383527+0000","last_became_peered":"2023-12-08T08:39:41.383527+0000","last_unstale":"2023-12-08T08:39:42.405143+0000","last_undegraded":"2023-12-08T08:39:42.405143+0000","last_fullsized":"2023-12-08T08:39:42.405143+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T20:07:39.876198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.392163+0000","last_change":"2023-12-08T08:39:42.400648+0000","last_active":"2023-12-08T08:39:43.392163+0000","last_peered":"2023-12-08T08:39:43.392163+0000","last_clean":"2023-12-08T08:39:43.392163+0000","last_became_active":"2023-12-08T08:39:42.400297+0000","last_became_peered":"2023-12-08T08:39:42.400297+0000","last_unstale":"2023-12-08T08:39:43.392163+0000","last_undegraded":"2023-12-08T08:39:43.392163+0000","last_fullsized":"2023-12-08T08:39:43.392163+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:10:31.013022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.380255+0000","last_change":"2023-12-08T08:39:41.387186+0000","last_active":"2023-12-08T08:39:42.380255+0000","last_peered":"2023-12-08T08:39:42.380255+0000","last_clean":"2023-12-08T08:39:42.380255+0000","last_became_active":"2023-12-08T08:39:41.386906+0000","last_became_peered":"2023-12-08T08:39:41.386906+0000","last_unstale":"2023-12-08T08:39:42.380255+0000","last_undegraded":"2023-12-08T08:39:42.380255+0000","last_fullsized":"2023-12-08T08:39:42.380255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:44:58.038506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.163254+0000","last_change":"2023-12-08T08:39:42.404403+0000","last_active":"2023-12-08T08:39:44.163254+0000","last_peered":"2023-12-08T08:39:44.163254+0000","last_clean":"2023-12-08T08:39:44.163254+0000","last_became_active":"2023-12-08T08:39:42.402955+0000","last_became_peered":"2023-12-08T08:39:42.402955+0000","last_unstale":"2023-12-08T08:39:44.163254+0000","last_undegraded":"2023-12-08T08:39:44.163254+0000","last_fullsized":"2023-12-08T08:39:44.163254+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:30:46.427503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.381360+0000","last_change":"2023-12-08T08:39:41.387515+0000","last_active":"2023-12-08T08:39:42.381360+0000","last_peered":"2023-12-08T08:39:42.381360+0000","last_clean":"2023-12-08T08:39:42.381360+0000","last_became_active":"2023-12-08T08:39:41.387249+0000","last_became_peered":"2023-12-08T08:39:41.387249+0000","last_unstale":"2023-12-08T08:39:42.381360+0000","last_undegraded":"2023-12-08T08:39:42.381360+0000","last_fullsized":"2023-12-08T08:39:42.381360+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:46:59.905007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.392256+0000","last_change":"2023-12-08T08:39:42.396034+0000","last_active":"2023-12-08T08:39:43.392256+0000","last_peered":"2023-12-08T08:39:43.392256+0000","last_clean":"2023-12-08T08:39:43.392256+0000","last_became_active":"2023-12-08T08:39:42.395744+0000","last_became_peered":"2023-12-08T08:39:42.395744+0000","last_unstale":"2023-12-08T08:39:43.392256+0000","last_undegraded":"2023-12-08T08:39:43.392256+0000","last_fullsized":"2023-12-08T08:39:43.392256+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T12:24:07.734572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.406990+0000","last_change":"2023-12-08T08:39:41.390767+0000","last_active":"2023-12-08T08:39:42.406990+0000","last_peered":"2023-12-08T08:39:42.406990+0000","last_clean":"2023-12-08T08:39:42.406990+0000","last_became_active":"2023-12-08T08:39:41.390480+0000","last_became_peered":"2023-12-08T08:39:41.390480+0000","last_unstale":"2023-12-08T08:39:42.406990+0000","last_undegraded":"2023-12-08T08:39:42.406990+0000","last_fullsized":"2023-12-08T08:39:42.406990+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:46:59.905007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.392481+0000","last_change":"2023-12-08T08:39:42.391262+0000","last_active":"2023-12-08T08:39:43.392481+0000","last_peered":"2023-12-08T08:39:43.392481+0000","last_clean":"2023-12-08T08:39:43.392481+0000","last_became_active":"2023-12-08T08:39:42.391000+0000","last_became_peered":"2023-12-08T08:39:42.391000+0000","last_unstale":"2023-12-08T08:39:43.392481+0000","last_undegraded":"2023-12-08T08:39:43.392481+0000","last_fullsized":"2023-12-08T08:39:43.392481+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:33:51.731709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.403174+0000","last_change":"2023-12-08T08:39:41.388904+0000","last_active":"2023-12-08T08:39:42.403174+0000","last_peered":"2023-12-08T08:39:42.403174+0000","last_clean":"2023-12-08T08:39:42.403174+0000","last_became_active":"2023-12-08T08:39:41.388633+0000","last_became_peered":"2023-12-08T08:39:41.388633+0000","last_unstale":"2023-12-08T08:39:42.403174+0000","last_undegraded":"2023-12-08T08:39:42.403174+0000","last_fullsized":"2023-12-08T08:39:42.403174+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:29:09.071561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472635+0000","last_change":"2023-12-08T08:39:42.402202+0000","last_active":"2023-12-08T08:39:44.472635+0000","last_peered":"2023-12-08T08:39:44.472635+0000","last_clean":"2023-12-08T08:39:44.472635+0000","last_became_active":"2023-12-08T08:39:42.401805+0000","last_became_peered":"2023-12-08T08:39:42.401805+0000","last_unstale":"2023-12-08T08:39:44.472635+0000","last_undegraded":"2023-12-08T08:39:44.472635+0000","last_fullsized":"2023-12-08T08:39:44.472635+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:19:45.648896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.404453+0000","last_change":"2023-12-08T08:39:41.398374+0000","last_active":"2023-12-08T08:39:42.404453+0000","last_peered":"2023-12-08T08:39:42.404453+0000","last_clean":"2023-12-08T08:39:42.404453+0000","last_became_active":"2023-12-08T08:39:41.397955+0000","last_became_peered":"2023-12-08T08:39:41.397955+0000","last_unstale":"2023-12-08T08:39:42.404453+0000","last_undegraded":"2023-12-08T08:39:42.404453+0000","last_fullsized":"2023-12-08T08:39:42.404453+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:52:49.752781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.398015+0000","last_change":"2023-12-08T08:39:42.395231+0000","last_active":"2023-12-08T08:39:43.398015+0000","last_peered":"2023-12-08T08:39:43.398015+0000","last_clean":"2023-12-08T08:39:43.398015+0000","last_became_active":"2023-12-08T08:39:42.394804+0000","last_became_peered":"2023-12-08T08:39:42.394804+0000","last_unstale":"2023-12-08T08:39:43.398015+0000","last_undegraded":"2023-12-08T08:39:43.398015+0000","last_fullsized":"2023-12-08T08:39:43.398015+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:40:59.879364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.406944+0000","last_change":"2023-12-08T08:39:41.389339+0000","last_active":"2023-12-08T08:39:42.406944+0000","last_peered":"2023-12-08T08:39:42.406944+0000","last_clean":"2023-12-08T08:39:42.406944+0000","last_became_active":"2023-12-08T08:39:41.389080+0000","last_became_peered":"2023-12-08T08:39:41.389080+0000","last_unstale":"2023-12-08T08:39:42.406944+0000","last_undegraded":"2023-12-08T08:39:42.406944+0000","last_fullsized":"2023-12-08T08:39:42.406944+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:18:11.165751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.411267+0000","last_change":"2023-12-08T08:39:42.396267+0000","last_active":"2023-12-08T08:39:43.411267+0000","last_peered":"2023-12-08T08:39:43.411267+0000","last_clean":"2023-12-08T08:39:43.411267+0000","last_became_active":"2023-12-08T08:39:42.395983+0000","last_became_peered":"2023-12-08T08:39:42.395983+0000","last_unstale":"2023-12-08T08:39:43.411267+0000","last_undegraded":"2023-12-08T08:39:43.411267+0000","last_fullsized":"2023-12-08T08:39:43.411267+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T10:52:25.376404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.379822+0000","last_change":"2023-12-08T08:39:41.385857+0000","last_active":"2023-12-08T08:39:42.379822+0000","last_peered":"2023-12-08T08:39:42.379822+0000","last_clean":"2023-12-08T08:39:42.379822+0000","last_became_active":"2023-12-08T08:39:41.385526+0000","last_became_peered":"2023-12-08T08:39:41.385526+0000","last_unstale":"2023-12-08T08:39:42.379822+0000","last_undegraded":"2023-12-08T08:39:42.379822+0000","last_fullsized":"2023-12-08T08:39:42.379822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:30:21.258643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.163312+0000","last_change":"2023-12-08T08:39:42.405268+0000","last_active":"2023-12-08T08:39:44.163312+0000","last_peered":"2023-12-08T08:39:44.163312+0000","last_clean":"2023-12-08T08:39:44.163312+0000","last_became_active":"2023-12-08T08:39:42.404764+0000","last_became_peered":"2023-12-08T08:39:42.404764+0000","last_unstale":"2023-12-08T08:39:44.163312+0000","last_undegraded":"2023-12-08T08:39:44.163312+0000","last_fullsized":"2023-12-08T08:39:44.163312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T20:07:40.881713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.400250+0000","last_change":"2023-12-08T08:39:41.388074+0000","last_active":"2023-12-08T08:39:42.400250+0000","last_peered":"2023-12-08T08:39:42.400250+0000","last_clean":"2023-12-08T08:39:42.400250+0000","last_became_active":"2023-12-08T08:39:41.387785+0000","last_became_peered":"2023-12-08T08:39:41.387785+0000","last_unstale":"2023-12-08T08:39:42.400250+0000","last_undegraded":"2023-12-08T08:39:42.400250+0000","last_fullsized":"2023-12-08T08:39:42.400250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:40:35.890328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.158549+0000","last_change":"2023-12-08T08:39:42.398291+0000","last_active":"2023-12-08T08:39:44.158549+0000","last_peered":"2023-12-08T08:39:44.158549+0000","last_clean":"2023-12-08T08:39:44.158549+0000","last_became_active":"2023-12-08T08:39:42.397991+0000","last_became_peered":"2023-12-08T08:39:42.397991+0000","last_unstale":"2023-12-08T08:39:44.158549+0000","last_undegraded":"2023-12-08T08:39:44.158549+0000","last_fullsized":"2023-12-08T08:39:44.158549+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:12:35.472868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.375487+0000","last_change":"2023-12-08T08:39:41.386856+0000","last_active":"2023-12-08T08:39:42.375487+0000","last_peered":"2023-12-08T08:39:42.375487+0000","last_clean":"2023-12-08T08:39:42.375487+0000","last_became_active":"2023-12-08T08:39:41.386554+0000","last_became_peered":"2023-12-08T08:39:41.386554+0000","last_unstale":"2023-12-08T08:39:42.375487+0000","last_undegraded":"2023-12-08T08:39:42.375487+0000","last_fullsized":"2023-12-08T08:39:42.375487+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T09:14:55.418483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472267+0000","last_change":"2023-12-08T08:39:42.397033+0000","last_active":"2023-12-08T08:39:44.472267+0000","last_peered":"2023-12-08T08:39:44.472267+0000","last_clean":"2023-12-08T08:39:44.472267+0000","last_became_active":"2023-12-08T08:39:42.394191+0000","last_became_peered":"2023-12-08T08:39:42.394191+0000","last_unstale":"2023-12-08T08:39:44.472267+0000","last_undegraded":"2023-12-08T08:39:44.472267+0000","last_fullsized":"2023-12-08T08:39:44.472267+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:49:40.395063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.389195+0000","last_change":"2023-12-08T08:39:41.391302+0000","last_active":"2023-12-08T08:39:42.389195+0000","last_peered":"2023-12-08T08:39:42.389195+0000","last_clean":"2023-12-08T08:39:42.389195+0000","last_became_active":"2023-12-08T08:39:41.391031+0000","last_became_peered":"2023-12-08T08:39:41.391031+0000","last_unstale":"2023-12-08T08:39:42.389195+0000","last_undegraded":"2023-12-08T08:39:42.389195+0000","last_fullsized":"2023-12-08T08:39:42.389195+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:14:36.072447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.392333+0000","last_change":"2023-12-08T08:39:42.406430+0000","last_active":"2023-12-08T08:39:43.392333+0000","last_peered":"2023-12-08T08:39:43.392333+0000","last_clean":"2023-12-08T08:39:43.392333+0000","last_became_active":"2023-12-08T08:39:42.406095+0000","last_became_peered":"2023-12-08T08:39:42.406095+0000","last_unstale":"2023-12-08T08:39:43.392333+0000","last_undegraded":"2023-12-08T08:39:43.392333+0000","last_fullsized":"2023-12-08T08:39:43.392333+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:11:13.945747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.375923+0000","last_change":"2023-12-08T08:39:41.388680+0000","last_active":"2023-12-08T08:39:42.375923+0000","last_peered":"2023-12-08T08:39:42.375923+0000","last_clean":"2023-12-08T08:39:42.375923+0000","last_became_active":"2023-12-08T08:39:41.388422+0000","last_became_peered":"2023-12-08T08:39:41.388422+0000","last_unstale":"2023-12-08T08:39:42.375923+0000","last_undegraded":"2023-12-08T08:39:42.375923+0000","last_fullsized":"2023-12-08T08:39:42.375923+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:09:45.614968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472128+0000","last_change":"2023-12-08T08:39:42.388897+0000","last_active":"2023-12-08T08:39:44.472128+0000","last_peered":"2023-12-08T08:39:44.472128+0000","last_clean":"2023-12-08T08:39:44.472128+0000","last_became_active":"2023-12-08T08:39:42.388609+0000","last_became_peered":"2023-12-08T08:39:42.388609+0000","last_unstale":"2023-12-08T08:39:44.472128+0000","last_undegraded":"2023-12-08T08:39:44.472128+0000","last_fullsized":"2023-12-08T08:39:44.472128+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:40:59.879364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.402295+0000","last_change":"2023-12-08T08:39:41.390026+0000","last_active":"2023-12-08T08:39:42.402295+0000","last_peered":"2023-12-08T08:39:42.402295+0000","last_clean":"2023-12-08T08:39:42.402295+0000","last_became_active":"2023-12-08T08:39:41.389595+0000","last_became_peered":"2023-12-08T08:39:41.389595+0000","last_unstale":"2023-12-08T08:39:42.402295+0000","last_undegraded":"2023-12-08T08:39:42.402295+0000","last_fullsized":"2023-12-08T08:39:42.402295+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:11:30.049252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.163586+0000","last_change":"2023-12-08T08:39:42.402194+0000","last_active":"2023-12-08T08:39:44.163586+0000","last_peered":"2023-12-08T08:39:44.163586+0000","last_clean":"2023-12-08T08:39:44.163586+0000","last_became_active":"2023-12-08T08:39:42.401758+0000","last_became_peered":"2023-12-08T08:39:42.401758+0000","last_unstale":"2023-12-08T08:39:44.163586+0000","last_undegraded":"2023-12-08T08:39:44.163586+0000","last_fullsized":"2023-12-08T08:39:44.163586+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:19:45.648896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.411220+0000","last_change":"2023-12-08T08:39:41.391017+0000","last_active":"2023-12-08T08:39:43.411220+0000","last_peered":"2023-12-08T08:39:43.411220+0000","last_clean":"2023-12-08T08:39:43.411220+0000","last_became_active":"2023-12-08T08:39:41.390670+0000","last_became_peered":"2023-12-08T08:39:41.390670+0000","last_unstale":"2023-12-08T08:39:43.411220+0000","last_undegraded":"2023-12-08T08:39:43.411220+0000","last_fullsized":"2023-12-08T08:39:43.411220+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:08:45.582616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.392390+0000","last_change":"2023-12-08T08:39:42.397462+0000","last_active":"2023-12-08T08:39:43.392390+0000","last_peered":"2023-12-08T08:39:43.392390+0000","last_clean":"2023-12-08T08:39:43.392390+0000","last_became_active":"2023-12-08T08:39:42.396868+0000","last_became_peered":"2023-12-08T08:39:42.396868+0000","last_unstale":"2023-12-08T08:39:43.392390+0000","last_undegraded":"2023-12-08T08:39:43.392390+0000","last_fullsized":"2023-12-08T08:39:43.392390+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T16:37:15.590199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.382428+0000","last_change":"2023-12-08T08:39:41.384742+0000","last_active":"2023-12-08T08:39:42.382428+0000","last_peered":"2023-12-08T08:39:42.382428+0000","last_clean":"2023-12-08T08:39:42.382428+0000","last_became_active":"2023-12-08T08:39:41.384430+0000","last_became_peered":"2023-12-08T08:39:41.384430+0000","last_unstale":"2023-12-08T08:39:42.382428+0000","last_undegraded":"2023-12-08T08:39:42.382428+0000","last_fullsized":"2023-12-08T08:39:42.382428+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:52:49.752781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.158717+0000","last_change":"2023-12-08T08:39:42.390596+0000","last_active":"2023-12-08T08:39:44.158717+0000","last_peered":"2023-12-08T08:39:44.158717+0000","last_clean":"2023-12-08T08:39:44.158717+0000","last_became_active":"2023-12-08T08:39:42.390095+0000","last_became_peered":"2023-12-08T08:39:42.390095+0000","last_unstale":"2023-12-08T08:39:44.158717+0000","last_undegraded":"2023-12-08T08:39:44.158717+0000","last_fullsized":"2023-12-08T08:39:44.158717+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T13:29:10.077076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.381300+0000","last_change":"2023-12-08T08:39:41.393187+0000","last_active":"2023-12-08T08:39:42.381300+0000","last_peered":"2023-12-08T08:39:42.381300+0000","last_clean":"2023-12-08T08:39:42.381300+0000","last_became_active":"2023-12-08T08:39:41.392886+0000","last_became_peered":"2023-12-08T08:39:41.392886+0000","last_unstale":"2023-12-08T08:39:42.381300+0000","last_undegraded":"2023-12-08T08:39:42.381300+0000","last_fullsized":"2023-12-08T08:39:42.381300+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:44:58.038506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.471841+0000","last_change":"2023-12-08T08:39:42.389121+0000","last_active":"2023-12-08T08:39:44.471841+0000","last_peered":"2023-12-08T08:39:44.471841+0000","last_clean":"2023-12-08T08:39:44.471841+0000","last_became_active":"2023-12-08T08:39:42.388838+0000","last_became_peered":"2023-12-08T08:39:42.388838+0000","last_unstale":"2023-12-08T08:39:44.471841+0000","last_undegraded":"2023-12-08T08:39:44.471841+0000","last_fullsized":"2023-12-08T08:39:44.471841+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:52:50.758297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.381514+0000","last_change":"2023-12-08T08:39:41.382979+0000","last_active":"2023-12-08T08:39:42.381514+0000","last_peered":"2023-12-08T08:39:42.381514+0000","last_clean":"2023-12-08T08:39:42.381514+0000","last_became_active":"2023-12-08T08:39:41.382716+0000","last_became_peered":"2023-12-08T08:39:41.382716+0000","last_unstale":"2023-12-08T08:39:42.381514+0000","last_undegraded":"2023-12-08T08:39:42.381514+0000","last_fullsized":"2023-12-08T08:39:42.381514+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:49:24.358272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:43.397716+0000","last_change":"2023-12-08T08:39:42.401633+0000","last_active":"2023-12-08T08:39:43.397716+0000","last_peered":"2023-12-08T08:39:43.397716+0000","last_clean":"2023-12-08T08:39:43.397716+0000","last_became_active":"2023-12-08T08:39:42.398818+0000","last_became_peered":"2023-12-08T08:39:42.398818+0000","last_unstale":"2023-12-08T08:39:43.397716+0000","last_undegraded":"2023-12-08T08:39:43.397716+0000","last_fullsized":"2023-12-08T08:39:43.397716+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T11:57:08.623518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.382835+0000","last_change":"2023-12-08T08:39:41.390769+0000","last_active":"2023-12-08T08:39:42.382835+0000","last_peered":"2023-12-08T08:39:42.382835+0000","last_clean":"2023-12-08T08:39:42.382835+0000","last_became_active":"2023-12-08T08:39:41.390509+0000","last_became_peered":"2023-12-08T08:39:41.390509+0000","last_unstale":"2023-12-08T08:39:42.382835+0000","last_undegraded":"2023-12-08T08:39:42.382835+0000","last_fullsized":"2023-12-08T08:39:42.382835+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T17:44:58.038506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.163438+0000","last_change":"2023-12-08T08:39:42.400753+0000","last_active":"2023-12-08T08:39:44.163438+0000","last_peered":"2023-12-08T08:39:44.163438+0000","last_clean":"2023-12-08T08:39:44.163438+0000","last_became_active":"2023-12-08T08:39:42.400509+0000","last_became_peered":"2023-12-08T08:39:42.400509+0000","last_unstale":"2023-12-08T08:39:44.163438+0000","last_undegraded":"2023-12-08T08:39:44.163438+0000","last_fullsized":"2023-12-08T08:39:44.163438+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:36.410906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.384072+0000","last_change":"2023-12-08T08:39:41.391141+0000","last_active":"2023-12-08T08:39:42.384072+0000","last_peered":"2023-12-08T08:39:42.384072+0000","last_clean":"2023-12-08T08:39:42.384072+0000","last_became_active":"2023-12-08T08:39:41.390865+0000","last_became_peered":"2023-12-08T08:39:41.390865+0000","last_unstale":"2023-12-08T08:39:42.384072+0000","last_undegraded":"2023-12-08T08:39:42.384072+0000","last_fullsized":"2023-12-08T08:39:42.384072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:35.405391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-12-08T08:39:44.472476+0000","last_change":"2023-12-08T08:39:42.396447+0000","last_active":"2023-12-08T08:39:44.472476+0000","last_peered":"2023-12-08T08:39:44.472476+0000","last_clean":"2023-12-08T08:39:44.472476+0000","last_became_active":"2023-12-08T08:39:42.396144+0000","last_became_peered":"2023-12-08T08:39:42.396144+0000","last_unstale":"2023-12-08T08:39:44.472476+0000","last_undegraded":"2023-12-08T08:39:44.472476+0000","last_fullsized":"2023-12-08T08:39:44.472476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:41.366630+0000","last_clean_scrub_stamp":"2023-12-08T08:39:41.366630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T18:08:36.410906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.391089+0000","last_change":"2023-12-08T08:39:41.391855+0000","last_active":"2023-12-08T08:39:42.391089+0000","last_peered":"2023-12-08T08:39:42.391089+0000","last_clean":"2023-12-08T08:39:42.391089+0000","last_became_active":"2023-12-08T08:39:41.391336+0000","last_became_peered":"2023-12-08T08:39:41.391336+0000","last_unstale":"2023-12-08T08:39:42.391089+0000","last_undegraded":"2023-12-08T08:39:42.391089+0000","last_fullsized":"2023-12-08T08:39:42.391089+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:39:40.361115+0000","last_clean_scrub_stamp":"2023-12-08T08:39:40.361115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T19:52:49.752781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":179,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-12-08T08:39:42.387743+0000","last_change":"2023-12-08T08:38:08.276815+0000","last_active":"2023-12-08T08:39:42.387743+0000","last_peered":"2023-12-08T08:39:42.387743+0000","last_clean":"2023-12-08T08:39:42.387743+0000","last_became_active":"2023-12-08T08:38:08.275828+0000","last_became_peered":"2023-12-08T08:38:08.275828+0000","last_unstale":"2023-12-08T08:39:42.387743+0000","last_undegraded":"2023-12-08T08:39:42.387743+0000","last_fullsized":"2023-12-08T08:39:42.387743+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-08T08:38:07.104197+0000","last_clean_scrub_stamp":"2023-12-08T08:38:07.104197+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-09T15:35:34.851149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":786331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.58699999999999997}]},{"osd":2,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.374},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.54100000000000004}]},{"osd":3,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.45100000000000001}]},{"osd":5,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.47099999999999997}]},{"osd":6,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56000000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":194863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.56899999999999995}]},{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.54100000000000004}]},{"osd":2,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.51000000000000001}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.63800000000000001}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.626},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.65000000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":194393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.628}]},{"osd":1,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.57799999999999996}]},{"osd":2,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.48699999999999999}]},{"osd":3,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.66400000000000003}]},{"osd":4,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.5}]},{"osd":6,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.59899999999999998}]},{"osd":7,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.52300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":194885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.60899999999999999}]},{"osd":1,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.629}]},{"osd":2,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.51600000000000001}]},{"osd":3,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Dec 8 08:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.56299999999999994}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31644,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340196,"statfs":{"total":96636764160,"available":96604360704,"internally_reserved":0,"allocated":466944,"data_stored":195927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":7.3159999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":3.448},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":7.5030000000000001}]},{"osd":2,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":2.7639999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":8.1440000000000001}]},{"osd":4,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.115,"5min":1.115,"15min":1.115},"last":4.8259999999999996}]},{"osd":5,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":7.0410000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":7.6020000000000003}]},{"osd":6,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":7.3840000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.93500000000000005}]},{"osd":7,"last update":"Fri Dec 8 08:39:07 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":2.839}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":454656,"data_stored":194351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.36099999999999999}]},{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.61699999999999999}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.59599999999999997}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.51000000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":194371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.417,"5min":1.417,"15min":1.417},"last":0.52200000000000002}]},{"osd":2,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.61799999999999999}]},{"osd":3,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.81899999999999995}]},{"osd":5,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.79900000000000004}]},{"osd":6,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.76800000000000002}]},{"osd":7,"last update":"Fri Dec 8 08:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.751}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1048576,"data_stored":784683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.42199999999999999}]},{"osd":2,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.438},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.377}]},{"osd":3,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.496}]},{"osd":4,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.74299999999999999}]},{"osd":5,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.524,"5min":1.524,"15min":1.524},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Dec 8 08:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-08T08:39:50.169 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-08T08:39:50.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-08T08:39:50.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.315+0000 7fed48941640 1 Processor -- start 2023-12-08T08:39:50.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.316+0000 7fed48941640 1 -- start start 2023-12-08T08:39:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.316+0000 7fed48941640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed441608a0 0x7fed440d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.316+0000 7fed48941640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fed440d1c90 0x7fed440c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.316+0000 7fed48941640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed440c6af0 con 0x7fed44160c80 2023-12-08T08:39:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.316+0000 7fed48941640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed440d12c0 con 0x7fed440d1c90 2023-12-08T08:39:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.316+0000 7fed48941640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed440d0500 con 0x7fed441608a0 2023-12-08T08:39:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.317+0000 7fed4259c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed441608a0 0x7fed440d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.317+0000 7fed42d9d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7fed440d1c90 0x7fed440c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.317+0000 7fed4259c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed441608a0 0x7fed440d0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3301/0 says I am v2:172.21.15.16:35546/0 (socket says 172.21.15.16:35546) 2023-12-08T08:39:50.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.317+0000 7fed4259c640 1 -- 172.21.15.16:0/1876001611 learned_addr learned my addr 172.21.15.16:0/1876001611 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.317+0000 7fed3bfff640 1 --1- >> v1:172.21.15.112:6789/0 conn(0x7fed44160c80 0x7fed440d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:34336/0 (socket says 172.21.15.16:34336) 2023-12-08T08:39:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.317+0000 7fed4259c640 1 -- 172.21.15.16:0/1876001611 >> v2:172.21.15.112:3300/0 conn(0x7fed440d1c90 msgr2=0x7fed440c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.317+0000 7fed4259c640 1 --2- 172.21.15.16:0/1876001611 >> v2:172.21.15.112:3300/0 conn(0x7fed440d1c90 0x7fed440c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.317+0000 7fed4259c640 1 -- 172.21.15.16:0/1876001611 >> v1:172.21.15.112:6789/0 conn(0x7fed44160c80 legacy=0x7fed440d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.317+0000 7fed4259c640 1 -- 172.21.15.16:0/1876001611 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed440d07e0 con 0x7fed441608a0 2023-12-08T08:39:50.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.317+0000 7fed42d9d640 1 --2- 172.21.15.16:0/1876001611 >> v2:172.21.15.112:3300/0 conn(0x7fed440d1c90 0x7fed440c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:39:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.318+0000 7fed4259c640 1 --2- 172.21.15.16:0/1876001611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed441608a0 0x7fed440d0d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fed34009870 tx=0x7fed3402b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c214a44a2a0cb2f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.318+0000 7fed41d9b640 1 -- 172.21.15.16:0/1876001611 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed340317c0 con 0x7fed441608a0 2023-12-08T08:39:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.318+0000 7fed41d9b640 1 -- 172.21.15.16:0/1876001611 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed34031920 con 0x7fed441608a0 2023-12-08T08:39:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.318+0000 7fed41d9b640 1 -- 172.21.15.16:0/1876001611 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed34031cb0 con 0x7fed441608a0 2023-12-08T08:39:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.319+0000 7fed48941640 1 -- 172.21.15.16:0/1876001611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed441608a0 msgr2=0x7fed440d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:50.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.319+0000 7fed48941640 1 --2- 172.21.15.16:0/1876001611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed441608a0 0x7fed440d0d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fed34009870 tx=0x7fed3402b860 comp rx=0 tx=0).stop 2023-12-08T08:39:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.319+0000 7fed48941640 1 -- 172.21.15.16:0/1876001611 shutdown_connections 2023-12-08T08:39:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.319+0000 7fed48941640 1 --2- 172.21.15.16:0/1876001611 >> v2:172.21.15.112:3300/0 conn(0x7fed440d1c90 0x7fed440c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.319+0000 7fed48941640 1 --2- 172.21.15.16:0/1876001611 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed441608a0 0x7fed440d0d80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.319+0000 7fed48941640 1 -- 172.21.15.16:0/1876001611 >> 172.21.15.16:0/1876001611 conn(0x7fed440d2fa0 msgr2=0x7fed440d33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.319+0000 7fed48941640 1 -- 172.21.15.16:0/1876001611 shutdown_connections 2023-12-08T08:39:50.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.319+0000 7fed48941640 1 -- 172.21.15.16:0/1876001611 wait complete. 2023-12-08T08:39:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.320+0000 7fed48941640 1 Processor -- start 2023-12-08T08:39:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.320+0000 7fed48941640 1 -- start start 2023-12-08T08:39:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed48941640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed440d1c90 0x7fed440c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed48941640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed441608a0 0x7fed44152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed48941640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed44160c80 0x7fed44153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed48941640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed440b5c00 con 0x7fed441608a0 2023-12-08T08:39:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed48941640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed440b5ea0 con 0x7fed440d1c90 2023-12-08T08:39:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed48941640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed440b6180 con 0x7fed44160c80 2023-12-08T08:39:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed441608a0 0x7fed44152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed3bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed441608a0 0x7fed44152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41554/0 (socket says 172.21.15.16:41554) 2023-12-08T08:39:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed3bfff640 1 -- 172.21.15.16:0/2543532164 learned_addr learned my addr 172.21.15.16:0/2543532164 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed42d9d640 1 --2- 172.21.15.16:0/2543532164 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed44160c80 0x7fed44153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed4259c640 1 --2- 172.21.15.16:0/2543532164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed440d1c90 0x7fed440c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed3bfff640 1 -- 172.21.15.16:0/2543532164 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed44160c80 msgr2=0x7fed44153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed3bfff640 1 --2- 172.21.15.16:0/2543532164 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed44160c80 0x7fed44153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed3bfff640 1 -- 172.21.15.16:0/2543532164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed440d1c90 msgr2=0x7fed440c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed3bfff640 1 --2- 172.21.15.16:0/2543532164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed440d1c90 0x7fed440c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed3bfff640 1 -- 172.21.15.16:0/2543532164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c009a70 con 0x7fed441608a0 2023-12-08T08:39:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.321+0000 7fed3bfff640 1 --2- 172.21.15.16:0/2543532164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed441608a0 0x7fed44152b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00c440 tx=0x7fed2c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.322+0000 7fed3affd640 1 -- 172.21.15.16:0/2543532164 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c0074e0 con 0x7fed441608a0 2023-12-08T08:39:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.322+0000 7fed3affd640 1 -- 172.21.15.16:0/2543532164 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2c007640 con 0x7fed441608a0 2023-12-08T08:39:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.322+0000 7fed48941640 1 -- 172.21.15.16:0/2543532164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed340094d0 con 0x7fed441608a0 2023-12-08T08:39:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.322+0000 7fed3affd640 1 -- 172.21.15.16:0/2543532164 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c004a20 con 0x7fed441608a0 2023-12-08T08:39:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.322+0000 7fed48941640 1 -- 172.21.15.16:0/2543532164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed440b6740 con 0x7fed441608a0 2023-12-08T08:39:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.323+0000 7fed48941640 1 -- 172.21.15.16:0/2543532164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed4415e8d0 con 0x7fed441608a0 2023-12-08T08:39:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.323+0000 7fed3affd640 1 -- 172.21.15.16:0/2543532164 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed2c004b80 con 0x7fed441608a0 2023-12-08T08:39:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.324+0000 7fed3affd640 1 --2- 172.21.15.16:0/2543532164 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fed2406ae40 0x7fed2406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.324+0000 7fed3affd640 1 -- 172.21.15.16:0/2543532164 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fed2c0891f0 con 0x7fed441608a0 2023-12-08T08:39:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.324+0000 7fed4259c640 1 --2- 172.21.15.16:0/2543532164 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fed2406ae40 0x7fed2406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.324+0000 7fed4259c640 1 --2- 172.21.15.16:0/2543532164 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fed2406ae40 0x7fed2406d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fed34036000 tx=0x7fed3403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.328+0000 7fed3affd640 1 -- 172.21.15.16:0/2543532164 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed2c054050 con 0x7fed441608a0 2023-12-08T08:39:50.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.574+0000 7fed48941640 1 -- 172.21.15.16:0/2543532164 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fed44067430 con 0x7fed441608a0 2023-12-08T08:39:50.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.576+0000 7fed3affd640 1 -- 172.21.15.16:0/2543532164 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7fed2c057670 con 0x7fed441608a0 2023-12-08T08:39:50.579 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-12-08T08:39:50.579 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC51XJlZAHPCxAAZLN3jMh+Z9XRet/zdd6MzQ== 2023-12-08T08:39:50.579 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow" 2023-12-08T08:39:50.580 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow r" 2023-12-08T08:39:50.580 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-12-08T08:39:50.580 INFO:teuthology.orchestra.run.smithi016.stderr:updated caps for client.0 2023-12-08T08:39:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.578+0000 7fed48941640 1 -- 172.21.15.16:0/2543532164 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fed2406ae40 msgr2=0x7fed2406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.578+0000 7fed48941640 1 --2- 172.21.15.16:0/2543532164 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fed2406ae40 0x7fed2406d300 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fed34036000 tx=0x7fed3403d040 comp rx=0 tx=0).stop 2023-12-08T08:39:50.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.578+0000 7fed48941640 1 -- 172.21.15.16:0/2543532164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed441608a0 msgr2=0x7fed44152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:50.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.578+0000 7fed48941640 1 --2- 172.21.15.16:0/2543532164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed441608a0 0x7fed44152b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00c440 tx=0x7fed2c00c900 comp rx=0 tx=0).stop 2023-12-08T08:39:50.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.578+0000 7fed48941640 1 -- 172.21.15.16:0/2543532164 shutdown_connections 2023-12-08T08:39:50.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.578+0000 7fed48941640 1 --2- 172.21.15.16:0/2543532164 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fed2406ae40 0x7fed2406d300 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.578+0000 7fed48941640 1 --2- 172.21.15.16:0/2543532164 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fed44160c80 0x7fed44153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.578+0000 7fed48941640 1 --2- 172.21.15.16:0/2543532164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fed441608a0 0x7fed44152b50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.578+0000 7fed48941640 1 --2- 172.21.15.16:0/2543532164 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fed440d1c90 0x7fed440c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.578+0000 7fed48941640 1 -- 172.21.15.16:0/2543532164 >> 172.21.15.16:0/2543532164 conn(0x7fed440d2fa0 msgr2=0x7fed440ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:50.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.579+0000 7fed48941640 1 -- 172.21.15.16:0/2543532164 shutdown_connections 2023-12-08T08:39:50.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.579+0000 7fed48941640 1 -- 172.21.15.16:0/2543532164 wait complete. 2023-12-08T08:39:50.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-08T08:39:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.698+0000 7f180811d640 1 Processor -- start 2023-12-08T08:39:50.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.698+0000 7f180811d640 1 -- start start 2023-12-08T08:39:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.699+0000 7f180811d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000b0cf0 0x7f18000d1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.699+0000 7f180811d640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f18000b1d70 0x7f18000cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.699+0000 7f180811d640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18000d20b0 con 0x7f18001608a0 2023-12-08T08:39:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.699+0000 7f180811d640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18000d0620 con 0x7f18000b1d70 2023-12-08T08:39:50.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.699+0000 7f180811d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18000d1b80 con 0x7f18000b0cf0 2023-12-08T08:39:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.700+0000 7f18056b8640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000b0cf0 0x7f18000d1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.700+0000 7f18066ba640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f18000b1d70 0x7f18000cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.700+0000 7f18066ba640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f18000b1d70 0x7f18000cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.112:3300/0 says I am v2:172.21.15.16:58574/0 (socket says 172.21.15.16:58574) 2023-12-08T08:39:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.700+0000 7f18066ba640 1 -- 172.21.15.16:0/1649081552 learned_addr learned my addr 172.21.15.16:0/1649081552 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.700+0000 7f1805eb9640 1 --1- 172.21.15.16:0/1649081552 >> v1:172.21.15.112:6789/0 conn(0x7f18001608a0 0x7f18000d0f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.112:6789/0 says I am v1:172.21.15.16:34344/0 (socket says 172.21.15.16:34344) 2023-12-08T08:39:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.700+0000 7f18056b8640 1 -- 172.21.15.16:0/1649081552 >> v2:172.21.15.112:3300/0 conn(0x7f18000b1d70 msgr2=0x7f18000cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:50.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.700+0000 7f18056b8640 1 --2- 172.21.15.16:0/1649081552 >> v2:172.21.15.112:3300/0 conn(0x7f18000b1d70 0x7f18000cdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.700+0000 7f18056b8640 1 -- 172.21.15.16:0/1649081552 >> v1:172.21.15.112:6789/0 conn(0x7f18001608a0 legacy=0x7f18000d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.700+0000 7f18056b8640 1 -- 172.21.15.16:0/1649081552 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18000d1dd0 con 0x7f18000b0cf0 2023-12-08T08:39:50.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.700+0000 7f18066ba640 1 --2- 172.21.15.16:0/1649081552 >> v2:172.21.15.112:3300/0 conn(0x7f18000b1d70 0x7f18000cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-08T08:39:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.701+0000 7f18056b8640 1 --2- 172.21.15.16:0/1649081552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000b0cf0 0x7f18000d1640 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f17f40097a0 tx=0x7f17f402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d6dee3c62b1cd3 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.701+0000 7f1804eb7640 1 -- 172.21.15.16:0/1649081552 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f402e070 con 0x7f18000b0cf0 2023-12-08T08:39:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.701+0000 7f1804eb7640 1 -- 172.21.15.16:0/1649081552 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f40315a0 con 0x7f18000b0cf0 2023-12-08T08:39:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.701+0000 7f1804eb7640 1 -- 172.21.15.16:0/1649081552 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4031930 con 0x7f18000b0cf0 2023-12-08T08:39:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.702+0000 7f180811d640 1 -- 172.21.15.16:0/1649081552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000b0cf0 msgr2=0x7f18000d1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.702+0000 7f180811d640 1 --2- 172.21.15.16:0/1649081552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000b0cf0 0x7f18000d1640 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f17f40097a0 tx=0x7f17f402b840 comp rx=0 tx=0).stop 2023-12-08T08:39:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.702+0000 7f180811d640 1 -- 172.21.15.16:0/1649081552 shutdown_connections 2023-12-08T08:39:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.702+0000 7f180811d640 1 --2- 172.21.15.16:0/1649081552 >> v2:172.21.15.112:3300/0 conn(0x7f18000b1d70 0x7f18000cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.702+0000 7f180811d640 1 --2- 172.21.15.16:0/1649081552 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000b0cf0 0x7f18000d1640 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.702+0000 7f180811d640 1 -- 172.21.15.16:0/1649081552 >> 172.21.15.16:0/1649081552 conn(0x7f18000d3030 msgr2=0x7f18000d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:50.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.702+0000 7f180811d640 1 -- 172.21.15.16:0/1649081552 shutdown_connections 2023-12-08T08:39:50.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.702+0000 7f180811d640 1 -- 172.21.15.16:0/1649081552 wait complete. 2023-12-08T08:39:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.703+0000 7f180811d640 1 Processor -- start 2023-12-08T08:39:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.703+0000 7f180811d640 1 -- start start 2023-12-08T08:39:50.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f180811d640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000b0cf0 0x7f18000a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f180811d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18000b1d70 0x7f18000cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f180811d640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18001608a0 0x7f18000cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f180811d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18000c79e0 con 0x7f18000b1d70 2023-12-08T08:39:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f180811d640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18000c6b10 con 0x7f18001608a0 2023-12-08T08:39:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f180811d640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18000c6d60 con 0x7f18000b0cf0 2023-12-08T08:39:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f18056b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18000b1d70 0x7f18000cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f18056b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18000b1d70 0x7f18000cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41570/0 (socket says 172.21.15.16:41570) 2023-12-08T08:39:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f18056b8640 1 -- 172.21.15.16:0/4061350091 learned_addr learned my addr 172.21.15.16:0/4061350091 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f18066ba640 1 --2- 172.21.15.16:0/4061350091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18001608a0 0x7f18000cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f1805eb9640 1 --2- 172.21.15.16:0/4061350091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000b0cf0 0x7f18000a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f18056b8640 1 -- 172.21.15.16:0/4061350091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000b0cf0 msgr2=0x7f18000a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f18056b8640 1 --2- 172.21.15.16:0/4061350091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000b0cf0 0x7f18000a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f18056b8640 1 -- 172.21.15.16:0/4061350091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18001608a0 msgr2=0x7f18000cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:50.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f18056b8640 1 --2- 172.21.15.16:0/4061350091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18001608a0 0x7f18000cca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f18056b8640 1 -- 172.21.15.16:0/4061350091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f0009b30 con 0x7f18000b1d70 2023-12-08T08:39:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.704+0000 7f18056b8640 1 --2- 172.21.15.16:0/4061350091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18000b1d70 0x7f18000cc510 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f17f4034040 tx=0x7f17f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.705+0000 7f17fe7fc640 1 -- 172.21.15.16:0/4061350091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4032840 con 0x7f18000b1d70 2023-12-08T08:39:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.705+0000 7f17fe7fc640 1 -- 172.21.15.16:0/4061350091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f40329c0 con 0x7f18000b1d70 2023-12-08T08:39:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.705+0000 7f180811d640 1 -- 172.21.15.16:0/4061350091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f4009400 con 0x7f18000b1d70 2023-12-08T08:39:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.705+0000 7f17fe7fc640 1 -- 172.21.15.16:0/4061350091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4032d70 con 0x7f18000b1d70 2023-12-08T08:39:50.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.705+0000 7f180811d640 1 -- 172.21.15.16:0/4061350091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18000c7320 con 0x7f18000b1d70 2023-12-08T08:39:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.705+0000 7f180811d640 1 -- 172.21.15.16:0/4061350091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1800067430 con 0x7f18000b1d70 2023-12-08T08:39:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.706+0000 7f17fe7fc640 1 -- 172.21.15.16:0/4061350091 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f17f404e430 con 0x7f18000b1d70 2023-12-08T08:39:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.706+0000 7f17fe7fc640 1 --2- 172.21.15.16:0/4061350091 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f17ec06ae40 0x7f17ec06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.706+0000 7f17fe7fc640 1 -- 172.21.15.16:0/4061350091 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f17f40ab710 con 0x7f18000b1d70 2023-12-08T08:39:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.707+0000 7f1805eb9640 1 --2- 172.21.15.16:0/4061350091 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f17ec06ae40 0x7f17ec06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.707+0000 7f1805eb9640 1 --2- 172.21.15.16:0/4061350091 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f17ec06ae40 0x7f17ec06d300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f17f8000f60 tx=0x7f17f8000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.711+0000 7f17fe7fc640 1 -- 172.21.15.16:0/4061350091 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f17f4076600 con 0x7f18000b1d70 2023-12-08T08:39:50.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.922+0000 7f180811d640 1 -- 172.21.15.16:0/4061350091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18000b8eb0 con 0x7f18000b1d70 2023-12-08T08:39:50.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.923+0000 7f17fe7fc640 1 -- 172.21.15.16:0/4061350091 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4916 (secure 0 0 0) 0x7f17f4079c20 con 0x7f18000b1d70 2023-12-08T08:39:50.926 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:50.926 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.112:6837/1455612594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6835","nonce":1455612594},{"type":"v1","addr":"172.21.15.112:6837","nonce":1455612594}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4904,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6834/4205380866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":4205380866},{"type":"v1","addr":"172.21.15.16:6834","nonce":4205380866}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4907,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/516241878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6833","nonce":516241878},{"type":"v1","addr":"172.21.15.16:6835","nonce":516241878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-08T08:39:41.743317+0000","modified":"2023-12-08T08:39:43.739526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5170},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5170":{"gid":5170,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.112:6836/1132982118","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:6834","nonce":1132982118},{"type":"v1","addr":"172.21.15.112:6836","nonce":1132982118}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-08T08:39:50.926 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 15 2023-12-08T08:39:50.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.925+0000 7f180811d640 1 -- 172.21.15.16:0/4061350091 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f17ec06ae40 msgr2=0x7f17ec06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:50.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.925+0000 7f180811d640 1 --2- 172.21.15.16:0/4061350091 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f17ec06ae40 0x7f17ec06d300 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f17f8000f60 tx=0x7f17f8000ed0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.925+0000 7f180811d640 1 -- 172.21.15.16:0/4061350091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18000b1d70 msgr2=0x7f18000cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:50.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.925+0000 7f180811d640 1 --2- 172.21.15.16:0/4061350091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18000b1d70 0x7f18000cc510 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f17f4034040 tx=0x7f17f40046c0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.925+0000 7f180811d640 1 -- 172.21.15.16:0/4061350091 shutdown_connections 2023-12-08T08:39:50.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.925+0000 7f180811d640 1 --2- 172.21.15.16:0/4061350091 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f17ec06ae40 0x7f17ec06d300 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.925+0000 7f180811d640 1 --2- 172.21.15.16:0/4061350091 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f18001608a0 0x7f18000cca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.925+0000 7f180811d640 1 --2- 172.21.15.16:0/4061350091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18000b1d70 0x7f18000cc510 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.925+0000 7f180811d640 1 --2- 172.21.15.16:0/4061350091 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f18000b0cf0 0x7f18000a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:50.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.925+0000 7f180811d640 1 -- 172.21.15.16:0/4061350091 >> 172.21.15.16:0/4061350091 conn(0x7f18000d3030 msgr2=0x7f18000ce540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:50.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.925+0000 7f180811d640 1 -- 172.21.15.16:0/4061350091 shutdown_connections 2023-12-08T08:39:50.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:50.925+0000 7f180811d640 1 -- 172.21.15.16:0/4061350091 wait complete. 2023-12-08T08:39:51.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-08T08:39:41.743317+0000', 'modified': '2023-12-08T08:39:43.739526+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5170}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5170': {'gid': 5170, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.112:6836/1132982118', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.112:6834', 'nonce': 1132982118}, {'type': 'v1', 'addr': '172.21.15.112:6836', 'nonce': 1132982118}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-08T08:39:51.342 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-08T08:39:51.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-08T08:39:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.449+0000 7f15e287c640 1 Processor -- start 2023-12-08T08:39:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.449+0000 7f15e287c640 1 -- start start 2023-12-08T08:39:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15e287c640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f15dc15d580 0x7f15dc0d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15e287c640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15dc0d3ad0 0x7f15dc15f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15e287c640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15dc15ee90 con 0x7f15dc15f6c0 2023-12-08T08:39:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15e287c640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15dc0d0fe0 con 0x7f15dc0d3ad0 2023-12-08T08:39:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15e287c640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15dc0d1280 con 0x7f15dc15d580 2023-12-08T08:39:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15db7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15dc0d3ad0 0x7f15dc15f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15e0e19640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f15dc15f6c0 0x7f15dc0b23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35410/0 (socket says 172.21.15.16:35410) 2023-12-08T08:39:51.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15db7fe640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15dc0d3ad0 0x7f15dc15f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41586/0 (socket says 172.21.15.16:41586) 2023-12-08T08:39:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15e0e19640 1 -- 172.21.15.16:0/4106610111 learned_addr learned my addr 172.21.15.16:0/4106610111 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15dbfff640 1 --2- 172.21.15.16:0/4106610111 >> v2:172.21.15.112:3300/0 conn(0x7f15dc15d580 0x7f15dc0d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15db7fe640 1 -- 172.21.15.16:0/4106610111 >> v2:172.21.15.112:3300/0 conn(0x7f15dc15d580 msgr2=0x7f15dc0d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15db7fe640 1 --2- 172.21.15.16:0/4106610111 >> v2:172.21.15.112:3300/0 conn(0x7f15dc15d580 0x7f15dc0d1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15db7fe640 1 -- 172.21.15.16:0/4106610111 >> v1:172.21.15.16:6789/0 conn(0x7f15dc15f6c0 legacy=0x7f15dc0b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15db7fe640 1 -- 172.21.15.16:0/4106610111 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15dc0d1560 con 0x7f15dc0d3ad0 2023-12-08T08:39:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.450+0000 7f15dbfff640 1 --2- 172.21.15.16:0/4106610111 >> v2:172.21.15.112:3300/0 conn(0x7f15dc15d580 0x7f15dc0d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.451+0000 7f15db7fe640 1 --2- 172.21.15.16:0/4106610111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc0d3ad0 0x7f15dc15f180 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0097a0 tx=0x7f15cc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ae9daadf28cecb server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.451+0000 7f15daffd640 1 -- 172.21.15.16:0/4106610111 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc02e070 con 0x7f15dc0d3ad0 2023-12-08T08:39:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.451+0000 7f15daffd640 1 -- 172.21.15.16:0/4106610111 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc0315a0 con 0x7f15dc0d3ad0 2023-12-08T08:39:51.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.451+0000 7f15daffd640 1 -- 172.21.15.16:0/4106610111 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc031930 con 0x7f15dc0d3ad0 2023-12-08T08:39:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.452+0000 7f15e287c640 1 -- 172.21.15.16:0/4106610111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc0d3ad0 msgr2=0x7f15dc15f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.452+0000 7f15e287c640 1 --2- 172.21.15.16:0/4106610111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc0d3ad0 0x7f15dc15f180 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0097a0 tx=0x7f15cc02b840 comp rx=0 tx=0).stop 2023-12-08T08:39:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.452+0000 7f15e287c640 1 -- 172.21.15.16:0/4106610111 shutdown_connections 2023-12-08T08:39:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.452+0000 7f15e287c640 1 --2- 172.21.15.16:0/4106610111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc0d3ad0 0x7f15dc15f180 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.452+0000 7f15e287c640 1 --2- 172.21.15.16:0/4106610111 >> v2:172.21.15.112:3300/0 conn(0x7f15dc15d580 0x7f15dc0d1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.452+0000 7f15e287c640 1 -- 172.21.15.16:0/4106610111 >> 172.21.15.16:0/4106610111 conn(0x7f15dc0d2fb0 msgr2=0x7f15dc0d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.452+0000 7f15e287c640 1 -- 172.21.15.16:0/4106610111 shutdown_connections 2023-12-08T08:39:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.452+0000 7f15e287c640 1 -- 172.21.15.16:0/4106610111 wait complete. 2023-12-08T08:39:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.453+0000 7f15e287c640 1 Processor -- start 2023-12-08T08:39:51.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.453+0000 7f15e287c640 1 -- start start 2023-12-08T08:39:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15e287c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc0d3ad0 0x7f15dc0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15e287c640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15dc15d580 0x7f15dc0c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15e287c640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15dc15f6c0 0x7f15dc0b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15e287c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc0cae60 con 0x7f15dc0d3ad0 2023-12-08T08:39:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15e287c640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc0ca200 con 0x7f15dc15f6c0 2023-12-08T08:39:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15e287c640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15dc0ca4c0 con 0x7f15dc15d580 2023-12-08T08:39:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc0d3ad0 0x7f15dc0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc0d3ad0 0x7f15dc0c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41594/0 (socket says 172.21.15.16:41594) 2023-12-08T08:39:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15dbfff640 1 -- 172.21.15.16:0/2161858023 learned_addr learned my addr 172.21.15.16:0/2161858023 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15db7fe640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15dc15d580 0x7f15dc0c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15e0e19640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15dc15f6c0 0x7f15dc0b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15dbfff640 1 -- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15dc15d580 msgr2=0x7f15dc0c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15dbfff640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15dc15d580 0x7f15dc0c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15dbfff640 1 -- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15dc15f6c0 msgr2=0x7f15dc0b65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15dbfff640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15dc15f6c0 0x7f15dc0b65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15dbfff640 1 -- 172.21.15.16:0/2161858023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc009400 con 0x7f15dc0d3ad0 2023-12-08T08:39:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15db7fe640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15dc15d580 0x7f15dc0c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15e0e19640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15dc15f6c0 0x7f15dc0b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15dbfff640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc0d3ad0 0x7f15dc0c9770 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f15c400c7f0 tx=0x7f15c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.454+0000 7f15d8ff9640 1 -- 172.21.15.16:0/2161858023 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4007a30 con 0x7f15dc0d3ad0 2023-12-08T08:39:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.455+0000 7f15d8ff9640 1 -- 172.21.15.16:0/2161858023 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c4007b90 con 0x7f15dc0d3ad0 2023-12-08T08:39:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.455+0000 7f15e287c640 1 -- 172.21.15.16:0/2161858023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15dc0ca7a0 con 0x7f15dc0d3ad0 2023-12-08T08:39:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.455+0000 7f15d8ff9640 1 -- 172.21.15.16:0/2161858023 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c400ce40 con 0x7f15dc0d3ad0 2023-12-08T08:39:51.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.455+0000 7f15e287c640 1 -- 172.21.15.16:0/2161858023 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15dc0c9cb0 con 0x7f15dc0d3ad0 2023-12-08T08:39:51.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.456+0000 7f15d8ff9640 1 -- 172.21.15.16:0/2161858023 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f15c40113f0 con 0x7f15dc0d3ad0 2023-12-08T08:39:51.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.457+0000 7f15d8ff9640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f15bc06ae40 0x7f15bc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.457+0000 7f15d8ff9640 1 -- 172.21.15.16:0/2161858023 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f15c40898f0 con 0x7f15dc0d3ad0 2023-12-08T08:39:51.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.457+0000 7f15db7fe640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f15bc06ae40 0x7f15bc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.457+0000 7f15db7fe640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f15bc06ae40 0x7f15bc06d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f15cc036000 tx=0x7f15cc03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.457+0000 7f15e5412740 1 Processor -- start 2023-12-08T08:39:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.457+0000 7f15e5412740 1 -- start start 2023-12-08T08:39:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15e5412740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21b7c50 0x564ea218abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15e5412740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b8030 0x564ea21ba9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15e5412740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea218b130 0x564ea21baf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15e5412740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea214b980 con 0x564ea21b7c50 2023-12-08T08:39:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15e5412740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea20fdb90 con 0x564ea21b8030 2023-12-08T08:39:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15e5412740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ea20fb280 con 0x564ea218b130 2023-12-08T08:39:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21b7c50 0x564ea218abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21b7c50 0x564ea218abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41598/0 (socket says 172.21.15.16:41598) 2023-12-08T08:39:51.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15dbfff640 1 -- 172.21.15.16:0/1137745558 learned_addr learned my addr 172.21.15.16:0/1137745558 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:51.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15db7fe640 1 --2- 172.21.15.16:0/1137745558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b8030 0x564ea21ba9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15e0e19640 1 --2- 172.21.15.16:0/1137745558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea218b130 0x564ea21baf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15dbfff640 1 -- 172.21.15.16:0/1137745558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea218b130 msgr2=0x564ea21baf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:51.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15dbfff640 1 --2- 172.21.15.16:0/1137745558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea218b130 0x564ea21baf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15dbfff640 1 -- 172.21.15.16:0/1137745558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b8030 msgr2=0x564ea21ba9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:51.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15dbfff640 1 --2- 172.21.15.16:0/1137745558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b8030 0x564ea21ba9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.458+0000 7f15dbfff640 1 -- 172.21.15.16:0/1137745558 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ea20bc500 con 0x564ea21b7c50 2023-12-08T08:39:51.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.459+0000 7f15dbfff640 1 --2- 172.21.15.16:0/1137745558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21b7c50 0x564ea218abf0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f15c40602f0 tx=0x7f15c4058520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea7965bfe65ef0f server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:51.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.459+0000 7f15d1ffb640 1 -- 172.21.15.16:0/1137745558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4060820 con 0x564ea21b7c50 2023-12-08T08:39:51.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.459+0000 7f15d1ffb640 1 -- 172.21.15.16:0/1137745558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c40568c0 con 0x564ea21b7c50 2023-12-08T08:39:51.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.459+0000 7f15d1ffb640 1 -- 172.21.15.16:0/1137745558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4060390 con 0x564ea21b7c50 2023-12-08T08:39:51.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.459+0000 7f15e5412740 1 -- 172.21.15.16:0/1137745558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21b7c50 msgr2=0x564ea218abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.459+0000 7f15e5412740 1 --2- 172.21.15.16:0/1137745558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21b7c50 0x564ea218abf0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f15c40602f0 tx=0x7f15c4058520 comp rx=0 tx=0).stop 2023-12-08T08:39:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.459+0000 7f15e5412740 1 -- 172.21.15.16:0/1137745558 shutdown_connections 2023-12-08T08:39:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.459+0000 7f15e5412740 1 --2- 172.21.15.16:0/1137745558 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea218b130 0x564ea21baf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.459+0000 7f15e5412740 1 --2- 172.21.15.16:0/1137745558 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b8030 0x564ea21ba9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.459+0000 7f15e5412740 1 --2- 172.21.15.16:0/1137745558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21b7c50 0x564ea218abf0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.459+0000 7f15e5412740 1 -- 172.21.15.16:0/1137745558 >> 172.21.15.16:0/1137745558 conn(0x564ea218ce60 msgr2=0x564ea21d59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.459+0000 7f15e5412740 1 -- 172.21.15.16:0/1137745558 shutdown_connections 2023-12-08T08:39:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.459+0000 7f15e5412740 1 -- 172.21.15.16:0/1137745558 wait complete. 2023-12-08T08:39:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.460+0000 7f15e5412740 1 -- start start 2023-12-08T08:39:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.460+0000 7f15e5412740 1 Processor -- start 2023-12-08T08:39:51.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15e5412740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b7c50 0x564ea21cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15e5412740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b8030 0x564ea21a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15e5412740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21d0010 0x564ea21a8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15e5412740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea20f6b70 con 0x564ea21d0010 2023-12-08T08:39:51.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15e5412740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea20b0e00 con 0x564ea21b7c50 2023-12-08T08:39:51.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15e5412740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ea20b0aa0 con 0x564ea21b8030 2023-12-08T08:39:51.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15e5412740 10 client.0 _ll_register_callbacks cb 0x564ea1e08ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-08T08:39:51.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15e0e19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21d0010 0x564ea21a8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15e0e19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21d0010 0x564ea21a8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41604/0 (socket says 172.21.15.16:41604) 2023-12-08T08:39:51.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15e0e19640 1 -- 172.21.15.16:0/896246335 learned_addr learned my addr 172.21.15.16:0/896246335 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:51.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15db7fe640 1 --2- 172.21.15.16:0/896246335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b8030 0x564ea21a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15dbfff640 1 --2- 172.21.15.16:0/896246335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b7c50 0x564ea21cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15e0e19640 1 -- 172.21.15.16:0/896246335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b8030 msgr2=0x564ea21a96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:51.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15e0e19640 1 --2- 172.21.15.16:0/896246335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b8030 0x564ea21a96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15e0e19640 1 -- 172.21.15.16:0/896246335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b7c50 msgr2=0x564ea21cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:51.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.461+0000 7f15e0e19640 1 --2- 172.21.15.16:0/896246335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b7c50 0x564ea21cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15e0e19640 1 -- 172.21.15.16:0/896246335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15dc0c9cb0 con 0x564ea21d0010 2023-12-08T08:39:51.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15dbfff640 1 --2- 172.21.15.16:0/896246335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b7c50 0x564ea21cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:51.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15e0e19640 1 --2- 172.21.15.16:0/896246335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21d0010 0x564ea21a8e70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f15d400f880 tx=0x7f15d400fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140246d3f2e74b75 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:51.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15baffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-12-08T08:39:51.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15baffd640 1 -- 172.21.15.16:0/896246335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d400d8e0 con 0x564ea21d0010 2023-12-08T08:39:51.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15baffd640 1 -- 172.21.15.16:0/896246335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f15d400da40 con 0x564ea21d0010 2023-12-08T08:39:51.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15e5412740 1 -- 172.21.15.16:0/896246335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ea20efb50 con 0x564ea21d0010 2023-12-08T08:39:51.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15baffd640 1 -- 172.21.15.16:0/896246335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15d4001b50 con 0x564ea21d0010 2023-12-08T08:39:51.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15e5412740 10 client.5760 fetch_fsmap learned FSMap version 15 2023-12-08T08:39:51.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15e5412740 1 -- 172.21.15.16:0/896246335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x564ea20f51f0 con 0x564ea21d0010 2023-12-08T08:39:51.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15baffd640 1 -- 172.21.15.16:0/896246335 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564ea20efb50 con 0x564ea21d0010 2023-12-08T08:39:51.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15baffd640 1 -- 172.21.15.16:0/896246335 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f15d4012070 con 0x564ea21d0010 2023-12-08T08:39:51.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15e5412740 10 client.5760 fetch_fsmap finished waiting for FSMap version 15 2023-12-08T08:39:51.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15e5412740 10 client.5760 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-08T08:39:51.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15e5412740 20 client.5760 populate_metadata read hostname 'smithi016' 2023-12-08T08:39:51.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15e5412740 1 --2- 172.21.15.16:0/896246335 >> [v2:172.21.15.112:6834/1132982118,v1:172.21.15.112:6836/1132982118] conn(0x564ea21cef70 0x564ea21f3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15e5412740 4 client.5760 mds_command: new command op to 5170 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-08T08:39:51.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.462+0000 7f15e5412740 1 -- 172.21.15.16:0/896246335 --> [v2:172.21.15.112:6834/1132982118,v1:172.21.15.112:6836/1132982118] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ea20ef870 con 0x564ea21cef70 2023-12-08T08:39:51.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.463+0000 7f15dbfff640 1 --2- 172.21.15.16:0/896246335 >> [v2:172.21.15.112:6834/1132982118,v1:172.21.15.112:6836/1132982118] conn(0x564ea21cef70 0x564ea21f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.463+0000 7f15dbfff640 1 --2- 172.21.15.16:0/896246335 >> [v2:172.21.15.112:6834/1132982118,v1:172.21.15.112:6836/1132982118] conn(0x564ea21cef70 0x564ea21f3160 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8094dfa7597d6ab5 server_cookie=de4c08d72a84494e in_seq=0 out_seq=0 2023-12-08T08:39:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.463+0000 7f15baffd640 10 client.5760 ms_handle_connect on v2:172.21.15.112:6834/1132982118 2023-12-08T08:39:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15baffd640 1 -- 172.21.15.16:0/896246335 <== mds.0 v2:172.21.15.112:6834/1132982118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x564ea20ef870 con 0x564ea21cef70 2023-12-08T08:39:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15baffd640 10 client.5760 handle_command_reply: tid=0 2023-12-08T08:39:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15e5412740 1 client.5760 shutdown 2023-12-08T08:39:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15e5412740 1 -- 172.21.15.16:0/896246335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21d0010 msgr2=0x564ea21a8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15e5412740 1 --2- 172.21.15.16:0/896246335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21d0010 0x564ea21a8e70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f15d400f880 tx=0x7f15d400fd40 comp rx=0 tx=0).stop 2023-12-08T08:39:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15e5412740 1 -- 172.21.15.16:0/896246335 shutdown_connections 2023-12-08T08:39:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15e5412740 1 --2- 172.21.15.16:0/896246335 >> [v2:172.21.15.112:6834/1132982118,v1:172.21.15.112:6836/1132982118] conn(0x564ea21cef70 0x564ea21f3160 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15e5412740 1 --2- 172.21.15.16:0/896246335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21d0010 0x564ea21a8e70 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15e5412740 1 --2- 172.21.15.16:0/896246335 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b8030 0x564ea21a96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15baffd640 0 client.5760 ms_handle_reset on v2:172.21.15.112:6834/1132982118 2023-12-08T08:39:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15e5412740 1 --2- 172.21.15.16:0/896246335 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b7c50 0x564ea21cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15e5412740 1 -- 172.21.15.16:0/896246335 >> 172.21.15.16:0/896246335 conn(0x564ea218b130 msgr2=0x564ea21ba9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:51.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15e5412740 1 -- 172.21.15.16:0/896246335 shutdown_connections 2023-12-08T08:39:51.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15e5412740 1 -- 172.21.15.16:0/896246335 wait complete. 2023-12-08T08:39:51.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.464+0000 7f15e5412740 20 client.5760 trim_cache size 0 max 16384 2023-12-08T08:39:51.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.480+0000 7f15e5412740 1 Processor -- start 2023-12-08T08:39:51.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.480+0000 7f15e5412740 1 -- start start 2023-12-08T08:39:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.480+0000 7f15e5412740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b7c50 0x564ea21d6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.480+0000 7f15e5412740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b8030 0x564ea21a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.480+0000 7f15e5412740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21d0010 0x564ea21d8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.480+0000 7f15e5412740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea2076270 con 0x564ea21d0010 2023-12-08T08:39:51.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.480+0000 7f15e5412740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea21d9d10 con 0x564ea21b8030 2023-12-08T08:39:51.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.480+0000 7f15e5412740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ea207f1f0 con 0x564ea21b7c50 2023-12-08T08:39:51.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.480+0000 7f15e0e19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21d0010 0x564ea21d8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.481+0000 7f15e0e19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21d0010 0x564ea21d8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41612/0 (socket says 172.21.15.16:41612) 2023-12-08T08:39:51.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.481+0000 7f15e0e19640 1 -- 172.21.15.16:0/2555850766 learned_addr learned my addr 172.21.15.16:0/2555850766 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:51.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.481+0000 7f15dbfff640 1 --2- 172.21.15.16:0/2555850766 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b7c50 0x564ea21d6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.481+0000 7f15db7fe640 1 --2- 172.21.15.16:0/2555850766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b8030 0x564ea21a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.481+0000 7f15e0e19640 1 -- 172.21.15.16:0/2555850766 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b7c50 msgr2=0x564ea21d6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:51.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.481+0000 7f15e0e19640 1 --2- 172.21.15.16:0/2555850766 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b7c50 0x564ea21d6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.481+0000 7f15e0e19640 1 -- 172.21.15.16:0/2555850766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b8030 msgr2=0x564ea21a7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.481+0000 7f15e0e19640 1 --2- 172.21.15.16:0/2555850766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b8030 0x564ea21a7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.481+0000 7f15e0e19640 1 -- 172.21.15.16:0/2555850766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ea2092d00 con 0x564ea21d0010 2023-12-08T08:39:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.481+0000 7f15dbfff640 1 --2- 172.21.15.16:0/2555850766 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b7c50 0x564ea21d6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:39:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.481+0000 7f15e0e19640 1 --2- 172.21.15.16:0/2555850766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21d0010 0x564ea21d8ef0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f15d4001920 tx=0x7f15d4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d74dccd10c6c4d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.481+0000 7f15d27fc640 1 -- 172.21.15.16:0/2555850766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d400d520 con 0x564ea21d0010 2023-12-08T08:39:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.481+0000 7f15d27fc640 1 -- 172.21.15.16:0/2555850766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d400d8e0 con 0x564ea21d0010 2023-12-08T08:39:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.481+0000 7f15d27fc640 1 -- 172.21.15.16:0/2555850766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d4007390 con 0x564ea21d0010 2023-12-08T08:39:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.482+0000 7f15e5412740 1 -- 172.21.15.16:0/2555850766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21d0010 msgr2=0x564ea21d8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.482+0000 7f15e5412740 1 --2- 172.21.15.16:0/2555850766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21d0010 0x564ea21d8ef0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f15d4001920 tx=0x7f15d4013000 comp rx=0 tx=0).stop 2023-12-08T08:39:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.482+0000 7f15e5412740 1 -- 172.21.15.16:0/2555850766 shutdown_connections 2023-12-08T08:39:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.482+0000 7f15e5412740 1 --2- 172.21.15.16:0/2555850766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21d0010 0x564ea21d8ef0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.482+0000 7f15e5412740 1 --2- 172.21.15.16:0/2555850766 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21b8030 0x564ea21a7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.482+0000 7f15e5412740 1 --2- 172.21.15.16:0/2555850766 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b7c50 0x564ea21d6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.482+0000 7f15e5412740 1 -- 172.21.15.16:0/2555850766 >> 172.21.15.16:0/2555850766 conn(0x564ea218b130 msgr2=0x564ea21a9490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.482+0000 7f15e5412740 1 -- 172.21.15.16:0/2555850766 shutdown_connections 2023-12-08T08:39:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.482+0000 7f15e5412740 1 -- 172.21.15.16:0/2555850766 wait complete. 2023-12-08T08:39:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.482+0000 7f15e5412740 1 -- start start 2023-12-08T08:39:51.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.483+0000 7f15e5412740 1 Processor -- start 2023-12-08T08:39:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.483+0000 7f15e5412740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21b7c50 0x564ea21ef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.483+0000 7f15e5412740 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b8030 0x564ea21f7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.483+0000 7f15e5412740 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21d0010 0x564ea21f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.483+0000 7f15e5412740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea21bb7c0 con 0x564ea21b7c50 2023-12-08T08:39:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.483+0000 7f15e5412740 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ea21bbc20 con 0x564ea21d0010 2023-12-08T08:39:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.483+0000 7f15e5412740 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x564ea218b720 con 0x564ea21b8030 2023-12-08T08:39:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.483+0000 7f15e5412740 10 client.0 _ll_register_callbacks cb 0x7f15c40582f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-08T08:39:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.483+0000 7f15dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21b7c50 0x564ea21ef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.483+0000 7f15dbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21b7c50 0x564ea21ef7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41628/0 (socket says 172.21.15.16:41628) 2023-12-08T08:39:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.483+0000 7f15dbfff640 1 -- 172.21.15.16:0/4181962340 learned_addr learned my addr 172.21.15.16:0/4181962340 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.483+0000 7f15e0e19640 1 --2- 172.21.15.16:0/4181962340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21d0010 0x564ea21f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.483+0000 7f15db7fe640 1 --2- 172.21.15.16:0/4181962340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b8030 0x564ea21f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15dbfff640 1 -- 172.21.15.16:0/4181962340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b8030 msgr2=0x564ea21f7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15dbfff640 1 --2- 172.21.15.16:0/4181962340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b8030 0x564ea21f7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15dbfff640 1 -- 172.21.15.16:0/4181962340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21d0010 msgr2=0x564ea21f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:39:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15dbfff640 1 --2- 172.21.15.16:0/4181962340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21d0010 0x564ea21f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15dbfff640 1 -- 172.21.15.16:0/4181962340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ea20f51f0 con 0x564ea21b7c50 2023-12-08T08:39:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15db7fe640 1 --2- 172.21.15.16:0/4181962340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b8030 0x564ea21f7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:39:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15dbfff640 1 --2- 172.21.15.16:0/4181962340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21b7c50 0x564ea21ef7a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f15c405c830 tx=0x7f15c401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32f1b440695065b server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15baffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-12-08T08:39:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15baffd640 1 -- 172.21.15.16:0/4181962340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4060390 con 0x564ea21b7c50 2023-12-08T08:39:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15baffd640 1 -- 172.21.15.16:0/4181962340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f15c4074070 con 0x564ea21b7c50 2023-12-08T08:39:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15e5412740 1 -- 172.21.15.16:0/4181962340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15c40568c0 con 0x564ea21b7c50 2023-12-08T08:39:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15baffd640 1 -- 172.21.15.16:0/4181962340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15c40604f0 con 0x564ea21b7c50 2023-12-08T08:39:51.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15e5412740 10 client.5766 fetch_fsmap learned FSMap version 15 2023-12-08T08:39:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15e5412740 1 -- 172.21.15.16:0/4181962340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x564ea20b84f0 con 0x564ea21b7c50 2023-12-08T08:39:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15baffd640 1 -- 172.21.15.16:0/4181962340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c406a080 con 0x564ea21b7c50 2023-12-08T08:39:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15baffd640 1 -- 172.21.15.16:0/4181962340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 15) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f15c4064030 con 0x564ea21b7c50 2023-12-08T08:39:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15e5412740 10 client.5766 fetch_fsmap finished waiting for FSMap version 15 2023-12-08T08:39:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15e5412740 10 client.5766 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-08T08:39:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15e5412740 20 client.5766 populate_metadata read hostname 'smithi016' 2023-12-08T08:39:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15e5412740 1 --2- 172.21.15.16:0/4181962340 >> [v2:172.21.15.112:6834/1132982118,v1:172.21.15.112:6836/1132982118] conn(0x564ea21a9820 0x564ea2243430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15e5412740 4 client.5766 mds_command: new command op to 5170 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-08T08:39:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.484+0000 7f15e5412740 1 -- 172.21.15.16:0/4181962340 --> [v2:172.21.15.112:6834/1132982118,v1:172.21.15.112:6836/1132982118] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564ea1df76c0 con 0x564ea21a9820 2023-12-08T08:39:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.485+0000 7f15db7fe640 1 --2- 172.21.15.16:0/4181962340 >> [v2:172.21.15.112:6834/1132982118,v1:172.21.15.112:6836/1132982118] conn(0x564ea21a9820 0x564ea2243430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.485+0000 7f15db7fe640 1 --2- 172.21.15.16:0/4181962340 >> [v2:172.21.15.112:6834/1132982118,v1:172.21.15.112:6836/1132982118] conn(0x564ea21a9820 0x564ea2243430 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1773528ccad22ac server_cookie=71ff2c6f8781a3e8 in_seq=0 out_seq=0 2023-12-08T08:39:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.485+0000 7f15baffd640 10 client.5766 ms_handle_connect on v2:172.21.15.112:6834/1132982118 2023-12-08T08:39:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.485+0000 7f15baffd640 1 -- 172.21.15.16:0/4181962340 <== mds.0 v2:172.21.15.112:6834/1132982118 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x564ea1df76c0 con 0x564ea21a9820 2023-12-08T08:39:51.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.485+0000 7f15baffd640 10 client.5766 handle_command_reply: tid=0 2023-12-08T08:39:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.485+0000 7f15e5412740 1 client.5766 shutdown 2023-12-08T08:39:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e5412740 1 -- 172.21.15.16:0/4181962340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21b7c50 msgr2=0x564ea21ef7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e5412740 1 --2- 172.21.15.16:0/4181962340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21b7c50 0x564ea21ef7a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f15c405c830 tx=0x7f15c401b000 comp rx=0 tx=0).stop 2023-12-08T08:39:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e5412740 1 -- 172.21.15.16:0/4181962340 shutdown_connections 2023-12-08T08:39:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e5412740 1 --2- 172.21.15.16:0/4181962340 >> [v2:172.21.15.112:6834/1132982118,v1:172.21.15.112:6836/1132982118] conn(0x564ea21a9820 0x564ea2243430 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e5412740 1 --2- 172.21.15.16:0/4181962340 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x564ea21d0010 0x564ea21f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e5412740 1 --2- 172.21.15.16:0/4181962340 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x564ea21b8030 0x564ea21f7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e5412740 1 --2- 172.21.15.16:0/4181962340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ea21b7c50 0x564ea21ef7a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e5412740 1 -- 172.21.15.16:0/4181962340 >> 172.21.15.16:0/4181962340 conn(0x564ea218b130 msgr2=0x564ea21d8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15baffd640 0 client.5766 ms_handle_reset on v2:172.21.15.112:6834/1132982118 2023-12-08T08:39:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e5412740 1 -- 172.21.15.16:0/4181962340 shutdown_connections 2023-12-08T08:39:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e5412740 1 -- 172.21.15.16:0/4181962340 wait complete. 2023-12-08T08:39:51.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e5412740 20 client.5766 trim_cache size 0 max 16384 2023-12-08T08:39:51.496 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-12-08T08:39:51.496 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ead7ea87-fe4a-461b-916c-c6458a26d625", 2023-12-08T08:39:51.496 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-12-08T08:39:51.496 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5170, 2023-12-08T08:39:51.496 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-12-08T08:39:51.496 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-12-08T08:39:51.496 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-12-08T08:39:51.496 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 9.1131670650000007, 2023-12-08T08:39:51.496 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 15, 2023-12-08T08:39:51.496 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 39, 2023-12-08T08:39:51.496 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 39, 2023-12-08T08:39:51.496 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 16.987409924000001 2023-12-08T08:39:51.497 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-12-08T08:39:51.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e287c640 1 -- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f15bc06ae40 msgr2=0x7f15bc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e287c640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f15bc06ae40 0x7f15bc06d300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f15cc036000 tx=0x7f15cc03d040 comp rx=0 tx=0).stop 2023-12-08T08:39:51.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e287c640 1 -- 172.21.15.16:0/2161858023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc0d3ad0 msgr2=0x7f15dc0c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e287c640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc0d3ad0 0x7f15dc0c9770 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f15c400c7f0 tx=0x7f15c400cbf0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e287c640 1 -- 172.21.15.16:0/2161858023 shutdown_connections 2023-12-08T08:39:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e287c640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f15bc06ae40 0x7f15bc06d300 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e287c640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f15dc15f6c0 0x7f15dc0b65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e287c640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f15dc15d580 0x7f15dc0c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e287c640 1 --2- 172.21.15.16:0/2161858023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15dc0d3ad0 0x7f15dc0c9770 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.486+0000 7f15e287c640 1 -- 172.21.15.16:0/2161858023 >> 172.21.15.16:0/2161858023 conn(0x7f15dc0d2fb0 msgr2=0x7f15dc0cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.487+0000 7f15e287c640 1 -- 172.21.15.16:0/2161858023 shutdown_connections 2023-12-08T08:39:51.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.487+0000 7f15e287c640 1 -- 172.21.15.16:0/2161858023 wait complete. 2023-12-08T08:39:51.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-08T08:39:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.604+0000 7f64131de640 1 Processor -- start 2023-12-08T08:39:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.604+0000 7f64131de640 1 -- start start 2023-12-08T08:39:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.604+0000 7f64131de640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f640c0b3bf0 0x7f640c0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.604+0000 7f64131de640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f640c0cdc20 con 0x7f640c0d1d00 2023-12-08T08:39:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.604+0000 7f64131de640 1 -- --> v1:172.21.15.112:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f640c0d11d0 con 0x7f640c0b3340 2023-12-08T08:39:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.604+0000 7f64131de640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f640c0d3a60 con 0x7f640c0b3bf0 2023-12-08T08:39:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.605+0000 7f640bfff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f640c0d1d00 0x7f640c0ce3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35422/0 (socket says 172.21.15.16:35422) 2023-12-08T08:39:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.605+0000 7f640bfff640 1 -- 172.21.15.16:0/1660895440 learned_addr learned my addr 172.21.15.16:0/1660895440 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:39:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.605+0000 7f641177b640 1 --2- 172.21.15.16:0/1660895440 >> v2:172.21.15.112:3300/0 conn(0x7f640c0b3bf0 0x7f640c0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.605+0000 7f640b7fe640 1 -- 172.21.15.16:0/1660895440 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479001559 0 0) 0x7f640c0cdc20 con 0x7f640c0d1d00 2023-12-08T08:39:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.605+0000 7f640b7fe640 1 -- 172.21.15.16:0/1660895440 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63f4003520 con 0x7f640c0d1d00 2023-12-08T08:39:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.605+0000 7f640b7fe640 1 -- 172.21.15.16:0/1660895440 <== mon.1 v1:172.21.15.112:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545488784 0 0) 0x7f640c0d11d0 con 0x7f640c0b3340 2023-12-08T08:39:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.605+0000 7f640b7fe640 1 -- 172.21.15.16:0/1660895440 --> v1:172.21.15.112:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f640c0cdc20 con 0x7f640c0b3340 2023-12-08T08:39:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.605+0000 7f640b7fe640 1 -- 172.21.15.16:0/1660895440 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4187559700 0 0) 0x7f63f4003520 con 0x7f640c0d1d00 2023-12-08T08:39:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.605+0000 7f641177b640 1 -- 172.21.15.16:0/1660895440 >> v1:172.21.15.112:6789/0 conn(0x7f640c0b3340 legacy=0x7f640c0d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.605+0000 7f641177b640 1 -- 172.21.15.16:0/1660895440 >> v1:172.21.15.16:6789/0 conn(0x7f640c0d1d00 legacy=0x7f640c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.605+0000 7f641177b640 1 -- 172.21.15.16:0/1660895440 --> v2:172.21.15.112:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c0d3d40 con 0x7f640c0b3bf0 2023-12-08T08:39:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.606+0000 7f641177b640 1 --2- 172.21.15.16:0/1660895440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c0b3bf0 0x7f640c0d00c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0097d0 tx=0x7f63fc02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d84289d48fc7033e server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.606+0000 7f640b7fe640 1 -- 172.21.15.16:0/1660895440 <== mon.1 v2:172.21.15.112:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6400001670 con 0x7f640c0b3bf0 2023-12-08T08:39:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.606+0000 7f640b7fe640 1 -- 172.21.15.16:0/1660895440 <== mon.1 v2:172.21.15.112:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63fc02e050 con 0x7f640c0b3bf0 2023-12-08T08:39:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.606+0000 7f640b7fe640 1 -- 172.21.15.16:0/1660895440 <== mon.1 v2:172.21.15.112:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63fc030710 con 0x7f640c0b3bf0 2023-12-08T08:39:51.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.607+0000 7f64131de640 1 -- 172.21.15.16:0/1660895440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c0b3bf0 msgr2=0x7f640c0d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.607+0000 7f64131de640 1 --2- 172.21.15.16:0/1660895440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c0b3bf0 0x7f640c0d00c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0097d0 tx=0x7f63fc02b790 comp rx=0 tx=0).stop 2023-12-08T08:39:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.607+0000 7f64131de640 1 -- 172.21.15.16:0/1660895440 shutdown_connections 2023-12-08T08:39:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.607+0000 7f64131de640 1 --2- 172.21.15.16:0/1660895440 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c0b3bf0 0x7f640c0d00c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.607+0000 7f64131de640 1 -- 172.21.15.16:0/1660895440 >> 172.21.15.16:0/1660895440 conn(0x7f640c09c2f0 msgr2=0x7f640c0d3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.607+0000 7f64131de640 1 -- 172.21.15.16:0/1660895440 shutdown_connections 2023-12-08T08:39:51.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.607+0000 7f64131de640 1 -- 172.21.15.16:0/1660895440 wait complete. 2023-12-08T08:39:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.608+0000 7f64131de640 1 Processor -- start 2023-12-08T08:39:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.608+0000 7f64131de640 1 -- start start 2023-12-08T08:39:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f64131de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c0b3340 0x7f640c152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f64131de640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c0b3bf0 0x7f640c152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f64131de640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f640c0d1d00 0x7f640c09d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f64131de640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0ca0e0 con 0x7f640c0b3340 2023-12-08T08:39:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f64131de640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640c0ca910 con 0x7f640c0b3bf0 2023-12-08T08:39:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f64131de640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640c09e070 con 0x7f640c0d1d00 2023-12-08T08:39:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f6410f7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c0b3340 0x7f640c152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f6410f7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c0b3340 0x7f640c152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41640/0 (socket says 172.21.15.16:41640) 2023-12-08T08:39:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f6410f7a640 1 -- 172.21.15.16:0/682590906 learned_addr learned my addr 172.21.15.16:0/682590906 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:39:51.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f641177b640 1 --2- 172.21.15.16:0/682590906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f640c0d1d00 0x7f640c09d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f640bfff640 1 --2- 172.21.15.16:0/682590906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c0b3bf0 0x7f640c152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f6410f7a640 1 -- 172.21.15.16:0/682590906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f640c0d1d00 msgr2=0x7f640c09d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f6410f7a640 1 --2- 172.21.15.16:0/682590906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f640c0d1d00 0x7f640c09d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f6410f7a640 1 -- 172.21.15.16:0/682590906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c0b3bf0 msgr2=0x7f640c152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f6410f7a640 1 --2- 172.21.15.16:0/682590906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c0b3bf0 0x7f640c152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f6410f7a640 1 -- 172.21.15.16:0/682590906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc009400 con 0x7f640c0b3340 2023-12-08T08:39:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f6410f7a640 1 --2- 172.21.15.16:0/682590906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c0b3340 0x7f640c152f80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f640400c800 tx=0x7f640400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.609+0000 7f64097fa640 1 -- 172.21.15.16:0/682590906 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404003b70 con 0x7f640c0b3340 2023-12-08T08:39:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.610+0000 7f64097fa640 1 -- 172.21.15.16:0/682590906 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6404003cd0 con 0x7f640c0b3340 2023-12-08T08:39:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.610+0000 7f64131de640 1 -- 172.21.15.16:0/682590906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f640c0ca2c0 con 0x7f640c0b3340 2023-12-08T08:39:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.610+0000 7f64097fa640 1 -- 172.21.15.16:0/682590906 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640400aa90 con 0x7f640c0b3340 2023-12-08T08:39:51.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.610+0000 7f64131de640 1 -- 172.21.15.16:0/682590906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f640c152970 con 0x7f640c0b3340 2023-12-08T08:39:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.611+0000 7f64131de640 1 -- 172.21.15.16:0/682590906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640c0b2f40 con 0x7f640c0b3340 2023-12-08T08:39:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.611+0000 7f64097fa640 1 -- 172.21.15.16:0/682590906 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f640400abf0 con 0x7f640c0b3340 2023-12-08T08:39:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.611+0000 7f64097fa640 1 --2- 172.21.15.16:0/682590906 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f63f806ae70 0x7f63f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:39:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.612+0000 7f64097fa640 1 -- 172.21.15.16:0/682590906 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f64040896c0 con 0x7f640c0b3340 2023-12-08T08:39:51.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.612+0000 7f640bfff640 1 --2- 172.21.15.16:0/682590906 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f63f806ae70 0x7f63f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:39:51.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.612+0000 7f640bfff640 1 --2- 172.21.15.16:0/682590906 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f63f806ae70 0x7f63f806d330 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f640c0b2f10 tx=0x7f6400009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:39:51.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.616+0000 7f64097fa640 1 -- 172.21.15.16:0/682590906 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6404054630 con 0x7f640c0b3340 2023-12-08T08:39:51.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.829+0000 7f64131de640 1 -- 172.21.15.16:0/682590906 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f640c067430 con 0x7f640c0b3340 2023-12-08T08:39:51.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.829+0000 7f64097fa640 1 -- 172.21.15.16:0/682590906 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6404057c50 con 0x7f640c0b3340 2023-12-08T08:39:51.832 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-08T08:39:51.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.831+0000 7f64131de640 1 -- 172.21.15.16:0/682590906 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f63f806ae70 msgr2=0x7f63f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.831+0000 7f64131de640 1 --2- 172.21.15.16:0/682590906 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f63f806ae70 0x7f63f806d330 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f640c0b2f10 tx=0x7f6400009040 comp rx=0 tx=0).stop 2023-12-08T08:39:51.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.831+0000 7f64131de640 1 -- 172.21.15.16:0/682590906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c0b3340 msgr2=0x7f640c152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:39:51.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.831+0000 7f64131de640 1 --2- 172.21.15.16:0/682590906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c0b3340 0x7f640c152f80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f640400c800 tx=0x7f640400ccc0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.831+0000 7f64131de640 1 -- 172.21.15.16:0/682590906 shutdown_connections 2023-12-08T08:39:51.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.831+0000 7f64131de640 1 --2- 172.21.15.16:0/682590906 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f63f806ae70 0x7f63f806d330 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.831+0000 7f64131de640 1 --2- 172.21.15.16:0/682590906 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f640c0d1d00 0x7f640c09d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.831+0000 7f64131de640 1 --2- 172.21.15.16:0/682590906 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f640c0b3bf0 0x7f640c152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.831+0000 7f64131de640 1 --2- 172.21.15.16:0/682590906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c0b3340 0x7f640c152f80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:39:51.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.831+0000 7f64131de640 1 -- 172.21.15.16:0/682590906 >> 172.21.15.16:0/682590906 conn(0x7f640c09c2f0 msgr2=0x7f640c0d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:39:51.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.831+0000 7f64131de640 1 -- 172.21.15.16:0/682590906 shutdown_connections 2023-12-08T08:39:51.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:51.831+0000 7f64131de640 1 -- 172.21.15.16:0/682590906 wait complete. 2023-12-08T08:39:51.845 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-08T08:39:51.845 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-08T08:39:51.845 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2023-12-08T08:39:51.845 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-08T08:39:51.845 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-08T08:39:51.845 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-08T08:39:51.845 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-08T08:39:51.845 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-08T08:39:51.846 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-08T08:39:51.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:51.846 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2023-12-08T08:39:51.861 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-08T08:39:51.862 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-08T08:39:51.862 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2023-12-08T08:39:51.862 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:39:51.862 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2023-12-08T08:39:51.862 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:39:51.862 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:39:51.862 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:39:51.862 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2023-12-08T08:39:51.862 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:39:51.862 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:39:51.862 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42520sec preferred_lft 42520sec 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link noprefixroute 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 1e:54:fb:9f:6e:b6 brd ff:ff:ff:ff:ff:ff 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:39:51.863 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::20b7:78ff:fec0:4f49/64 scope link 2023-12-08T08:39:51.864 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:39:51.864 INFO:teuthology.orchestra.run.smithi016.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-08T08:39:51.864 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 26:12:f3:8e:79:c7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:39:51.864 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::2412:f3ff:fe8e:79c7/64 scope link 2023-12-08T08:39:51.864 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:39:51.864 INFO:teuthology.orchestra.run.smithi016.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-08T08:39:51.864 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 1e:54:fb:9f:6e:b6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-08T08:39:51.864 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::1c54:fbff:fe9f:6eb6/64 scope link 2023-12-08T08:39:51.864 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2023-12-08T08:39:51.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:51.864 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-08T08:39:51.931 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-12-08T08:39:51.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:51.932 DEBUG:teuthology.orchestra.run.smithi016:> route 2023-12-08T08:39:51.989 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2023-12-08T08:39:51.989 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-08T08:39:51.989 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-08T08:39:51.989 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-08T08:39:51.989 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-08T08:39:51.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:51.990 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec bash -c ' 2023-12-08T08:39:51.990 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2023-12-08T08:39:51.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-08T08:39:51.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-08T08:39:51.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-08T08:39:51.990 DEBUG:teuthology.orchestra.run.smithi016:> ') 2023-12-08T08:39:52.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:52.095 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2023-12-08T08:39:52.110 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-08T08:39:52.110 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-08T08:39:52.111 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-08T08:39:52.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:52.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2023-12-08T08:39:52.182 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-08T08:39:52.182 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-08T08:39:52.182 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-08T08:39:52.237 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-08T08:39:52.237 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-08T08:39:52.237 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-08T08:39:52.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-12-08T08:39:52.364 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-12-08T08:39:52.420 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-12-08T08:39:52.420 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-12-08T08:39:52.420 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-12-08T08:39:52.420 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-12-08T08:39:52.420 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-12-08T08:39:52.420 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-12-08T08:39:52.420 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-12-08T08:39:52.420 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-12-08T08:39:52.420 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-12-08T08:39:52.420 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2023-12-08T08:39:52.420 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-12-08T08:39:52.420 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-12-08T08:39:52.421 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-12-08T08:39:52.422 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-12-08T08:39:52.422 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-12-08T08:39:52.422 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:39:52.422 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:39:52.422 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-08T08:39:52.422 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-08T08:39:52.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:52.422 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-12-08T08:39:52.475 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-08T08:39:52.475 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-08T08:39:52.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo modprobe fuse 2023-12-08T08:39:52.548 DEBUG:teuthology.orchestra.run.smithi016:> cat /proc/self/mounts | awk '{print $2}' 2023-12-08T08:39:52.603 INFO:teuthology.orchestra.run.smithi016.stdout:/proc 2023-12-08T08:39:52.604 INFO:teuthology.orchestra.run.smithi016.stdout:/sys 2023-12-08T08:39:52.604 INFO:teuthology.orchestra.run.smithi016.stdout:/dev 2023-12-08T08:39:52.604 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/security 2023-12-08T08:39:52.604 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/shm 2023-12-08T08:39:52.604 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/pts 2023-12-08T08:39:52.604 INFO:teuthology.orchestra.run.smithi016.stdout:/run 2023-12-08T08:39:52.604 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/cgroup 2023-12-08T08:39:52.604 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/pstore 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/bpf 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/ 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/selinux 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/proc/sys/fs/binfmt_misc 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/mqueue 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/dev/hugepages 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/debug 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/tracing 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/fs/fuse/connections 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/sys/kernel/config 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-sysctl.service 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/nfs/rpc_pipefs 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/run/user/1000 2023-12-08T08:39:52.605 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph 2023-12-08T08:39:52.606 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-0 2023-12-08T08:39:52.606 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-1 2023-12-08T08:39:52.606 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-2 2023-12-08T08:39:52.606 INFO:teuthology.orchestra.run.smithi016.stdout:/var/lib/ceph/osd/ceph-3 2023-12-08T08:39:52.606 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns 2023-12-08T08:39:52.606 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:39:52.606 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-08T08:39:52.606 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-08T08:39:52.606 INFO:teuthology.orchestra.run.smithi016.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-08T08:39:52.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:52.606 DEBUG:teuthology.orchestra.run.smithi016:> ls /sys/fs/fuse/connections 2023-12-08T08:39:52.610 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:2023-12-08T08:39:52.607+0000 7f2a08037480 -1 init, newargv = 0x55fa75296140 newargc=15 2023-12-08T08:39:52.610 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[105452]: starting ceph client 2023-12-08T08:39:52.615 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[105452]: starting fuse 2023-12-08T08:39:52.625 INFO:teuthology.orchestra.run.smithi016.stdout:41 2023-12-08T08:39:52.626 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-08T08:39:52.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo stdin-killer -- python3 -c ' 2023-12-08T08:39:52.626 DEBUG:teuthology.orchestra.run.smithi016:> import glob 2023-12-08T08:39:52.626 DEBUG:teuthology.orchestra.run.smithi016:> import re 2023-12-08T08:39:52.626 DEBUG:teuthology.orchestra.run.smithi016:> import os 2023-12-08T08:39:52.626 DEBUG:teuthology.orchestra.run.smithi016:> import subprocess 2023-12-08T08:39:52.626 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:39:52.626 DEBUG:teuthology.orchestra.run.smithi016:> def _find_admin_socket(client_name): 2023-12-08T08:39:52.626 DEBUG:teuthology.orchestra.run.smithi016:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-08T08:39:52.626 DEBUG:teuthology.orchestra.run.smithi016:> files = glob.glob(asok_path) 2023-12-08T08:39:52.627 DEBUG:teuthology.orchestra.run.smithi016:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-08T08:39:52.627 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:39:52.627 DEBUG:teuthology.orchestra.run.smithi016:> # Given a non-glob path, it better be there 2023-12-08T08:39:52.627 DEBUG:teuthology.orchestra.run.smithi016:> if "*" not in asok_path: 2023-12-08T08:39:52.627 DEBUG:teuthology.orchestra.run.smithi016:> assert(len(files) == 1) 2023-12-08T08:39:52.627 DEBUG:teuthology.orchestra.run.smithi016:> return files[0] 2023-12-08T08:39:52.627 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:39:52.627 DEBUG:teuthology.orchestra.run.smithi016:> for f in files: 2023-12-08T08:39:52.627 DEBUG:teuthology.orchestra.run.smithi016:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-08T08:39:52.627 DEBUG:teuthology.orchestra.run.smithi016:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-08T08:39:52.627 DEBUG:teuthology.orchestra.run.smithi016:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-08T08:39:52.628 DEBUG:teuthology.orchestra.run.smithi016:> contents = proc_f.read() 2023-12-08T08:39:52.628 DEBUG:teuthology.orchestra.run.smithi016:> if mountpoint in contents: 2023-12-08T08:39:52.628 DEBUG:teuthology.orchestra.run.smithi016:> return f 2023-12-08T08:39:52.628 DEBUG:teuthology.orchestra.run.smithi016:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-08T08:39:52.628 DEBUG:teuthology.orchestra.run.smithi016:> 2023-12-08T08:39:52.628 DEBUG:teuthology.orchestra.run.smithi016:> print(_find_admin_socket("client.0")) 2023-12-08T08:39:52.628 DEBUG:teuthology.orchestra.run.smithi016:> ' 2023-12-08T08:39:52.750 INFO:teuthology.orchestra.run.smithi016.stdout:/var/run/ceph/ceph-client.0.105452.asok 2023-12-08T08:39:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:39:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-08T08:39:52.760 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.105452.asok 2023-12-08T08:39:52.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:52.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.105452.asok status 2023-12-08T08:39:52.904 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-12-08T08:39:52.904 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": { 2023-12-08T08:39:52.904 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_sha1": "a2cd52c00149b5caff897ea7f79d0529b62620da", 2023-12-08T08:39:52.904 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.0.0-46-ga2cd52c0 (a2cd52c00149b5caff897ea7f79d0529b62620da) reef (dev)", 2023-12-08T08:39:52.904 INFO:teuthology.orchestra.run.smithi016.stdout: "entity_id": "0", 2023-12-08T08:39:52.904 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": "smithi016", 2023-12-08T08:39:52.904 INFO:teuthology.orchestra.run.smithi016.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-08T08:39:52.904 INFO:teuthology.orchestra.run.smithi016.stdout: "pid": "105452", 2023-12-08T08:39:52.904 INFO:teuthology.orchestra.run.smithi016.stdout: "root": "/" 2023-12-08T08:39:52.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:52.904 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_count": 0, 2023-12-08T08:39:52.904 INFO:teuthology.orchestra.run.smithi016.stdout: "dentry_pinned_count": 0, 2023-12-08T08:39:52.904 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 5781, 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: "inst": { 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: "name": { 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "client", 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: "num": 5781 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1471096426 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": { 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "192.168.0.1:0", 2023-12-08T08:39:52.905 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1471096426 2023-12-08T08:39:52.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-12-08T08:39:52.906 INFO:teuthology.orchestra.run.smithi016.stdout: "inst_str": "client.5781 192.168.0.1:0/1471096426", 2023-12-08T08:39:52.906 INFO:teuthology.orchestra.run.smithi016.stdout: "addr_str": "192.168.0.1:0/1471096426", 2023-12-08T08:39:52.906 INFO:teuthology.orchestra.run.smithi016.stdout: "inode_count": 1, 2023-12-08T08:39:52.906 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_epoch": 15, 2023-12-08T08:39:52.906 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch": 39, 2023-12-08T08:39:52.906 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_epoch_barrier": 0, 2023-12-08T08:39:52.906 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklisted": false, 2023-12-08T08:39:52.906 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs" 2023-12-08T08:39:52.906 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-12-08T08:39:52.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:52.912 DEBUG:teuthology.orchestra.run.smithi016:> stat --file-system '--printf=%T 2023-12-08T08:39:52.912 DEBUG:teuthology.orchestra.run.smithi016:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-08T08:39:52.928 INFO:teuthology.orchestra.run.smithi016.stdout:fuseblk 2023-12-08T08:39:52.928 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-08T08:39:52.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:52.928 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-08T08:39:53.000 DEBUG:teuthology.orchestra.run.smithi016:> mktemp -d --suffix=xfstests-dev 2023-12-08T08:39:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.MkHH5XcxMMxfstests-dev 2023-12-08T08:39:53.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:53.056 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.MkHH5XcxMMxfstests-dev) 2023-12-08T08:39:53.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:53.130 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.MkHH5XcxMMxfstests-dev) 2023-12-08T08:39:53.190 INFO:teuthology.orchestra.run.smithi016.stderr:Cloning into '/tmp/tmp.MkHH5XcxMMxfstests-dev'... 2023-12-08T08:39:53.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:53.869 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2023-12-08T08:39:53.931 DEBUG:teuthology.orchestra.run.smithi016:> mktemp -d --suffix=test 2023-12-08T08:39:53.987 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.hY9PAzp4zMtest 2023-12-08T08:39:53.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:39:53.987 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2023-12-08T08:39:54.045 DEBUG:teuthology.orchestra.run.smithi016:> mktemp -d --suffix=scratch 2023-12-08T08:39:54.100 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.sNDd1g2I6qscratch 2023-12-08T08:39:54.101 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2023-12-08T08:39:54.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs 2023-12-08T08:39:54.759 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:04:07 ago on Fri 08 Dec 2023 08:35:47 AM UTC. 2023-12-08T08:39:54.859 INFO:teuthology.orchestra.run.smithi016.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2023-12-08T08:39:54.860 INFO:teuthology.orchestra.run.smithi016.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2023-12-08T08:39:54.861 INFO:teuthology.orchestra.run.smithi016.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2023-12-08T08:39:54.863 INFO:teuthology.orchestra.run.smithi016.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2023-12-08T08:39:54.864 INFO:teuthology.orchestra.run.smithi016.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2023-12-08T08:39:54.866 INFO:teuthology.orchestra.run.smithi016.stdout:Package e2fsprogs-1.46.5-3.el9.x86_64 is already installed. 2023-12-08T08:39:54.868 INFO:teuthology.orchestra.run.smithi016.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2023-12-08T08:39:54.869 INFO:teuthology.orchestra.run.smithi016.stdout:Package gcc-11.4.1-2.1.el9.x86_64 is already installed. 2023-12-08T08:39:54.871 INFO:teuthology.orchestra.run.smithi016.stdout:Package git-2.39.3-1.el9.x86_64 is already installed. 2023-12-08T08:39:54.874 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2023-12-08T08:39:54.875 INFO:teuthology.orchestra.run.smithi016.stdout:Package libaio-devel-0.3.111-13.el9.x86_64 is already installed. 2023-12-08T08:39:54.877 INFO:teuthology.orchestra.run.smithi016.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2023-12-08T08:39:54.878 INFO:teuthology.orchestra.run.smithi016.stdout:Package libuuid-devel-2.37.4-15.el9.x86_64 is already installed. 2023-12-08T08:39:54.879 INFO:teuthology.orchestra.run.smithi016.stdout:Package lvm2-9:2.03.21-3.el9.x86_64 is already installed. 2023-12-08T08:39:54.880 INFO:teuthology.orchestra.run.smithi016.stdout:Package make-1:4.3-7.el9.x86_64 is already installed. 2023-12-08T08:39:54.881 INFO:teuthology.orchestra.run.smithi016.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2023-12-08T08:39:54.882 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-3.9.17-2.el9.x86_64 is already installed. 2023-12-08T08:39:54.883 INFO:teuthology.orchestra.run.smithi016.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2023-12-08T08:39:54.884 INFO:teuthology.orchestra.run.smithi016.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2023-12-08T08:39:54.888 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-12-08T08:39:54.949 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-12-08T08:39:54.952 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-12-08T08:39:54.952 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-12-08T08:39:54.952 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-12-08T08:39:54.953 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-12-08T08:39:54.953 INFO:teuthology.orchestra.run.smithi016.stdout: dump x86_64 1:0.4-0.51.b47.el9 epel 88 k 2023-12-08T08:39:54.953 INFO:teuthology.orchestra.run.smithi016.stdout: fio x86_64 3.35-1.el9 CentOS-AppStream 6.2 M 2023-12-08T08:39:54.953 INFO:teuthology.orchestra.run.smithi016.stdout: gdbm-devel x86_64 1:1.19-4.el9 crb 59 k 2023-12-08T08:39:54.953 INFO:teuthology.orchestra.run.smithi016.stdout: indent x86_64 2.2.13-4.el9 epel 179 k 2023-12-08T08:39:54.953 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-devel x86_64 5.14.0-390.el9 appstream 21 M 2023-12-08T08:39:54.953 INFO:teuthology.orchestra.run.smithi016.stdout: libcap-devel x86_64 2.48-9.el9 CentOS-AppStream 33 k 2023-12-08T08:39:54.953 INFO:teuthology.orchestra.run.smithi016.stdout: sqlite x86_64 3.34.1-6.el9 CentOS-AppStream 747 k 2023-12-08T08:39:54.953 INFO:teuthology.orchestra.run.smithi016.stdout: udftools x86_64 2.2-5.el9 CentOS-AppStream 144 k 2023-12-08T08:39:54.953 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2023-12-08T08:39:54.953 INFO:teuthology.orchestra.run.smithi016.stdout: cpp x86_64 11.4.1-2.3.el9 CentOS-AppStream 11 M 2023-12-08T08:39:54.953 INFO:teuthology.orchestra.run.smithi016.stdout: e2fsprogs x86_64 1.46.5-4.el9 CentOS-BaseOS 1.0 M 2023-12-08T08:39:54.953 INFO:teuthology.orchestra.run.smithi016.stdout: e2fsprogs-libs x86_64 1.46.5-4.el9 CentOS-BaseOS 221 k 2023-12-08T08:39:54.953 INFO:teuthology.orchestra.run.smithi016.stdout: gcc x86_64 11.4.1-2.3.el9 CentOS-AppStream 32 M 2023-12-08T08:39:54.954 INFO:teuthology.orchestra.run.smithi016.stdout: gcc-c++ x86_64 11.4.1-2.3.el9 CentOS-AppStream 13 M 2023-12-08T08:39:54.954 INFO:teuthology.orchestra.run.smithi016.stdout: gcc-plugin-annobin x86_64 11.4.1-2.3.el9 CentOS-AppStream 47 k 2023-12-08T08:39:54.954 INFO:teuthology.orchestra.run.smithi016.stdout: libcom_err x86_64 1.46.5-4.el9 CentOS-BaseOS 25 k 2023-12-08T08:39:54.954 INFO:teuthology.orchestra.run.smithi016.stdout: libgcc x86_64 11.4.1-2.3.el9 CentOS-BaseOS 92 k 2023-12-08T08:39:54.954 INFO:teuthology.orchestra.run.smithi016.stdout: libgomp x86_64 11.4.1-2.3.el9 CentOS-BaseOS 268 k 2023-12-08T08:39:54.954 INFO:teuthology.orchestra.run.smithi016.stdout: libss x86_64 1.46.5-4.el9 CentOS-BaseOS 31 k 2023-12-08T08:39:54.954 INFO:teuthology.orchestra.run.smithi016.stdout: libstdc++ x86_64 11.4.1-2.3.el9 CentOS-BaseOS 735 k 2023-12-08T08:39:54.954 INFO:teuthology.orchestra.run.smithi016.stdout: libstdc++-devel x86_64 11.4.1-2.3.el9 CentOS-AppStream 2.2 M 2023-12-08T08:39:54.954 INFO:teuthology.orchestra.run.smithi016.stdout: python-unversioned-command noarch 3.9.18-1.el9 CentOS-AppStream 8.8 k 2023-12-08T08:39:54.954 INFO:teuthology.orchestra.run.smithi016.stdout: python3 x86_64 3.9.18-1.el9 CentOS-BaseOS 26 k 2023-12-08T08:39:54.954 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.18-1.el9 CentOS-AppStream 204 k 2023-12-08T08:39:54.954 INFO:teuthology.orchestra.run.smithi016.stdout: python3-libs x86_64 3.9.18-1.el9 CentOS-BaseOS 7.3 M 2023-12-08T08:39:54.954 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-12-08T08:39:54.954 INFO:teuthology.orchestra.run.smithi016.stdout: gdbm x86_64 1:1.19-4.el9 crb 132 k 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout: libnbd x86_64 1.18.1-3.el9 appstream 166 k 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout: rmt x86_64 2:1.6-6.el9 CentOS-BaseOS 45 k 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout: fio-engine-http x86_64 3.35-1.el9 CentOS-AppStream 17 k 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout: fio-engine-libaio x86_64 3.35-1.el9 CentOS-AppStream 14 k 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout: fio-engine-nbd x86_64 3.35-1.el9 CentOS-AppStream 13 k 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout: fio-engine-rados x86_64 3.35-1.el9 CentOS-AppStream 15 k 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout: fio-engine-rbd x86_64 3.35-1.el9 CentOS-AppStream 15 k 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout: fio-engine-rdma x86_64 3.35-1.el9 CentOS-AppStream 20 k 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout:Install 17 Packages 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 16 Packages 2023-12-08T08:39:54.955 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:39:54.956 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 97 M 2023-12-08T08:39:54.956 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-12-08T08:39:55.607 INFO:teuthology.orchestra.run.smithi016.stdout:(1/33): fio-engine-http-3.35-1.el9.x86_64.rpm 86 kB/s | 17 kB 00:00 2023-12-08T08:39:55.632 INFO:teuthology.orchestra.run.smithi016.stdout:(2/33): fio-engine-libaio-3.35-1.el9.x86_64.rpm 65 kB/s | 14 kB 00:00 2023-12-08T08:39:55.758 INFO:teuthology.orchestra.run.smithi016.stdout:(3/33): fio-engine-nbd-3.35-1.el9.x86_64.rpm 90 kB/s | 13 kB 00:00 2023-12-08T08:39:55.783 INFO:teuthology.orchestra.run.smithi016.stdout:(4/33): fio-engine-rados-3.35-1.el9.x86_64.rpm 97 kB/s | 15 kB 00:00 2023-12-08T08:39:55.933 INFO:teuthology.orchestra.run.smithi016.stdout:(5/33): fio-3.35-1.el9.x86_64.rpm 12 MB/s | 6.2 MB 00:00 2023-12-08T08:39:55.950 INFO:teuthology.orchestra.run.smithi016.stdout:(6/33): fio-engine-rbd-3.35-1.el9.x86_64.rpm 79 kB/s | 15 kB 00:00 2023-12-08T08:39:55.967 INFO:teuthology.orchestra.run.smithi016.stdout:(7/33): fio-engine-rdma-3.35-1.el9.x86_64.rpm 108 kB/s | 20 kB 00:00 2023-12-08T08:39:56.000 INFO:teuthology.orchestra.run.smithi016.stdout:(8/33): libcap-devel-2.48-9.el9.x86_64.rpm 1.0 MB/s | 33 kB 00:00 2023-12-08T08:39:56.026 INFO:teuthology.orchestra.run.smithi016.stdout:(9/33): udftools-2.2-5.el9.x86_64.rpm 2.4 MB/s | 144 kB 00:00 2023-12-08T08:39:56.051 INFO:teuthology.orchestra.run.smithi016.stdout:(10/33): sqlite-3.34.1-6.el9.x86_64.rpm 8.7 MB/s | 747 kB 00:00 2023-12-08T08:39:56.068 INFO:teuthology.orchestra.run.smithi016.stdout:(11/33): rmt-1.6-6.el9.x86_64.rpm 674 kB/s | 45 kB 00:00 2023-12-08T08:39:56.177 INFO:teuthology.orchestra.run.smithi016.stdout:(12/33): libnbd-1.18.1-3.el9.x86_64.rpm 1.3 MB/s | 166 kB 00:00 2023-12-08T08:39:56.202 INFO:teuthology.orchestra.run.smithi016.stdout:(13/33): gdbm-1.19-4.el9.x86_64.rpm 989 kB/s | 132 kB 00:00 2023-12-08T08:39:56.227 INFO:teuthology.orchestra.run.smithi016.stdout:(14/33): gdbm-devel-1.19-4.el9.x86_64.rpm 1.1 MB/s | 59 kB 00:00 2023-12-08T08:39:56.745 INFO:teuthology.orchestra.run.smithi016.stdout:(15/33): kernel-devel-5.14.0-390.el9.x86_64.rpm 29 MB/s | 21 MB 00:00 2023-12-08T08:39:56.761 INFO:teuthology.orchestra.run.smithi016.stdout:(16/33): dump-0.4-0.51.b47.el9.x86_64.rpm 157 kB/s | 88 kB 00:00 2023-12-08T08:39:56.945 INFO:teuthology.orchestra.run.smithi016.stdout:(17/33): indent-2.2.13-4.el9.x86_64.rpm 250 kB/s | 179 kB 00:00 2023-12-08T08:39:57.470 INFO:teuthology.orchestra.run.smithi016.stdout:(18/33): cpp-11.4.1-2.3.el9.x86_64.rpm 15 MB/s | 11 MB 00:00 2023-12-08T08:39:58.087 INFO:teuthology.orchestra.run.smithi016.stdout:(19/33): gcc-11.4.1-2.3.el9.x86_64.rpm 24 MB/s | 32 MB 00:01 2023-12-08T08:39:58.221 INFO:teuthology.orchestra.run.smithi016.stdout:(20/33): gcc-c++-11.4.1-2.3.el9.x86_64.rpm 10 MB/s | 13 MB 00:01 2023-12-08T08:39:58.246 INFO:teuthology.orchestra.run.smithi016.stdout:(21/33): gcc-plugin-annobin-11.4.1-2.3.el9.x86_ 60 kB/s | 47 kB 00:00 2023-12-08T08:39:58.296 INFO:teuthology.orchestra.run.smithi016.stdout:(22/33): libstdc++-devel-11.4.1-2.3.el9.x86_64. 11 MB/s | 2.2 MB 00:00 2023-12-08T08:39:58.322 INFO:teuthology.orchestra.run.smithi016.stdout:(23/33): python-unversioned-command-3.9.18-1.el 121 kB/s | 8.8 kB 00:00 2023-12-08T08:39:58.338 INFO:teuthology.orchestra.run.smithi016.stdout:(24/33): python3-devel-3.9.18-1.el9.x86_64.rpm 2.2 MB/s | 204 kB 00:00 2023-12-08T08:39:58.372 INFO:teuthology.orchestra.run.smithi016.stdout:(25/33): e2fsprogs-1.46.5-4.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2023-12-08T08:39:58.397 INFO:teuthology.orchestra.run.smithi016.stdout:(26/33): e2fsprogs-libs-1.46.5-4.el9.x86_64.rpm 3.8 MB/s | 221 kB 00:00 2023-12-08T08:39:58.423 INFO:teuthology.orchestra.run.smithi016.stdout:(27/33): libcom_err-1.46.5-4.el9.x86_64.rpm 308 kB/s | 25 kB 00:00 2023-12-08T08:39:58.448 INFO:teuthology.orchestra.run.smithi016.stdout:(28/33): libgcc-11.4.1-2.3.el9.x86_64.rpm 1.2 MB/s | 92 kB 00:00 2023-12-08T08:39:58.473 INFO:teuthology.orchestra.run.smithi016.stdout:(29/33): libss-1.46.5-4.el9.x86_64.rpm 617 kB/s | 31 kB 00:00 2023-12-08T08:39:58.498 INFO:teuthology.orchestra.run.smithi016.stdout:(30/33): libgomp-11.4.1-2.3.el9.x86_64.rpm 3.5 MB/s | 268 kB 00:00 2023-12-08T08:39:58.532 INFO:teuthology.orchestra.run.smithi016.stdout:(31/33): libstdc++-11.4.1-2.3.el9.x86_64.rpm 8.6 MB/s | 735 kB 00:00 2023-12-08T08:39:58.557 INFO:teuthology.orchestra.run.smithi016.stdout:(32/33): python3-3.9.18-1.el9.x86_64.rpm 312 kB/s | 26 kB 00:00 2023-12-08T08:39:58.733 INFO:teuthology.orchestra.run.smithi016.stdout:(33/33): python3-libs-3.9.18-1.el9.x86_64.rpm 36 MB/s | 7.3 MB 00:00 2023-12-08T08:39:58.735 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-12-08T08:39:58.735 INFO:teuthology.orchestra.run.smithi016.stdout:Total 26 MB/s | 97 MB 00:03 2023-12-08T08:39:58.964 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-12-08T08:39:59.246 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-12-08T08:39:59.246 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-12-08T08:40:00.397 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-12-08T08:40:00.399 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-12-08T08:40:02.576 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-12-08T08:40:02.653 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libcom_err-1.46.5-4.el9.x86_64 1/49 2023-12-08T08:40:02.692 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libgcc-11.4.1-2.3.el9.x86_64 2/49 2023-12-08T08:40:02.796 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libgcc-11.4.1-2.3.el9.x86_64 2/49 2023-12-08T08:40:02.856 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libstdc++-11.4.1-2.3.el9.x86_64 3/49 2023-12-08T08:40:03.426 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : e2fsprogs-libs-1.46.5-4.el9.x86_64 4/49 2023-12-08T08:40:03.528 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libstdc++-devel-11.4.1-2.3.el9.x86_64 5/49 2023-12-08T08:40:04.791 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libss-1.46.5-4.el9.x86_64 6/49 2023-12-08T08:40:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : python3-libs-3.9.18-1.el9.x86_64 7/49 2023-12-08T08:40:04.993 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : python3-3.9.18-1.el9.x86_64 8/49 2023-12-08T08:40:05.048 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : python-unversioned-command-3.9.18-1.el9.noarch 9/49 2023-12-08T08:40:05.437 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libgomp-11.4.1-2.3.el9.x86_64 10/49 2023-12-08T08:40:06.393 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : cpp-11.4.1-2.3.el9.x86_64 11/49 2023-12-08T08:40:06.558 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : gcc-11.4.1-2.3.el9.x86_64 12/49 2023-12-08T08:40:06.598 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: gcc-11.4.1-2.3.el9.x86_64 12/49 2023-12-08T08:40:06.632 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gdbm-1:1.19-4.el9.x86_64 13/49 2023-12-08T08:40:06.669 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libnbd-1.18.1-3.el9.x86_64 14/49 2023-12-08T08:40:06.694 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 15/49 2023-12-08T08:40:06.719 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 16/49 2023-12-08T08:40:06.745 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 17/49 2023-12-08T08:40:06.778 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 18/49 2023-12-08T08:40:06.804 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 19/49 2023-12-08T08:40:06.965 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 20/49 2023-12-08T08:40:07.026 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fio-3.35-1.el9.x86_64 21/49 2023-12-08T08:40:07.062 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rmt-2:1.6-6.el9.x86_64 22/49 2023-12-08T08:40:07.093 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : dump-1:0.4-0.51.b47.el9.x86_64 23/49 2023-12-08T08:40:10.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:40:10.651 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:40:16.902 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gdbm-devel-1:1.19-4.el9.x86_64 24/49 2023-12-08T08:40:17.115 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-devel-5.14.0-390.el9.x86_64 25/49 2023-12-08T08:40:28.092 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-devel-5.14.0-390.el9.x86_64 25/49 2023-12-08T08:40:28.298 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : gcc-c++-11.4.1-2.3.el9.x86_64 26/49 2023-12-08T08:40:28.329 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : gcc-plugin-annobin-11.4.1-2.3.el9.x86_64 27/49 2023-12-08T08:40:28.446 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-2.3.el9.x86_64 27/49 2023-12-08T08:40:28.573 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : python3-devel-3.9.18-1.el9.x86_64 28/49 2023-12-08T08:40:28.717 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : e2fsprogs-1.46.5-4.el9.x86_64 29/49 2023-12-08T08:40:28.756 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : indent-2.2.13-4.el9.x86_64 30/49 2023-12-08T08:40:28.798 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : udftools-2.2-5.el9.x86_64 31/49 2023-12-08T08:40:28.841 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : sqlite-3.34.1-6.el9.x86_64 32/49 2023-12-08T08:40:28.868 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcap-devel-2.48-9.el9.x86_64 33/49 2023-12-08T08:40:28.901 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : e2fsprogs-1.46.5-3.el9.x86_64 34/49 2023-12-08T08:40:28.937 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : gcc-c++-11.4.1-2.1.el9.x86_64 35/49 2023-12-08T08:40:28.968 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : python3-devel-3.9.17-2.el9.x86_64 36/49 2023-12-08T08:40:28.993 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : python3-3.9.17-2.el9.x86_64 37/49 2023-12-08T08:40:29.018 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : python-unversioned-command-3.9.17-2.el9.noarch 38/49 2023-12-08T08:40:29.019 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 39/49 2023-12-08T08:40:29.065 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 39/49 2023-12-08T08:40:29.103 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : gcc-11.4.1-2.1.el9.x86_64 40/49 2023-12-08T08:40:29.128 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : e2fsprogs-libs-1.46.5-3.el9.x86_64 41/49 2023-12-08T08:40:29.166 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libss-1.46.5-3.el9.x86_64 42/49 2023-12-08T08:40:29.196 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libstdc++-devel-11.4.1-2.1.el9.x86_64 43/49 2023-12-08T08:40:29.221 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libstdc++-11.4.1-2.1.el9.x86_64 44/49 2023-12-08T08:40:29.221 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libgcc-11.4.1-2.1.el9.x86_64 45/49 2023-12-08T08:40:29.306 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libgcc-11.4.1-2.1.el9.x86_64 45/49 2023-12-08T08:40:29.339 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libcom_err-1.46.5-3.el9.x86_64 46/49 2023-12-08T08:40:29.373 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : cpp-11.4.1-2.1.el9.x86_64 47/49 2023-12-08T08:40:29.449 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libgomp-11.4.1-2.1.el9.x86_64 48/49 2023-12-08T08:40:29.563 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : python3-libs-3.9.17-2.el9.x86_64 49/49 2023-12-08T08:40:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: python3-libs-3.9.17-2.el9.x86_64 49/49 2023-12-08T08:40:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fio-3.35-1.el9.x86_64 1/49 2023-12-08T08:40:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 2/49 2023-12-08T08:40:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 3/49 2023-12-08T08:40:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 4/49 2023-12-08T08:40:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 5/49 2023-12-08T08:40:31.721 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 6/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 7/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcap-devel-2.48-9.el9.x86_64 8/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : sqlite-3.34.1-6.el9.x86_64 9/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : udftools-2.2-5.el9.x86_64 10/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rmt-2:1.6-6.el9.x86_64 11/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-devel-5.14.0-390.el9.x86_64 12/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libnbd-1.18.1-3.el9.x86_64 13/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gdbm-1:1.19-4.el9.x86_64 14/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gdbm-devel-1:1.19-4.el9.x86_64 15/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : dump-1:0.4-0.51.b47.el9.x86_64 16/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : indent-2.2.13-4.el9.x86_64 17/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cpp-11.4.1-2.3.el9.x86_64 18/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cpp-11.4.1-2.1.el9.x86_64 19/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gcc-11.4.1-2.3.el9.x86_64 20/49 2023-12-08T08:40:31.722 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gcc-11.4.1-2.1.el9.x86_64 21/49 2023-12-08T08:40:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gcc-c++-11.4.1-2.3.el9.x86_64 22/49 2023-12-08T08:40:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gcc-c++-11.4.1-2.1.el9.x86_64 23/49 2023-12-08T08:40:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gcc-plugin-annobin-11.4.1-2.3.el9.x86_64 24/49 2023-12-08T08:40:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gcc-plugin-annobin-11.4.1-2.1.el9.x86_64 25/49 2023-12-08T08:40:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libstdc++-devel-11.4.1-2.3.el9.x86_64 26/49 2023-12-08T08:40:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libstdc++-devel-11.4.1-2.1.el9.x86_64 27/49 2023-12-08T08:40:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python-unversioned-command-3.9.18-1.el9.noarch 28/49 2023-12-08T08:40:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python-unversioned-command-3.9.17-2.el9.noarch 29/49 2023-12-08T08:40:31.723 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.18-1.el9.x86_64 30/49 2023-12-08T08:40:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 31/49 2023-12-08T08:40:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : e2fsprogs-1.46.5-4.el9.x86_64 32/49 2023-12-08T08:40:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : e2fsprogs-1.46.5-3.el9.x86_64 33/49 2023-12-08T08:40:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : e2fsprogs-libs-1.46.5-4.el9.x86_64 34/49 2023-12-08T08:40:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : e2fsprogs-libs-1.46.5-3.el9.x86_64 35/49 2023-12-08T08:40:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcom_err-1.46.5-4.el9.x86_64 36/49 2023-12-08T08:40:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcom_err-1.46.5-3.el9.x86_64 37/49 2023-12-08T08:40:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgcc-11.4.1-2.3.el9.x86_64 38/49 2023-12-08T08:40:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgcc-11.4.1-2.1.el9.x86_64 39/49 2023-12-08T08:40:31.725 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgomp-11.4.1-2.3.el9.x86_64 40/49 2023-12-08T08:40:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgomp-11.4.1-2.1.el9.x86_64 41/49 2023-12-08T08:40:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libss-1.46.5-4.el9.x86_64 42/49 2023-12-08T08:40:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libss-1.46.5-3.el9.x86_64 43/49 2023-12-08T08:40:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libstdc++-11.4.1-2.3.el9.x86_64 44/49 2023-12-08T08:40:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libstdc++-11.4.1-2.1.el9.x86_64 45/49 2023-12-08T08:40:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-3.9.18-1.el9.x86_64 46/49 2023-12-08T08:40:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-3.9.17-2.el9.x86_64 47/49 2023-12-08T08:40:31.726 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-libs-3.9.18-1.el9.x86_64 48/49 2023-12-08T08:40:32.596 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-libs-3.9.17-2.el9.x86_64 49/49 2023-12-08T08:40:32.596 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:40:32.596 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2023-12-08T08:40:32.596 INFO:teuthology.orchestra.run.smithi016.stdout: cpp-11.4.1-2.3.el9.x86_64 2023-12-08T08:40:32.596 INFO:teuthology.orchestra.run.smithi016.stdout: e2fsprogs-1.46.5-4.el9.x86_64 2023-12-08T08:40:32.596 INFO:teuthology.orchestra.run.smithi016.stdout: e2fsprogs-libs-1.46.5-4.el9.x86_64 2023-12-08T08:40:32.596 INFO:teuthology.orchestra.run.smithi016.stdout: gcc-11.4.1-2.3.el9.x86_64 2023-12-08T08:40:32.596 INFO:teuthology.orchestra.run.smithi016.stdout: gcc-c++-11.4.1-2.3.el9.x86_64 2023-12-08T08:40:32.597 INFO:teuthology.orchestra.run.smithi016.stdout: gcc-plugin-annobin-11.4.1-2.3.el9.x86_64 2023-12-08T08:40:32.597 INFO:teuthology.orchestra.run.smithi016.stdout: libcom_err-1.46.5-4.el9.x86_64 2023-12-08T08:40:32.597 INFO:teuthology.orchestra.run.smithi016.stdout: libgcc-11.4.1-2.3.el9.x86_64 2023-12-08T08:40:32.597 INFO:teuthology.orchestra.run.smithi016.stdout: libgomp-11.4.1-2.3.el9.x86_64 2023-12-08T08:40:32.597 INFO:teuthology.orchestra.run.smithi016.stdout: libss-1.46.5-4.el9.x86_64 2023-12-08T08:40:32.597 INFO:teuthology.orchestra.run.smithi016.stdout: libstdc++-11.4.1-2.3.el9.x86_64 2023-12-08T08:40:32.597 INFO:teuthology.orchestra.run.smithi016.stdout: libstdc++-devel-11.4.1-2.3.el9.x86_64 2023-12-08T08:40:32.597 INFO:teuthology.orchestra.run.smithi016.stdout: python-unversioned-command-3.9.18-1.el9.noarch 2023-12-08T08:40:32.597 INFO:teuthology.orchestra.run.smithi016.stdout: python3-3.9.18-1.el9.x86_64 2023-12-08T08:40:32.597 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.18-1.el9.x86_64 2023-12-08T08:40:32.597 INFO:teuthology.orchestra.run.smithi016.stdout: python3-libs-3.9.18-1.el9.x86_64 2023-12-08T08:40:32.597 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-12-08T08:40:32.597 INFO:teuthology.orchestra.run.smithi016.stdout: dump-1:0.4-0.51.b47.el9.x86_64 fio-3.35-1.el9.x86_64 2023-12-08T08:40:32.597 INFO:teuthology.orchestra.run.smithi016.stdout: fio-engine-http-3.35-1.el9.x86_64 fio-engine-libaio-3.35-1.el9.x86_64 2023-12-08T08:40:32.598 INFO:teuthology.orchestra.run.smithi016.stdout: fio-engine-nbd-3.35-1.el9.x86_64 fio-engine-rados-3.35-1.el9.x86_64 2023-12-08T08:40:32.598 INFO:teuthology.orchestra.run.smithi016.stdout: fio-engine-rbd-3.35-1.el9.x86_64 fio-engine-rdma-3.35-1.el9.x86_64 2023-12-08T08:40:32.598 INFO:teuthology.orchestra.run.smithi016.stdout: gdbm-1:1.19-4.el9.x86_64 gdbm-devel-1:1.19-4.el9.x86_64 2023-12-08T08:40:32.598 INFO:teuthology.orchestra.run.smithi016.stdout: indent-2.2.13-4.el9.x86_64 kernel-devel-5.14.0-390.el9.x86_64 2023-12-08T08:40:32.598 INFO:teuthology.orchestra.run.smithi016.stdout: libcap-devel-2.48-9.el9.x86_64 libnbd-1.18.1-3.el9.x86_64 2023-12-08T08:40:32.598 INFO:teuthology.orchestra.run.smithi016.stdout: rmt-2:1.6-6.el9.x86_64 sqlite-3.34.1-6.el9.x86_64 2023-12-08T08:40:32.598 INFO:teuthology.orchestra.run.smithi016.stdout: udftools-2.2-5.el9.x86_64 2023-12-08T08:40:32.598 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:40:32.598 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-12-08T08:40:33.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo useradd -m fsgqa 2023-12-08T08:40:33.342 INFO:teuthology.orchestra.run.smithi016.stderr:useradd: user 'fsgqa' already exists 2023-12-08T08:40:33.343 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-12-08T08:40:33.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo groupadd fsgqa 2023-12-08T08:40:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:groupadd: group 'fsgqa' already exists 2023-12-08T08:40:33.416 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-12-08T08:40:33.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo useradd fsgqa2 2023-12-08T08:40:34.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo useradd 123456-fsgqa 2023-12-08T08:40:34.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2023-12-08T08:40:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.921+0000 7fd486f7b640 1 Processor -- start 2023-12-08T08:40:34.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.921+0000 7fd486f7b640 1 -- start start 2023-12-08T08:40:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.922+0000 7fd486f7b640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd480159a10 0x7fd4800b3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:40:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.922+0000 7fd486f7b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd480071760 0x7fd4800b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:40:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.922+0000 7fd486f7b640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd48006dae0 con 0x7fd480071dc0 2023-12-08T08:40:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.922+0000 7fd486f7b640 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4800b3fa0 con 0x7fd480159a10 2023-12-08T08:40:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.922+0000 7fd486f7b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4800ae2e0 con 0x7fd480071760 2023-12-08T08:40:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.922+0000 7fd484d17640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd480159a10 0x7fd4800b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:40:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.922+0000 7fd484d17640 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd480159a10 0x7fd4800b3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42586/0 (socket says 172.21.15.16:42586) 2023-12-08T08:40:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.922+0000 7fd47ffff640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd480071dc0 0x7fd48006c0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56400/0 (socket says 172.21.15.16:56400) 2023-12-08T08:40:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.922+0000 7fd484d17640 1 -- 172.21.15.16:0/1940791969 learned_addr learned my addr 172.21.15.16:0/1940791969 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:40:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.922+0000 7fd485518640 1 --2- 172.21.15.16:0/1940791969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd480071760 0x7fd4800b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:40:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.923+0000 7fd484d17640 1 -- 172.21.15.16:0/1940791969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd480071760 msgr2=0x7fd4800b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:40:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.923+0000 7fd484d17640 1 --2- 172.21.15.16:0/1940791969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd480071760 0x7fd4800b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:34.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.923+0000 7fd484d17640 1 -- 172.21.15.16:0/1940791969 >> v1:172.21.15.16:6789/0 conn(0x7fd480071dc0 legacy=0x7fd48006c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:40:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.923+0000 7fd484d17640 1 -- 172.21.15.16:0/1940791969 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48006dd30 con 0x7fd480159a10 2023-12-08T08:40:34.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.923+0000 7fd47f7fe640 1 -- 172.21.15.16:0/1940791969 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3043850261 0 0) 0x7fd48006dae0 con 0x7fd480071dc0 2023-12-08T08:40:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.923+0000 7fd484d17640 1 --2- 172.21.15.16:0/1940791969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd480159a10 0x7fd4800b3520 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd474009870 tx=0x7fd47402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463b604c2f72089b server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:40:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.923+0000 7fd47f7fe640 1 -- 172.21.15.16:0/1940791969 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4740317c0 con 0x7fd480159a10 2023-12-08T08:40:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.924+0000 7fd47f7fe640 1 -- 172.21.15.16:0/1940791969 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd474031920 con 0x7fd480159a10 2023-12-08T08:40:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.924+0000 7fd47f7fe640 1 -- 172.21.15.16:0/1940791969 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd474031cb0 con 0x7fd480159a10 2023-12-08T08:40:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.924+0000 7fd486f7b640 1 -- 172.21.15.16:0/1940791969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd480159a10 msgr2=0x7fd4800b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:40:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.924+0000 7fd486f7b640 1 --2- 172.21.15.16:0/1940791969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd480159a10 0x7fd4800b3520 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fd474009870 tx=0x7fd47402b860 comp rx=0 tx=0).stop 2023-12-08T08:40:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.924+0000 7fd486f7b640 1 -- 172.21.15.16:0/1940791969 shutdown_connections 2023-12-08T08:40:34.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.924+0000 7fd486f7b640 1 --2- 172.21.15.16:0/1940791969 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd480071760 0x7fd4800b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:34.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.924+0000 7fd486f7b640 1 --2- 172.21.15.16:0/1940791969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd480159a10 0x7fd4800b3520 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:34.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.924+0000 7fd486f7b640 1 -- 172.21.15.16:0/1940791969 >> 172.21.15.16:0/1940791969 conn(0x7fd4800b09f0 msgr2=0x7fd4800b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:40:34.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.924+0000 7fd486f7b640 1 -- 172.21.15.16:0/1940791969 shutdown_connections 2023-12-08T08:40:34.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.924+0000 7fd486f7b640 1 -- 172.21.15.16:0/1940791969 wait complete. 2023-12-08T08:40:34.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.925+0000 7fd486f7b640 1 Processor -- start 2023-12-08T08:40:34.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.925+0000 7fd486f7b640 1 -- start start 2023-12-08T08:40:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.926+0000 7fd486f7b640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd480071760 0x7fd4801f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:40:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.926+0000 7fd486f7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd480071dc0 0x7fd4801584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:40:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.926+0000 7fd486f7b640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd480159a10 0x7fd480153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:40:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.926+0000 7fd486f7b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48014d320 con 0x7fd480071dc0 2023-12-08T08:40:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.926+0000 7fd486f7b640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4801f1f70 con 0x7fd480159a10 2023-12-08T08:40:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.926+0000 7fd486f7b640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48009a980 con 0x7fd480071760 2023-12-08T08:40:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.926+0000 7fd47ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd480071dc0 0x7fd4801584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:40:34.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.926+0000 7fd47ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd480071dc0 0x7fd4801584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42598/0 (socket says 172.21.15.16:42598) 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.926+0000 7fd47ffff640 1 -- 172.21.15.16:0/3732582993 learned_addr learned my addr 172.21.15.16:0/3732582993 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.927+0000 7fd484d17640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd480071760 0x7fd4801f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.927+0000 7fd485518640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd480159a10 0x7fd480153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.927+0000 7fd47ffff640 1 -- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd480071760 msgr2=0x7fd4801f1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.927+0000 7fd47ffff640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd480071760 0x7fd4801f1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.927+0000 7fd47ffff640 1 -- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd480159a10 msgr2=0x7fd480153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.927+0000 7fd47ffff640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd480159a10 0x7fd480153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.927+0000 7fd47ffff640 1 -- 172.21.15.16:0/3732582993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4740094d0 con 0x7fd480071dc0 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.927+0000 7fd485518640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd480159a10 0x7fd480153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.927+0000 7fd47ffff640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd480071dc0 0x7fd4801584c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00e960 tx=0x7fd46c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.927+0000 7fd47d7fa640 1 -- 172.21.15.16:0/3732582993 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0072e0 con 0x7fd480071dc0 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.927+0000 7fd47d7fa640 1 -- 172.21.15.16:0/3732582993 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c007460 con 0x7fd480071dc0 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.927+0000 7fd486f7b640 1 -- 172.21.15.16:0/3732582993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48015c7e0 con 0x7fd480071dc0 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.927+0000 7fd47d7fa640 1 -- 172.21.15.16:0/3732582993 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c00c850 con 0x7fd480071dc0 2023-12-08T08:40:34.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.927+0000 7fd486f7b640 1 -- 172.21.15.16:0/3732582993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48015cb70 con 0x7fd480071dc0 2023-12-08T08:40:34.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.928+0000 7fd486f7b640 1 -- 172.21.15.16:0/3732582993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48009ab10 con 0x7fd480071dc0 2023-12-08T08:40:34.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.929+0000 7fd47d7fa640 1 -- 172.21.15.16:0/3732582993 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd46c00c9b0 con 0x7fd480071dc0 2023-12-08T08:40:34.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.929+0000 7fd47d7fa640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd46406ae40 0x7fd46406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:40:34.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.929+0000 7fd47d7fa640 1 -- 172.21.15.16:0/3732582993 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd46c08bbc0 con 0x7fd480071dc0 2023-12-08T08:40:34.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.929+0000 7fd484d17640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd46406ae40 0x7fd46406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:40:34.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.930+0000 7fd484d17640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd46406ae40 0x7fd46406d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd474036000 tx=0x7fd47403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:40:34.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:34.934+0000 7fd47d7fa640 1 -- 172.21.15.16:0/3732582993 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd46c056b10 con 0x7fd480071dc0 2023-12-08T08:40:35.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.180+0000 7fd486f7b640 1 -- 172.21.15.16:0/3732582993 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd4800add50 con 0x7fd480071dc0 2023-12-08T08:40:35.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.180+0000 7fd47d7fa640 1 -- 172.21.15.16:0/3732582993 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+1276 (secure 0 0 0) 0x7fd46c015070 con 0x7fd480071dc0 2023-12-08T08:40:35.183 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:40:35.183 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":1,"fsid":"ead7ea87-fe4a-461b-916c-c6458a26d625","modified":"2023-12-08T08:37:12.037867Z","created":"2023-12-08T08:37:12.037867Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3300","nonce":0},{"type":"v1","addr":"172.21.15.112:6789","nonce":0}]},"addr":"172.21.15.112:6789/0","public_addr":"172.21.15.112:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.112:3301","nonce":0},{"type":"v1","addr":"172.21.15.112:6790","nonce":0}]},"addr":"172.21.15.112:6790/0","public_addr":"172.21.15.112:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-12-08T08:40:35.183 INFO:teuthology.orchestra.run.smithi016.stderr:dumped monmap epoch 1 2023-12-08T08:40:35.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.182+0000 7fd486f7b640 1 -- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd46406ae40 msgr2=0x7fd46406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:40:35.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.182+0000 7fd486f7b640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd46406ae40 0x7fd46406d300 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd474036000 tx=0x7fd47403d040 comp rx=0 tx=0).stop 2023-12-08T08:40:35.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.182+0000 7fd486f7b640 1 -- 172.21.15.16:0/3732582993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd480071dc0 msgr2=0x7fd4801584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:40:35.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.182+0000 7fd486f7b640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd480071dc0 0x7fd4801584c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00e960 tx=0x7fd46c00ee20 comp rx=0 tx=0).stop 2023-12-08T08:40:35.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.182+0000 7fd486f7b640 1 -- 172.21.15.16:0/3732582993 shutdown_connections 2023-12-08T08:40:35.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.182+0000 7fd486f7b640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7fd46406ae40 0x7fd46406d300 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:35.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.182+0000 7fd486f7b640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7fd480159a10 0x7fd480153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:35.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.182+0000 7fd486f7b640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd480071dc0 0x7fd4801584c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:35.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.182+0000 7fd486f7b640 1 --2- 172.21.15.16:0/3732582993 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7fd480071760 0x7fd4801f1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:35.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.182+0000 7fd486f7b640 1 -- 172.21.15.16:0/3732582993 >> 172.21.15.16:0/3732582993 conn(0x7fd4800b09f0 msgr2=0x7fd4800b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:40:35.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.183+0000 7fd486f7b640 1 -- 172.21.15.16:0/3732582993 shutdown_connections 2023-12-08T08:40:35.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.183+0000 7fd486f7b640 1 -- 172.21.15.16:0/3732582993 wait complete. 2023-12-08T08:40:35.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2023-12-08T08:40:35.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.301+0000 7f2faa1b6640 1 Processor -- start 2023-12-08T08:40:35.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.301+0000 7f2faa1b6640 1 -- start start 2023-12-08T08:40:35.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.302+0000 7f2faa1b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fa4140a10 0x7f2fa40cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:40:35.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.302+0000 7f2faa1b6640 1 --2- >> v2:172.21.15.112:3300/0 conn(0x7f2fa40b2640 0x7f2fa40ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:40:35.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.302+0000 7f2faa1b6640 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fa4070db0 con 0x7f2fa4071e50 2023-12-08T08:40:35.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.302+0000 7f2faa1b6640 1 -- --> v2:172.21.15.112:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fa4070ba0 con 0x7f2fa40b2640 2023-12-08T08:40:35.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.302+0000 7f2faa1b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa4071090 con 0x7f2fa4140a10 2023-12-08T08:40:35.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.302+0000 7f2fa2ffd640 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2fa4071e50 0x7f2fa40b1f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56412/0 (socket says 172.21.15.16:56412) 2023-12-08T08:40:35.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.302+0000 7f2fa2ffd640 1 -- 172.21.15.16:0/3133181907 learned_addr learned my addr 172.21.15.16:0/3133181907 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-12-08T08:40:35.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.303+0000 7f2fa37fe640 1 --2- 172.21.15.16:0/3133181907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fa4140a10 0x7f2fa40cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:40:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.303+0000 7f2fa3fff640 1 --2- 172.21.15.16:0/3133181907 >> v2:172.21.15.112:3300/0 conn(0x7f2fa40b2640 0x7f2fa40ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:40:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.303+0000 7f2fa27fc640 1 -- 172.21.15.16:0/3133181907 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686261208 0 0) 0x7f2fa4070db0 con 0x7f2fa4071e50 2023-12-08T08:40:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.303+0000 7f2fa27fc640 1 -- 172.21.15.16:0/3133181907 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f8c003520 con 0x7f2fa4071e50 2023-12-08T08:40:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.303+0000 7f2fa37fe640 1 -- 172.21.15.16:0/3133181907 >> v2:172.21.15.112:3300/0 conn(0x7f2fa40b2640 msgr2=0x7f2fa40ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-08T08:40:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.303+0000 7f2fa37fe640 1 --2- 172.21.15.16:0/3133181907 >> v2:172.21.15.112:3300/0 conn(0x7f2fa40b2640 0x7f2fa40ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.303+0000 7f2fa37fe640 1 -- 172.21.15.16:0/3133181907 >> v1:172.21.15.16:6789/0 conn(0x7f2fa4071e50 legacy=0x7f2fa40b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:40:35.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.303+0000 7f2fa37fe640 1 -- 172.21.15.16:0/3133181907 --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa4071370 con 0x7f2fa4140a10 2023-12-08T08:40:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.304+0000 7f2fa37fe640 1 --2- 172.21.15.16:0/3133181907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fa4140a10 0x7f2fa40cdcd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f940097a0 tx=0x7f2f9402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d3e731708f2bb5b server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:40:35.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.304+0000 7f2fa27fc640 1 -- 172.21.15.16:0/3133181907 <== mon.2 v2:172.21.15.112:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94030610 con 0x7f2fa4140a10 2023-12-08T08:40:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.304+0000 7f2fa27fc640 1 -- 172.21.15.16:0/3133181907 <== mon.2 v2:172.21.15.112:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f94030770 con 0x7f2fa4140a10 2023-12-08T08:40:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.304+0000 7f2fa27fc640 1 -- 172.21.15.16:0/3133181907 <== mon.2 v2:172.21.15.112:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f94030ae0 con 0x7f2fa4140a10 2023-12-08T08:40:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.304+0000 7f2faa1b6640 1 -- 172.21.15.16:0/3133181907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fa4140a10 msgr2=0x7f2fa40cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:40:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.304+0000 7f2faa1b6640 1 --2- 172.21.15.16:0/3133181907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fa4140a10 0x7f2fa40cdcd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2f940097a0 tx=0x7f2f9402b960 comp rx=0 tx=0).stop 2023-12-08T08:40:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.304+0000 7f2faa1b6640 1 -- 172.21.15.16:0/3133181907 shutdown_connections 2023-12-08T08:40:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.304+0000 7f2faa1b6640 1 --2- 172.21.15.16:0/3133181907 >> v2:172.21.15.112:3300/0 conn(0x7f2fa40b2640 0x7f2fa40ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.304+0000 7f2faa1b6640 1 --2- 172.21.15.16:0/3133181907 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fa4140a10 0x7f2fa40cdcd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.304+0000 7f2faa1b6640 1 -- 172.21.15.16:0/3133181907 >> 172.21.15.16:0/3133181907 conn(0x7f2fa40b09f0 msgr2=0x7f2fa40b0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:40:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.304+0000 7f2faa1b6640 1 -- 172.21.15.16:0/3133181907 shutdown_connections 2023-12-08T08:40:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.304+0000 7f2faa1b6640 1 -- 172.21.15.16:0/3133181907 wait complete. 2023-12-08T08:40:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.306+0000 7f2faa1b6640 1 Processor -- start 2023-12-08T08:40:35.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.306+0000 7f2faa1b6640 1 -- start start 2023-12-08T08:40:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.306+0000 7f2faa1b6640 1 --2- >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fa4071e50 0x7f2fa40ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:40:35.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2faa1b6640 1 --2- >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa40b2640 0x7f2fa4140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:40:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2faa1b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa4140a10 0x7f2fa4142f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:40:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2faa1b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa4141830 con 0x7f2fa4140a10 2023-12-08T08:40:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2faa1b6640 1 -- --> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa4141b10 con 0x7f2fa40b2640 2023-12-08T08:40:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2faa1b6640 1 -- --> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fa4141df0 con 0x7f2fa4071e50 2023-12-08T08:40:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2fa3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa4140a10 0x7f2fa4142f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:40:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2fa3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa4140a10 0x7f2fa4142f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42602/0 (socket says 172.21.15.16:42602) 2023-12-08T08:40:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2fa3fff640 1 -- 172.21.15.16:0/573635344 learned_addr learned my addr 172.21.15.16:0/573635344 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-12-08T08:40:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2fa2ffd640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa40b2640 0x7f2fa4140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:40:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2fa37fe640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fa4071e50 0x7f2fa40ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:40:35.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2fa3fff640 1 -- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fa4071e50 msgr2=0x7f2fa40ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:40:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2fa3fff640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fa4071e50 0x7f2fa40ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2fa3fff640 1 -- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa40b2640 msgr2=0x7f2fa4140fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:40:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2fa3fff640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa40b2640 0x7f2fa4140fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2fa3fff640 1 -- 172.21.15.16:0/573635344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f94009400 con 0x7f2fa4140a10 2023-12-08T08:40:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2fa2ffd640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa40b2640 0x7f2fa4140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:40:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2fa37fe640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fa4071e50 0x7f2fa40ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-08T08:40:35.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2fa3fff640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa4140a10 0x7f2fa4142f70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00efc0 tx=0x7f2f9c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:40:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2f8bfff640 1 -- 172.21.15.16:0/573635344 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c014480 con 0x7f2fa4140a10 2023-12-08T08:40:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2f8bfff640 1 -- 172.21.15.16:0/573635344 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9c00f040 con 0x7f2fa4140a10 2023-12-08T08:40:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2faa1b6640 1 -- 172.21.15.16:0/573635344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fa41420d0 con 0x7f2fa4140a10 2023-12-08T08:40:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.307+0000 7f2f8bfff640 1 -- 172.21.15.16:0/573635344 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c014820 con 0x7f2fa4140a10 2023-12-08T08:40:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.308+0000 7f2faa1b6640 1 -- 172.21.15.16:0/573635344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fa41414e0 con 0x7f2fa4140a10 2023-12-08T08:40:35.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.308+0000 7f2faa1b6640 1 -- 172.21.15.16:0/573635344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fa4074320 con 0x7f2fa4140a10 2023-12-08T08:40:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.309+0000 7f2f8bfff640 1 -- 172.21.15.16:0/573635344 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f9c014980 con 0x7f2fa4140a10 2023-12-08T08:40:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.309+0000 7f2f8bfff640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f9006ae40 0x7f2f9006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-08T08:40:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.309+0000 7f2f8bfff640 1 -- 172.21.15.16:0/573635344 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2f9c08a6b0 con 0x7f2fa4140a10 2023-12-08T08:40:35.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.310+0000 7f2fa37fe640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f9006ae40 0x7f2f9006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-08T08:40:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.310+0000 7f2fa37fe640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f9006ae40 0x7f2f9006d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2f94036000 tx=0x7f2f9403b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-08T08:40:35.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.314+0000 7f2f8bfff640 1 -- 172.21.15.16:0/573635344 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f9c055600 con 0x7f2fa4140a10 2023-12-08T08:40:35.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.556+0000 7f2faa1b6640 1 -- 172.21.15.16:0/573635344 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) v1 -- 0x7f2fa4067430 con 0x7f2fa4140a10 2023-12-08T08:40:35.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.557+0000 7f2f8bfff640 1 -- 172.21.15.16:0/573635344 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v15) v1 ==== 82+0+151 (secure 0 0 0) 0x7f2f9c0180b0 con 0x7f2fa4140a10 2023-12-08T08:40:35.559 INFO:teuthology.orchestra.run.smithi016.stdout:[client.admin] 2023-12-08T08:40:35.559 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQC41XJlmZ4qEBAAXeuHA73CaY3xR9ZyBLHyDQ== 2023-12-08T08:40:35.559 INFO:teuthology.orchestra.run.smithi016.stdout: caps mds = "allow *" 2023-12-08T08:40:35.559 INFO:teuthology.orchestra.run.smithi016.stdout: caps mgr = "allow *" 2023-12-08T08:40:35.559 INFO:teuthology.orchestra.run.smithi016.stdout: caps mon = "allow *" 2023-12-08T08:40:35.559 INFO:teuthology.orchestra.run.smithi016.stdout: caps osd = "allow *" 2023-12-08T08:40:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.559+0000 7f2faa1b6640 1 -- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f9006ae40 msgr2=0x7f2f9006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:40:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.559+0000 7f2faa1b6640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f9006ae40 0x7f2f9006d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2f94036000 tx=0x7f2f9403b040 comp rx=0 tx=0).stop 2023-12-08T08:40:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.559+0000 7f2faa1b6640 1 -- 172.21.15.16:0/573635344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa4140a10 msgr2=0x7f2fa4142f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-08T08:40:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.559+0000 7f2faa1b6640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa4140a10 0x7f2fa4142f70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00efc0 tx=0x7f2f9c00c490 comp rx=0 tx=0).stop 2023-12-08T08:40:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.559+0000 7f2faa1b6640 1 -- 172.21.15.16:0/573635344 shutdown_connections 2023-12-08T08:40:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.559+0000 7f2faa1b6640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:6832/804067392,v1:172.21.15.112:6833/804067392] conn(0x7f2f9006ae40 0x7f2f9006d300 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.559+0000 7f2faa1b6640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa4140a10 0x7f2fa4142f70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.559+0000 7f2faa1b6640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:3300/0,v1:172.21.15.112:6789/0] conn(0x7f2fa40b2640 0x7f2fa4140fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:35.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.559+0000 7f2faa1b6640 1 --2- 172.21.15.16:0/573635344 >> [v2:172.21.15.112:3301/0,v1:172.21.15.112:6790/0] conn(0x7f2fa4071e50 0x7f2fa40ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-08T08:40:35.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.559+0000 7f2faa1b6640 1 -- 172.21.15.16:0/573635344 >> 172.21.15.16:0/573635344 conn(0x7f2fa40b09f0 msgr2=0x7f2fa406c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-08T08:40:35.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.559+0000 7f2faa1b6640 1 -- 172.21.15.16:0/573635344 shutdown_connections 2023-12-08T08:40:35.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-12-08T08:40:35.559+0000 7f2faa1b6640 1 -- 172.21.15.16:0/573635344 wait complete. 2023-12-08T08:40:35.572 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2023-12-08T08:40:35.587 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.q3tU3Qhi75 2023-12-08T08:40:35.588 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:40:35.588 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/tmp/tmp.q3tU3Qhi75 2023-12-08T08:40:35.644 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:40:35.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/tmp/tmp.MkHH5XcxMMxfstests-dev/local.config 2023-12-08T08:40:35.715 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.hY9PAzp4zMtest export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.sNDd1g2I6qscratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.16:6789 -k /tmp/tmp.q3tU3Qhi75 --client_mountpoint /test" 2023-12-08T08:40:35.715 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T08:40:35.716 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/tmp/tmp.MkHH5XcxMMxfstests-dev/ceph.exclude 2023-12-08T08:40:35.787 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo make) 2023-12-08T08:40:35.881 INFO:teuthology.orchestra.run.smithi016.stdout:libtoolize -cfi 2023-12-08T08:40:36.321 INFO:teuthology.orchestra.run.smithi016.stdout:libtoolize: putting auxiliary files in '.'. 2023-12-08T08:40:36.321 INFO:teuthology.orchestra.run.smithi016.stdout:libtoolize: copying file './config.guess' 2023-12-08T08:40:36.324 INFO:teuthology.orchestra.run.smithi016.stdout:libtoolize: copying file './config.sub' 2023-12-08T08:40:36.328 INFO:teuthology.orchestra.run.smithi016.stdout:libtoolize: copying file './install-sh' 2023-12-08T08:40:36.332 INFO:teuthology.orchestra.run.smithi016.stdout:libtoolize: copying file './ltmain.sh' 2023-12-08T08:40:36.354 INFO:teuthology.orchestra.run.smithi016.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2023-12-08T08:40:36.354 INFO:teuthology.orchestra.run.smithi016.stdout:libtoolize: copying file 'm4/libtool.m4' 2023-12-08T08:40:36.358 INFO:teuthology.orchestra.run.smithi016.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2023-12-08T08:40:36.363 INFO:teuthology.orchestra.run.smithi016.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2023-12-08T08:40:36.367 INFO:teuthology.orchestra.run.smithi016.stdout:libtoolize: copying file 'm4/ltversion.m4' 2023-12-08T08:40:36.372 INFO:teuthology.orchestra.run.smithi016.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2023-12-08T08:40:36.378 INFO:teuthology.orchestra.run.smithi016.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2023-12-08T08:40:36.380 INFO:teuthology.orchestra.run.smithi016.stdout:cp include/install-sh . 2023-12-08T08:40:36.381 INFO:teuthology.orchestra.run.smithi016.stdout:aclocal -I m4 2023-12-08T08:40:37.462 INFO:teuthology.orchestra.run.smithi016.stdout:autoheader 2023-12-08T08:40:37.761 INFO:teuthology.orchestra.run.smithi016.stdout:autoconf 2023-12-08T08:40:37.936 INFO:teuthology.orchestra.run.smithi016.stdout:./configure \ 2023-12-08T08:40:37.936 INFO:teuthology.orchestra.run.smithi016.stdout: --libexecdir=/usr/lib \ 2023-12-08T08:40:37.936 INFO:teuthology.orchestra.run.smithi016.stdout: --exec_prefix=/var/lib 2023-12-08T08:40:38.148 INFO:teuthology.orchestra.run.smithi016.stdout:checking build system type... x86_64-pc-linux-gnu 2023-12-08T08:40:38.148 INFO:teuthology.orchestra.run.smithi016.stdout:checking host system type... x86_64-pc-linux-gnu 2023-12-08T08:40:38.149 INFO:teuthology.orchestra.run.smithi016.stdout:checking how to print strings... printf 2023-12-08T08:40:38.149 INFO:teuthology.orchestra.run.smithi016.stdout:checking for gcc... gcc 2023-12-08T08:40:38.206 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether the C compiler works... yes 2023-12-08T08:40:38.206 INFO:teuthology.orchestra.run.smithi016.stdout:checking for C compiler default output file name... a.out 2023-12-08T08:40:38.233 INFO:teuthology.orchestra.run.smithi016.stdout:checking for suffix of executables... 2023-12-08T08:40:38.296 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether we are cross compiling... no 2023-12-08T08:40:38.313 INFO:teuthology.orchestra.run.smithi016.stdout:checking for suffix of object files... o 2023-12-08T08:40:38.329 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether we are using the GNU C compiler... yes 2023-12-08T08:40:38.344 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether gcc accepts -g... yes 2023-12-08T08:40:38.371 INFO:teuthology.orchestra.run.smithi016.stdout:checking for gcc option to accept ISO C89... none needed 2023-12-08T08:40:38.373 INFO:teuthology.orchestra.run.smithi016.stdout:checking for a sed that does not truncate output... /bin/sed 2023-12-08T08:40:38.375 INFO:teuthology.orchestra.run.smithi016.stdout:checking for grep that handles long lines and -e... /bin/grep 2023-12-08T08:40:38.376 INFO:teuthology.orchestra.run.smithi016.stdout:checking for egrep... /bin/grep -E 2023-12-08T08:40:38.378 INFO:teuthology.orchestra.run.smithi016.stdout:checking for fgrep... /bin/grep -F 2023-12-08T08:40:38.381 INFO:teuthology.orchestra.run.smithi016.stdout:checking for ld used by gcc... /bin/ld 2023-12-08T08:40:38.382 INFO:teuthology.orchestra.run.smithi016.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2023-12-08T08:40:38.387 INFO:teuthology.orchestra.run.smithi016.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2023-12-08T08:40:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2023-12-08T08:40:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether ln -s works... yes 2023-12-08T08:40:39.370 INFO:teuthology.orchestra.run.smithi016.stdout:checking the maximum length of command line arguments... 1572864 2023-12-08T08:40:39.373 INFO:teuthology.orchestra.run.smithi016.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2023-12-08T08:40:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2023-12-08T08:40:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:checking for /bin/ld option to reload object files... -r 2023-12-08T08:40:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:checking for objdump... objdump 2023-12-08T08:40:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:checking how to recognize dependent libraries... pass_all 2023-12-08T08:40:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:checking for dlltool... no 2023-12-08T08:40:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:checking how to associate runtime and link libraries... printf %s\n 2023-12-08T08:40:39.374 INFO:teuthology.orchestra.run.smithi016.stdout:checking for ar... ar 2023-12-08T08:40:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:checking for archiver @FILE support... @ 2023-12-08T08:40:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:checking for strip... strip 2023-12-08T08:40:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:checking for ranlib... ranlib 2023-12-08T08:40:39.417 INFO:teuthology.orchestra.run.smithi016.stdout:checking for gawk... gawk 2023-12-08T08:40:39.484 INFO:teuthology.orchestra.run.smithi016.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2023-12-08T08:40:39.486 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sysroot... no 2023-12-08T08:40:39.488 INFO:teuthology.orchestra.run.smithi016.stdout:checking for a working dd... /bin/dd 2023-12-08T08:40:39.491 INFO:teuthology.orchestra.run.smithi016.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2023-12-08T08:40:39.504 INFO:teuthology.orchestra.run.smithi016.stdout:checking for mt... no 2023-12-08T08:40:39.507 INFO:teuthology.orchestra.run.smithi016.stdout:checking if : is a manifest tool... no 2023-12-08T08:40:39.564 INFO:teuthology.orchestra.run.smithi016.stdout:checking how to run the C preprocessor... gcc -E 2023-12-08T08:40:39.754 INFO:teuthology.orchestra.run.smithi016.stdout:checking for ANSI C header files... yes 2023-12-08T08:40:39.774 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/types.h... yes 2023-12-08T08:40:39.809 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/stat.h... yes 2023-12-08T08:40:39.832 INFO:teuthology.orchestra.run.smithi016.stdout:checking for stdlib.h... yes 2023-12-08T08:40:39.857 INFO:teuthology.orchestra.run.smithi016.stdout:checking for string.h... yes 2023-12-08T08:40:39.890 INFO:teuthology.orchestra.run.smithi016.stdout:checking for memory.h... yes 2023-12-08T08:40:39.914 INFO:teuthology.orchestra.run.smithi016.stdout:checking for strings.h... yes 2023-12-08T08:40:39.942 INFO:teuthology.orchestra.run.smithi016.stdout:checking for inttypes.h... yes 2023-12-08T08:40:39.968 INFO:teuthology.orchestra.run.smithi016.stdout:checking for stdint.h... yes 2023-12-08T08:40:40.004 INFO:teuthology.orchestra.run.smithi016.stdout:checking for unistd.h... yes 2023-12-08T08:40:40.040 INFO:teuthology.orchestra.run.smithi016.stdout:checking for dlfcn.h... yes 2023-12-08T08:40:40.043 INFO:teuthology.orchestra.run.smithi016.stdout:checking for objdir... .libs 2023-12-08T08:40:40.106 INFO:teuthology.orchestra.run.smithi016.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2023-12-08T08:40:40.107 INFO:teuthology.orchestra.run.smithi016.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2023-12-08T08:40:40.122 INFO:teuthology.orchestra.run.smithi016.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2023-12-08T08:40:40.140 INFO:teuthology.orchestra.run.smithi016.stdout:checking if gcc static flag -static works... no 2023-12-08T08:40:40.160 INFO:teuthology.orchestra.run.smithi016.stdout:checking if gcc supports -c -o file.o... yes 2023-12-08T08:40:40.160 INFO:teuthology.orchestra.run.smithi016.stdout:checking if gcc supports -c -o file.o... (cached) yes 2023-12-08T08:40:40.166 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2023-12-08T08:40:40.193 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether -lc should be explicitly linked in... no 2023-12-08T08:40:40.249 INFO:teuthology.orchestra.run.smithi016.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2023-12-08T08:40:40.249 INFO:teuthology.orchestra.run.smithi016.stdout:checking how to hardcode library paths into programs... immediate 2023-12-08T08:40:40.251 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether stripping libraries is possible... yes 2023-12-08T08:40:40.251 INFO:teuthology.orchestra.run.smithi016.stdout:checking if libtool supports shared libraries... yes 2023-12-08T08:40:40.251 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether to build shared libraries... yes 2023-12-08T08:40:40.251 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether to build static libraries... yes 2023-12-08T08:40:40.257 INFO:teuthology.orchestra.run.smithi016.stdout:checking for gcc... (cached) gcc 2023-12-08T08:40:40.275 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-12-08T08:40:40.275 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether gcc accepts -g... (cached) yes 2023-12-08T08:40:40.275 INFO:teuthology.orchestra.run.smithi016.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-12-08T08:40:40.275 INFO:teuthology.orchestra.run.smithi016.stdout:checking for gmake... /usr/bin/gmake 2023-12-08T08:40:40.275 INFO:teuthology.orchestra.run.smithi016.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2023-12-08T08:40:40.276 INFO:teuthology.orchestra.run.smithi016.stdout:checking for tar... /bin/tar 2023-12-08T08:40:40.277 INFO:teuthology.orchestra.run.smithi016.stdout:checking for gzip... /bin/gzip 2023-12-08T08:40:40.284 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether gcc -MM is supported... yes 2023-12-08T08:40:40.284 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sort... /bin/sort 2023-12-08T08:40:40.284 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether ln -s works... yes 2023-12-08T08:40:40.284 INFO:teuthology.orchestra.run.smithi016.stdout:checking for rpm... /bin/rpm 2023-12-08T08:40:40.287 INFO:teuthology.orchestra.run.smithi016.stdout:checking for rpmbuild... no 2023-12-08T08:40:40.312 INFO:teuthology.orchestra.run.smithi016.stdout:checking assert.h usability... yes 2023-12-08T08:40:40.321 INFO:teuthology.orchestra.run.smithi016.stdout:checking assert.h presence... yes 2023-12-08T08:40:40.321 INFO:teuthology.orchestra.run.smithi016.stdout:checking for assert.h... yes 2023-12-08T08:40:40.347 INFO:teuthology.orchestra.run.smithi016.stdout:checking bstring.h usability... no 2023-12-08T08:40:40.358 INFO:teuthology.orchestra.run.smithi016.stdout:checking bstring.h presence... no 2023-12-08T08:40:40.358 INFO:teuthology.orchestra.run.smithi016.stdout:checking for bstring.h... no 2023-12-08T08:40:40.382 INFO:teuthology.orchestra.run.smithi016.stdout:checking libgen.h usability... yes 2023-12-08T08:40:40.391 INFO:teuthology.orchestra.run.smithi016.stdout:checking libgen.h presence... yes 2023-12-08T08:40:40.391 INFO:teuthology.orchestra.run.smithi016.stdout:checking for libgen.h... yes 2023-12-08T08:40:40.421 INFO:teuthology.orchestra.run.smithi016.stdout:checking dirent.h usability... yes 2023-12-08T08:40:40.431 INFO:teuthology.orchestra.run.smithi016.stdout:checking dirent.h presence... yes 2023-12-08T08:40:40.431 INFO:teuthology.orchestra.run.smithi016.stdout:checking for dirent.h... yes 2023-12-08T08:40:40.474 INFO:teuthology.orchestra.run.smithi016.stdout:checking errno.h usability... yes 2023-12-08T08:40:40.483 INFO:teuthology.orchestra.run.smithi016.stdout:checking errno.h presence... yes 2023-12-08T08:40:40.483 INFO:teuthology.orchestra.run.smithi016.stdout:checking for errno.h... yes 2023-12-08T08:40:40.520 INFO:teuthology.orchestra.run.smithi016.stdout:checking malloc.h usability... yes 2023-12-08T08:40:40.530 INFO:teuthology.orchestra.run.smithi016.stdout:checking malloc.h presence... yes 2023-12-08T08:40:40.530 INFO:teuthology.orchestra.run.smithi016.stdout:checking for malloc.h... yes 2023-12-08T08:40:40.557 INFO:teuthology.orchestra.run.smithi016.stdout:checking uuid.h usability... no 2023-12-08T08:40:40.567 INFO:teuthology.orchestra.run.smithi016.stdout:checking uuid.h presence... no 2023-12-08T08:40:40.567 INFO:teuthology.orchestra.run.smithi016.stdout:checking for uuid.h... no 2023-12-08T08:40:40.608 INFO:teuthology.orchestra.run.smithi016.stdout:checking uuid/uuid.h usability... yes 2023-12-08T08:40:40.618 INFO:teuthology.orchestra.run.smithi016.stdout:checking uuid/uuid.h presence... yes 2023-12-08T08:40:40.619 INFO:teuthology.orchestra.run.smithi016.stdout:checking for uuid/uuid.h... yes 2023-12-08T08:40:40.645 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/uuid.h usability... no 2023-12-08T08:40:40.655 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/uuid.h presence... no 2023-12-08T08:40:40.655 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/uuid.h... no 2023-12-08T08:40:40.686 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/file.h usability... yes 2023-12-08T08:40:40.694 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/file.h presence... yes 2023-12-08T08:40:40.694 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/file.h... yes 2023-12-08T08:40:40.729 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/fcntl.h usability... yes 2023-12-08T08:40:40.738 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/fcntl.h presence... yes 2023-12-08T08:40:40.738 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/fcntl.h... yes 2023-12-08T08:40:40.765 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/syssgi.h usability... no 2023-12-08T08:40:40.776 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/syssgi.h presence... no 2023-12-08T08:40:40.776 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/syssgi.h... no 2023-12-08T08:40:40.822 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/param.h usability... yes 2023-12-08T08:40:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/param.h presence... yes 2023-12-08T08:40:40.832 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/param.h... yes 2023-12-08T08:40:40.836 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/stat.h... (cached) yes 2023-12-08T08:40:40.864 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/statvfs.h usability... yes 2023-12-08T08:40:40.873 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/statvfs.h presence... yes 2023-12-08T08:40:40.873 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/statvfs.h... yes 2023-12-08T08:40:40.900 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/time.h usability... yes 2023-12-08T08:40:40.909 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/time.h presence... yes 2023-12-08T08:40:40.909 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/time.h... yes 2023-12-08T08:40:40.949 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/ioctl.h usability... yes 2023-12-08T08:40:40.957 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/ioctl.h presence... yes 2023-12-08T08:40:40.957 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/ioctl.h... yes 2023-12-08T08:40:40.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:40:40.970 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:40:40.993 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/wait.h usability... yes 2023-12-08T08:40:41.003 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/wait.h presence... yes 2023-12-08T08:40:41.004 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/wait.h... yes 2023-12-08T08:40:41.008 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/types.h... (cached) yes 2023-12-08T08:40:41.013 INFO:teuthology.orchestra.run.smithi016.stdout:checking for strings.h... (cached) yes 2023-12-08T08:40:41.041 INFO:teuthology.orchestra.run.smithi016.stdout:checking err.h usability... yes 2023-12-08T08:40:41.049 INFO:teuthology.orchestra.run.smithi016.stdout:checking err.h presence... yes 2023-12-08T08:40:41.050 INFO:teuthology.orchestra.run.smithi016.stdout:checking for err.h... yes 2023-12-08T08:40:41.078 INFO:teuthology.orchestra.run.smithi016.stdout:checking linux/falloc.h usability... yes 2023-12-08T08:40:41.086 INFO:teuthology.orchestra.run.smithi016.stdout:checking linux/falloc.h presence... yes 2023-12-08T08:40:41.086 INFO:teuthology.orchestra.run.smithi016.stdout:checking for linux/falloc.h... yes 2023-12-08T08:40:41.113 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/fs/xfs_fsops.h usability... no 2023-12-08T08:40:41.123 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/fs/xfs_fsops.h presence... no 2023-12-08T08:40:41.124 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/fs/xfs_fsops.h... no 2023-12-08T08:40:41.148 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/fs/xfs_itable.h usability... no 2023-12-08T08:40:41.158 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/fs/xfs_itable.h presence... no 2023-12-08T08:40:41.158 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/fs/xfs_itable.h... no 2023-12-08T08:40:41.183 INFO:teuthology.orchestra.run.smithi016.stdout:checking xfs/platform_defs.h usability... no 2023-12-08T08:40:41.194 INFO:teuthology.orchestra.run.smithi016.stdout:checking xfs/platform_defs.h presence... no 2023-12-08T08:40:41.194 INFO:teuthology.orchestra.run.smithi016.stdout:checking for xfs/platform_defs.h... no 2023-12-08T08:40:41.218 INFO:teuthology.orchestra.run.smithi016.stdout:checking btrfs/ioctl.h usability... no 2023-12-08T08:40:41.229 INFO:teuthology.orchestra.run.smithi016.stdout:checking btrfs/ioctl.h presence... no 2023-12-08T08:40:41.229 INFO:teuthology.orchestra.run.smithi016.stdout:checking for btrfs/ioctl.h... no 2023-12-08T08:40:41.253 INFO:teuthology.orchestra.run.smithi016.stdout:checking cifs/ioctl.h usability... no 2023-12-08T08:40:41.264 INFO:teuthology.orchestra.run.smithi016.stdout:checking cifs/ioctl.h presence... no 2023-12-08T08:40:41.264 INFO:teuthology.orchestra.run.smithi016.stdout:checking for cifs/ioctl.h... no 2023-12-08T08:40:41.291 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/mman.h usability... yes 2023-12-08T08:40:41.300 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/mman.h presence... yes 2023-12-08T08:40:41.300 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/mman.h... yes 2023-12-08T08:40:41.314 INFO:teuthology.orchestra.run.smithi016.stdout:checking for xfs/xfs_log_format.h... no 2023-12-08T08:40:41.352 INFO:teuthology.orchestra.run.smithi016.stdout:checking for uuid_compare... no 2023-12-08T08:40:41.384 INFO:teuthology.orchestra.run.smithi016.stdout:checking for uuid_compare in -luuid... yes 2023-12-08T08:40:41.453 INFO:teuthology.orchestra.run.smithi016.stdout:checking for xfs/xfs.h... yes 2023-12-08T08:40:41.466 INFO:teuthology.orchestra.run.smithi016.stdout:checking for xfs/libxfs.h... no 2023-12-08T08:40:41.481 INFO:teuthology.orchestra.run.smithi016.stdout:checking whether xlog_assign_lsn is declared... no 2023-12-08T08:40:41.511 INFO:teuthology.orchestra.run.smithi016.stdout:checking for xfs/xqm.h... yes 2023-12-08T08:40:41.560 INFO:teuthology.orchestra.run.smithi016.stdout:checking xfsctl from xfs/xfs.h... ok 2023-12-08T08:40:41.584 INFO:teuthology.orchestra.run.smithi016.stdout:checking xfs/handle.h usability... yes 2023-12-08T08:40:41.592 INFO:teuthology.orchestra.run.smithi016.stdout:checking xfs/handle.h presence... yes 2023-12-08T08:40:41.592 INFO:teuthology.orchestra.run.smithi016.stdout:checking for xfs/handle.h... yes 2023-12-08T08:40:41.628 INFO:teuthology.orchestra.run.smithi016.stdout:checking for attr_list_by_handle in -lhandle... yes 2023-12-08T08:40:41.651 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/acl.h usability... yes 2023-12-08T08:40:41.661 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/acl.h presence... yes 2023-12-08T08:40:41.661 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/acl.h... yes 2023-12-08T08:40:41.686 INFO:teuthology.orchestra.run.smithi016.stdout:checking acl/libacl.h usability... yes 2023-12-08T08:40:41.696 INFO:teuthology.orchestra.run.smithi016.stdout:checking acl/libacl.h presence... yes 2023-12-08T08:40:41.696 INFO:teuthology.orchestra.run.smithi016.stdout:checking for acl/libacl.h... yes 2023-12-08T08:40:41.729 INFO:teuthology.orchestra.run.smithi016.stdout:checking for acl_init in -lacl... yes 2023-12-08T08:40:41.752 INFO:teuthology.orchestra.run.smithi016.stdout:checking gdbm-ndbm.h usability... no 2023-12-08T08:40:41.763 INFO:teuthology.orchestra.run.smithi016.stdout:checking gdbm-ndbm.h presence... no 2023-12-08T08:40:41.763 INFO:teuthology.orchestra.run.smithi016.stdout:checking for gdbm-ndbm.h... no 2023-12-08T08:40:41.786 INFO:teuthology.orchestra.run.smithi016.stdout:checking gdbm/ndbm.h usability... yes 2023-12-08T08:40:41.796 INFO:teuthology.orchestra.run.smithi016.stdout:checking gdbm/ndbm.h presence... yes 2023-12-08T08:40:41.796 INFO:teuthology.orchestra.run.smithi016.stdout:checking for gdbm/ndbm.h... yes 2023-12-08T08:40:41.831 INFO:teuthology.orchestra.run.smithi016.stdout:checking for dbm_open in -lgdbm... no 2023-12-08T08:40:41.866 INFO:teuthology.orchestra.run.smithi016.stdout:checking for dbm_fetch in -lgdbm... no 2023-12-08T08:40:41.901 INFO:teuthology.orchestra.run.smithi016.stdout:checking for dbm_store in -lgdbm... no 2023-12-08T08:40:41.936 INFO:teuthology.orchestra.run.smithi016.stdout:checking for dbm_close in -lgdbm... no 2023-12-08T08:40:41.976 INFO:teuthology.orchestra.run.smithi016.stdout:checking for dbm_open in -lgdbm_compat... yes 2023-12-08T08:40:42.009 INFO:teuthology.orchestra.run.smithi016.stdout:checking for dbm_fetch in -lgdbm_compat... yes 2023-12-08T08:40:42.041 INFO:teuthology.orchestra.run.smithi016.stdout:checking for dbm_store in -lgdbm_compat... yes 2023-12-08T08:40:42.074 INFO:teuthology.orchestra.run.smithi016.stdout:checking for dbm_close in -lgdbm_compat... yes 2023-12-08T08:40:42.098 INFO:teuthology.orchestra.run.smithi016.stdout:checking libaio.h usability... yes 2023-12-08T08:40:42.110 INFO:teuthology.orchestra.run.smithi016.stdout:checking libaio.h presence... yes 2023-12-08T08:40:42.110 INFO:teuthology.orchestra.run.smithi016.stdout:checking for libaio.h... yes 2023-12-08T08:40:42.133 INFO:teuthology.orchestra.run.smithi016.stdout:checking liburing.h usability... no 2023-12-08T08:40:42.143 INFO:teuthology.orchestra.run.smithi016.stdout:checking liburing.h presence... no 2023-12-08T08:40:42.143 INFO:teuthology.orchestra.run.smithi016.stdout:checking for liburing.h... no 2023-12-08T08:40:42.167 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/capability.h usability... yes 2023-12-08T08:40:42.178 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/capability.h presence... yes 2023-12-08T08:40:42.178 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/capability.h... yes 2023-12-08T08:40:42.202 INFO:teuthology.orchestra.run.smithi016.stdout:checking linux/fiemap.h usability... yes 2023-12-08T08:40:42.211 INFO:teuthology.orchestra.run.smithi016.stdout:checking linux/fiemap.h presence... yes 2023-12-08T08:40:42.211 INFO:teuthology.orchestra.run.smithi016.stdout:checking for linux/fiemap.h... yes 2023-12-08T08:40:42.245 INFO:teuthology.orchestra.run.smithi016.stdout:checking for fallocate... yes 2023-12-08T08:40:42.279 INFO:teuthology.orchestra.run.smithi016.stdout:checking for open_by_handle_at... yes 2023-12-08T08:40:42.304 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/prctl.h usability... yes 2023-12-08T08:40:42.313 INFO:teuthology.orchestra.run.smithi016.stdout:checking sys/prctl.h presence... yes 2023-12-08T08:40:42.313 INFO:teuthology.orchestra.run.smithi016.stdout:checking for sys/prctl.h... yes 2023-12-08T08:40:42.338 INFO:teuthology.orchestra.run.smithi016.stdout:checking linux/fs.h usability... yes 2023-12-08T08:40:42.346 INFO:teuthology.orchestra.run.smithi016.stdout:checking linux/fs.h presence... yes 2023-12-08T08:40:42.346 INFO:teuthology.orchestra.run.smithi016.stdout:checking for linux/fs.h... yes 2023-12-08T08:40:42.368 INFO:teuthology.orchestra.run.smithi016.stdout:checking btrfsutil.h usability... no 2023-12-08T08:40:42.379 INFO:teuthology.orchestra.run.smithi016.stdout:checking btrfsutil.h presence... no 2023-12-08T08:40:42.379 INFO:teuthology.orchestra.run.smithi016.stdout:checking for btrfsutil.h... no 2023-12-08T08:40:42.424 INFO:teuthology.orchestra.run.smithi016.stdout:checking for copy_file_range... yes 2023-12-08T08:40:42.461 INFO:teuthology.orchestra.run.smithi016.stdout:checking for SEEK_DATA... yes 2023-12-08T08:40:42.511 INFO:teuthology.orchestra.run.smithi016.stdout:checking for BMV_OF_SHARED... yes 2023-12-08T08:40:42.549 INFO:teuthology.orchestra.run.smithi016.stdout:checking for nftw... yes 2023-12-08T08:40:42.588 INFO:teuthology.orchestra.run.smithi016.stdout:checking for RLIMIT_NOFILE... yes 2023-12-08T08:40:42.607 INFO:teuthology.orchestra.run.smithi016.stdout:checking for FIEXCHANGE_RANGE... no 2023-12-08T08:40:42.641 INFO:teuthology.orchestra.run.smithi016.stdout:checking for renameat2... yes 2023-12-08T08:40:42.675 INFO:teuthology.orchestra.run.smithi016.stdout:checking for reallocarray... yes 2023-12-08T08:40:42.706 INFO:teuthology.orchestra.run.smithi016.stdout:checking for struct mount_attr... yes 2023-12-08T08:40:42.738 INFO:teuthology.orchestra.run.smithi016.stdout:checking for struct btrfs_qgroup_limit... yes 2023-12-08T08:40:42.771 INFO:teuthology.orchestra.run.smithi016.stdout:checking for struct btrfs_qgroup_inherit... yes 2023-12-08T08:40:42.803 INFO:teuthology.orchestra.run.smithi016.stdout:checking for struct btrfs_ioctl_vol_args... yes 2023-12-08T08:40:42.836 INFO:teuthology.orchestra.run.smithi016.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2023-12-08T08:40:42.869 INFO:teuthology.orchestra.run.smithi016.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2023-12-08T08:40:42.901 INFO:teuthology.orchestra.run.smithi016.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2023-12-08T08:40:42.934 INFO:teuthology.orchestra.run.smithi016.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2023-12-08T08:40:42.962 INFO:teuthology.orchestra.run.smithi016.stdout:checking linux/btrfs.h usability... yes 2023-12-08T08:40:42.971 INFO:teuthology.orchestra.run.smithi016.stdout:checking linux/btrfs.h presence... yes 2023-12-08T08:40:42.971 INFO:teuthology.orchestra.run.smithi016.stdout:checking for linux/btrfs.h... yes 2023-12-08T08:40:43.001 INFO:teuthology.orchestra.run.smithi016.stdout:checking linux/btrfs_tree.h usability... yes 2023-12-08T08:40:43.010 INFO:teuthology.orchestra.run.smithi016.stdout:checking linux/btrfs_tree.h presence... yes 2023-12-08T08:40:43.010 INFO:teuthology.orchestra.run.smithi016.stdout:checking for linux/btrfs_tree.h... yes 2023-12-08T08:40:43.031 INFO:teuthology.orchestra.run.smithi016.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2023-12-08T08:40:43.047 INFO:teuthology.orchestra.run.smithi016.stdout:configure: creating ./config.status 2023-12-08T08:40:43.351 INFO:teuthology.orchestra.run.smithi016.stdout:config.status: creating include/builddefs 2023-12-08T08:40:43.362 INFO:teuthology.orchestra.run.smithi016.stdout:config.status: creating include/config.h 2023-12-08T08:40:43.370 INFO:teuthology.orchestra.run.smithi016.stdout:config.status: executing libtool commands 2023-12-08T08:40:43.409 INFO:teuthology.orchestra.run.smithi016.stdout:Building include 2023-12-08T08:40:43.412 INFO:teuthology.orchestra.run.smithi016.stdout:Building lib 2023-12-08T08:40:43.415 INFO:teuthology.orchestra.run.smithi016.stdout: [LTDEP] 2023-12-08T08:40:43.553 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] dataascii.lo 2023-12-08T08:40:43.662 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] databin.lo 2023-12-08T08:40:43.798 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] datapid.lo 2023-12-08T08:40:43.898 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] file_lock.lo 2023-12-08T08:40:44.038 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] forker.lo 2023-12-08T08:40:44.161 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] pattern.lo 2023-12-08T08:40:44.274 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] open_flags.lo 2023-12-08T08:40:44.404 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] random_range.lo 2023-12-08T08:40:44.621 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] string_to_tokens.lo 2023-12-08T08:40:44.715 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] str_to_bytes.lo 2023-12-08T08:40:44.840 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] tlibio.lo 2023-12-08T08:40:45.068 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] write_log.lo 2023-12-08T08:40:45.291 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] random.lo 2023-12-08T08:40:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: [LD] 2023-12-08T08:40:45.468 INFO:teuthology.orchestra.run.smithi016.stdout:Building ltp 2023-12-08T08:40:45.471 INFO:teuthology.orchestra.run.smithi016.stdout: [DEP] 2023-12-08T08:40:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] doio 2023-12-08T08:40:46.258 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] fsstress 2023-12-08T08:40:47.642 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] fsx 2023-12-08T08:40:48.331 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] growfiles 2023-12-08T08:40:48.796 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] iogen 2023-12-08T08:40:49.145 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-stress 2023-12-08T08:40:49.541 INFO:teuthology.orchestra.run.smithi016.stdout:Building src 2023-12-08T08:40:49.547 INFO:teuthology.orchestra.run.smithi016.stdout: [DEP] 2023-12-08T08:40:51.287 INFO:teuthology.orchestra.run.smithi016.stdout: [DEP] 2023-12-08T08:40:51.317 INFO:teuthology.orchestra.run.smithi016.stdout: [DEP] 2023-12-08T08:40:51.324 INFO:teuthology.orchestra.run.smithi016.stdout: [DEP] 2023-12-08T08:40:51.540 INFO:teuthology.orchestra.run.smithi016.stdout: [DEP] 2023-12-08T08:40:51.561 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] dirstress 2023-12-08T08:40:51.774 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] fill 2023-12-08T08:40:51.908 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] fill2 2023-12-08T08:40:52.113 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] getpagesize 2023-12-08T08:40:52.230 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] holes 2023-12-08T08:40:52.404 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] lstat64 2023-12-08T08:40:52.565 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] nametest 2023-12-08T08:40:52.781 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] permname 2023-12-08T08:40:52.960 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] randholes 2023-12-08T08:40:53.183 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] runas 2023-12-08T08:40:53.337 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] truncfile 2023-12-08T08:40:53.495 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] usemem 2023-12-08T08:40:53.626 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] mmapcat 2023-12-08T08:40:53.755 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] append_reader 2023-12-08T08:40:53.886 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] append_writer 2023-12-08T08:40:54.018 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] dirperf 2023-12-08T08:40:54.218 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] metaperf 2023-12-08T08:40:54.471 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] devzero 2023-12-08T08:40:54.623 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] feature 2023-12-08T08:40:54.815 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] alloc 2023-12-08T08:40:55.001 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] fault 2023-12-08T08:40:55.142 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] fstest 2023-12-08T08:40:55.356 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_access_root 2023-12-08T08:40:55.490 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] godown 2023-12-08T08:40:55.639 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] resvtest 2023-12-08T08:40:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] writemod 2023-12-08T08:40:55.927 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] writev_on_pagefault 2023-12-08T08:40:56.067 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] makeextents 2023-12-08T08:40:56.223 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] itrash 2023-12-08T08:40:56.354 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] rename 2023-12-08T08:40:56.478 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] multi_open_unlink 2023-12-08T08:40:56.624 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] unwritten_sync 2023-12-08T08:40:56.783 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] genhashnames 2023-12-08T08:40:56.953 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_holes 2023-12-08T08:40:57.080 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_mmap_writev 2023-12-08T08:40:57.208 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_truncate_cmtime 2023-12-08T08:40:57.347 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] dirhash_collide 2023-12-08T08:40:57.508 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_rename_overwrite 2023-12-08T08:40:57.630 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] holetest 2023-12-08T08:40:57.834 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_truncate_self 2023-12-08T08:40:57.956 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] af_unix 2023-12-08T08:40:58.087 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_mmap_stale_pmd 2023-12-08T08:40:58.220 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_mmap_cow_race 2023-12-08T08:40:58.365 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_mmap_fallocate 2023-12-08T08:40:58.496 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] fsync-err 2023-12-08T08:40:58.660 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_mmap_write_ro 2023-12-08T08:40:58.792 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_ext4_dax_journal_corruption 2023-12-08T08:40:58.932 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_ext4_dax_inline_corruption 2023-12-08T08:40:59.067 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_ofd_locks 2023-12-08T08:40:59.241 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_mmap_collision 2023-12-08T08:40:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] mmap-write-concurrent 2023-12-08T08:40:59.565 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_get_file_time 2023-12-08T08:40:59.693 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_create_short_dirs 2023-12-08T08:40:59.842 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_create_long_dirs 2023-12-08T08:40:59.987 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_enospc 2023-12-08T08:41:00.153 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_mmap_writev_overlap 2023-12-08T08:41:00.302 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] checkpoint_journal 2023-12-08T08:41:00.437 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] mmap-rw-fault 2023-12-08T08:41:00.600 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] allocstale 2023-12-08T08:41:00.749 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_mmap_cow_memory_failure 2023-12-08T08:41:00.900 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] fake-dump-rootino 2023-12-08T08:41:01.047 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] dio-buf-fault 2023-12-08T08:41:01.181 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] rewinddir-test 2023-12-08T08:41:01.331 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] xfsctl 2023-12-08T08:41:01.497 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] bstat 2023-12-08T08:41:01.684 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_mtab 2023-12-08T08:41:01.851 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] getdevicesize 2023-12-08T08:41:01.975 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] preallo_rw_pattern_reader 2023-12-08T08:41:02.113 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] preallo_rw_pattern_writer 2023-12-08T08:41:02.251 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] ftrunc 2023-12-08T08:41:02.379 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] trunc 2023-12-08T08:41:02.522 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] fs_perms 2023-12-08T08:41:02.666 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] testx 2023-12-08T08:41:02.781 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] looptest 2023-12-08T08:41:02.952 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] locktest 2023-12-08T08:41:03.232 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] unwritten_mmap 2023-12-08T08:41:03.375 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] bulkstat_unlink_test 2023-12-08T08:41:03.553 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] deduperace 2023-12-08T08:41:03.725 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] bulkstat_unlink_test_modified 2023-12-08T08:41:03.899 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_dir_offset 2023-12-08T08:41:04.028 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_futimens 2023-12-08T08:41:04.148 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_immutable 2023-12-08T08:41:04.731 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] stale_handle 2023-12-08T08:41:04.881 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] pwrite_mmap_blocked 2023-12-08T08:41:05.014 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_dir_offset2 2023-12-08T08:41:05.184 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] seek_sanity_test 2023-12-08T08:41:05.595 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] seek_copy_test 2023-12-08T08:41:05.751 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_readdir_1 2023-12-08T08:41:05.880 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_readdir_2 2023-12-08T08:41:06.012 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] fsync-tester 2023-12-08T08:41:06.242 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] nsexec 2023-12-08T08:41:06.396 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] cloner 2023-12-08T08:41:06.553 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] renameat2 2023-12-08T08:41:06.700 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_getcwd 2023-12-08T08:41:06.839 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] e4compact 2023-12-08T08:41:07.019 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] test-nextquota 2023-12-08T08:41:07.170 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] punch-alternating 2023-12-08T08:41:07.320 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] attr-list-by-handle-cursor-test 2023-12-08T08:41:07.467 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] listxattr 2023-12-08T08:41:07.597 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] dio-interleaved 2023-12-08T08:41:07.737 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_dir_type 2023-12-08T08:41:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] dio-invalidate-cache 2023-12-08T08:41:08.059 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] stat_test 2023-12-08T08:41:08.327 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_encrypted_d_revalidate 2023-12-08T08:41:08.472 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] attr_replace_test 2023-12-08T08:41:08.613 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] swapon 2023-12-08T08:41:08.758 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] mkswap 2023-12-08T08:41:08.891 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_attr_corruption 2023-12-08T08:41:09.022 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_open_tmpfiles 2023-12-08T08:41:09.181 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] fscrypt-crypt-util 2023-12-08T08:41:09.802 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] bulkstat_null_ocount 2023-12-08T08:41:09.940 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] splice-test 2023-12-08T08:41:10.100 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] chprojid_fail 2023-12-08T08:41:10.243 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] detached_mounts_propagation 2023-12-08T08:41:10.411 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] ext4_resize 2023-12-08T08:41:10.542 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_readdir_3 2023-12-08T08:41:10.715 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] splice2pipe 2023-12-08T08:41:10.857 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] uuid_ioctl 2023-12-08T08:41:10.993 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] fiemap-tester 2023-12-08T08:41:11.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:41:11.143 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:41:11.237 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_stripealign 2023-12-08T08:41:11.373 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] open_by_handle 2023-12-08T08:41:11.601 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] dbtest 2023-12-08T08:41:11.827 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] t_mmap_dio 2023-12-08T08:41:11.968 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] xfsfind 2023-12-08T08:41:12.142 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] fssum 2023-12-08T08:41:12.552 INFO:teuthology.orchestra.run.smithi016.stdout:Building log-writes 2023-12-08T08:41:12.557 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] replay-log 2023-12-08T08:41:12.846 INFO:teuthology.orchestra.run.smithi016.stdout:Building perf 2023-12-08T08:41:12.849 INFO:teuthology.orchestra.run.smithi016.stdout:Building aio-dio-regress 2023-12-08T08:41:12.855 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aiocp 2023-12-08T08:41:13.059 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-dio-append-write-fallocate-race 2023-12-08T08:41:13.208 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-dio-append-write-read-race 2023-12-08T08:41:13.363 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-dio-cow-race 2023-12-08T08:41:13.496 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-dio-cycle-write 2023-12-08T08:41:13.661 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-dio-eof-race 2023-12-08T08:41:13.826 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-dio-extend-stat 2023-12-08T08:41:13.967 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-dio-fcntl-race 2023-12-08T08:41:14.111 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-dio-hole-filling-race 2023-12-08T08:41:14.241 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-dio-invalidate-failure 2023-12-08T08:41:14.381 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-dio-invalidate-readahead 2023-12-08T08:41:14.527 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aiodio_sparse2 2023-12-08T08:41:14.725 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-dio-subblock-eof-read 2023-12-08T08:41:14.851 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-dio-write-verify 2023-12-08T08:41:15.023 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2023-12-08T08:41:15.140 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2023-12-08T08:41:15.254 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] aio-last-ref-held-by-io 2023-12-08T08:41:15.409 INFO:teuthology.orchestra.run.smithi016.stdout:Building vfs 2023-12-08T08:41:15.413 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] vfstest 2023-12-08T08:41:18.852 INFO:teuthology.orchestra.run.smithi016.stdout: [CC] mount-idmapped 2023-12-08T08:41:19.211 INFO:teuthology.orchestra.run.smithi016.stdout:Building m4 2023-12-08T08:41:19.215 INFO:teuthology.orchestra.run.smithi016.stdout:Building common 2023-12-08T08:41:19.218 INFO:teuthology.orchestra.run.smithi016.stdout:Building tests 2023-12-08T08:41:19.224 INFO:teuthology.orchestra.run.smithi016.stdout:Building btrfs 2023-12-08T08:41:19.229 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/btrfs/group.list 2023-12-08T08:41:19.290 INFO:teuthology.orchestra.run.smithi016.stdout:Building ceph 2023-12-08T08:41:19.295 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/ceph/group.list 2023-12-08T08:41:19.309 INFO:teuthology.orchestra.run.smithi016.stdout:Building cifs 2023-12-08T08:41:19.314 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/cifs/group.list 2023-12-08T08:41:19.326 INFO:teuthology.orchestra.run.smithi016.stdout:Building ext4 2023-12-08T08:41:19.332 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/ext4/group.list 2023-12-08T08:41:19.370 INFO:teuthology.orchestra.run.smithi016.stdout:Building f2fs 2023-12-08T08:41:19.375 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/f2fs/group.list 2023-12-08T08:41:19.392 INFO:teuthology.orchestra.run.smithi016.stdout:Building generic 2023-12-08T08:41:19.399 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/generic/group.list 2023-12-08T08:41:19.498 INFO:teuthology.orchestra.run.smithi016.stdout:Building nfs 2023-12-08T08:41:19.503 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/nfs/group.list 2023-12-08T08:41:19.517 INFO:teuthology.orchestra.run.smithi016.stdout:Building ocfs2 2023-12-08T08:41:19.522 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/ocfs2/group.list 2023-12-08T08:41:19.536 INFO:teuthology.orchestra.run.smithi016.stdout:Building overlay 2023-12-08T08:41:19.541 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/overlay/group.list 2023-12-08T08:41:19.582 INFO:teuthology.orchestra.run.smithi016.stdout:Building perf 2023-12-08T08:41:19.587 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/perf/group.list 2023-12-08T08:41:19.598 INFO:teuthology.orchestra.run.smithi016.stdout:Building selftest 2023-12-08T08:41:19.603 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/selftest/group.list 2023-12-08T08:41:19.616 INFO:teuthology.orchestra.run.smithi016.stdout:Building shared 2023-12-08T08:41:19.621 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/shared/group.list 2023-12-08T08:41:19.638 INFO:teuthology.orchestra.run.smithi016.stdout:Building tmpfs 2023-12-08T08:41:19.643 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/tmpfs/group.list 2023-12-08T08:41:19.657 INFO:teuthology.orchestra.run.smithi016.stdout:Building udf 2023-12-08T08:41:19.662 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/udf/group.list 2023-12-08T08:41:19.674 INFO:teuthology.orchestra.run.smithi016.stdout:Building xfs 2023-12-08T08:41:19.680 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/xfs/group.list 2023-12-08T08:41:19.780 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo make install) 2023-12-08T08:41:19.863 INFO:teuthology.orchestra.run.smithi016.stdout:Building include 2023-12-08T08:41:19.866 INFO:teuthology.orchestra.run.smithi016.stdout:Building lib 2023-12-08T08:41:19.870 INFO:teuthology.orchestra.run.smithi016.stdout:Building ltp 2023-12-08T08:41:19.874 INFO:teuthology.orchestra.run.smithi016.stdout:Building src 2023-12-08T08:41:19.908 INFO:teuthology.orchestra.run.smithi016.stdout:Building log-writes 2023-12-08T08:41:19.911 INFO:teuthology.orchestra.run.smithi016.stdout:Building perf 2023-12-08T08:41:19.915 INFO:teuthology.orchestra.run.smithi016.stdout:Building aio-dio-regress 2023-12-08T08:41:19.919 INFO:teuthology.orchestra.run.smithi016.stdout:Building vfs 2023-12-08T08:41:19.923 INFO:teuthology.orchestra.run.smithi016.stdout:Building m4 2023-12-08T08:41:19.926 INFO:teuthology.orchestra.run.smithi016.stdout:Building common 2023-12-08T08:41:19.930 INFO:teuthology.orchestra.run.smithi016.stdout:Building tests 2023-12-08T08:41:19.935 INFO:teuthology.orchestra.run.smithi016.stdout:Building btrfs 2023-12-08T08:41:19.941 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/btrfs/group.list 2023-12-08T08:41:20.000 INFO:teuthology.orchestra.run.smithi016.stdout:Building ceph 2023-12-08T08:41:20.005 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/ceph/group.list 2023-12-08T08:41:20.020 INFO:teuthology.orchestra.run.smithi016.stdout:Building cifs 2023-12-08T08:41:20.025 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/cifs/group.list 2023-12-08T08:41:20.037 INFO:teuthology.orchestra.run.smithi016.stdout:Building ext4 2023-12-08T08:41:20.043 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/ext4/group.list 2023-12-08T08:41:20.081 INFO:teuthology.orchestra.run.smithi016.stdout:Building f2fs 2023-12-08T08:41:20.086 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/f2fs/group.list 2023-12-08T08:41:20.103 INFO:teuthology.orchestra.run.smithi016.stdout:Building generic 2023-12-08T08:41:20.109 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/generic/group.list 2023-12-08T08:41:20.206 INFO:teuthology.orchestra.run.smithi016.stdout:Building nfs 2023-12-08T08:41:20.211 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/nfs/group.list 2023-12-08T08:41:20.226 INFO:teuthology.orchestra.run.smithi016.stdout:Building ocfs2 2023-12-08T08:41:20.231 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/ocfs2/group.list 2023-12-08T08:41:20.244 INFO:teuthology.orchestra.run.smithi016.stdout:Building overlay 2023-12-08T08:41:20.249 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/overlay/group.list 2023-12-08T08:41:20.290 INFO:teuthology.orchestra.run.smithi016.stdout:Building perf 2023-12-08T08:41:20.295 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/perf/group.list 2023-12-08T08:41:20.307 INFO:teuthology.orchestra.run.smithi016.stdout:Building selftest 2023-12-08T08:41:20.312 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/selftest/group.list 2023-12-08T08:41:20.324 INFO:teuthology.orchestra.run.smithi016.stdout:Building shared 2023-12-08T08:41:20.329 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/shared/group.list 2023-12-08T08:41:20.347 INFO:teuthology.orchestra.run.smithi016.stdout:Building tmpfs 2023-12-08T08:41:20.352 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/tmpfs/group.list 2023-12-08T08:41:20.365 INFO:teuthology.orchestra.run.smithi016.stdout:Building udf 2023-12-08T08:41:20.370 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/udf/group.list 2023-12-08T08:41:20.382 INFO:teuthology.orchestra.run.smithi016.stdout:Building xfs 2023-12-08T08:41:20.388 INFO:teuthology.orchestra.run.smithi016.stdout: [GROUP] /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/xfs/group.list 2023-12-08T08:41:20.485 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2023-12-08T08:41:20.487 INFO:teuthology.orchestra.run.smithi016.stdout:gmake[1]: Nothing to be done for 'install'. 2023-12-08T08:41:20.487 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2023-12-08T08:41:20.489 INFO:teuthology.orchestra.run.smithi016.stdout:gmake[1]: Nothing to be done for 'install'. 2023-12-08T08:41:20.489 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2023-12-08T08:41:20.491 INFO:teuthology.orchestra.run.smithi016.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2023-12-08T08:41:20.500 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2023-12-08T08:41:20.569 INFO:teuthology.orchestra.run.smithi016.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2023-12-08T08:41:20.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2023-12-08T08:41:20.594 INFO:teuthology.orchestra.run.smithi016.stdout:Building log-writes 2023-12-08T08:41:20.598 INFO:teuthology.orchestra.run.smithi016.stdout:Building perf 2023-12-08T08:41:20.601 INFO:teuthology.orchestra.run.smithi016.stdout:Building aio-dio-regress 2023-12-08T08:41:20.605 INFO:teuthology.orchestra.run.smithi016.stdout:Building vfs 2023-12-08T08:41:20.608 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake -C log-writes install 2023-12-08T08:41:20.610 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2023-12-08T08:41:20.618 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2023-12-08T08:41:20.627 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake -C perf install 2023-12-08T08:41:20.628 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2023-12-08T08:41:20.637 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2023-12-08T08:41:20.658 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake -C aio-dio-regress install 2023-12-08T08:41:20.660 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2023-12-08T08:41:20.668 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2023-12-08T08:41:20.717 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake -C vfs install 2023-12-08T08:41:20.719 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2023-12-08T08:41:20.727 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2023-12-08T08:41:20.739 INFO:teuthology.orchestra.run.smithi016.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2023-12-08T08:41:20.747 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl fiemap-tester t_stripealign open_by_handle dbtest t_mmap_dio xfsfind fssum /var/lib/xfstests/src 2023-12-08T08:41:21.888 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2023-12-08T08:41:21.982 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2023-12-08T08:41:22.007 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2023-12-08T08:41:22.009 INFO:teuthology.orchestra.run.smithi016.stdout:gmake[1]: Nothing to be done for 'install'. 2023-12-08T08:41:22.009 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2023-12-08T08:41:22.010 INFO:teuthology.orchestra.run.smithi016.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2023-12-08T08:41:22.019 INFO:teuthology.orchestra.run.smithi016.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2023-12-08T08:41:22.144 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2023-12-08T08:41:22.146 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/btrfs/ install 2023-12-08T08:41:22.149 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2023-12-08T08:41:22.158 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 /var/lib/xfstests/tests/btrfs 2023-12-08T08:41:22.922 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2023-12-08T08:41:22.931 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out /var/lib/xfstests/tests/btrfs 2023-12-08T08:41:23.697 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/ceph/ install 2023-12-08T08:41:23.699 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2023-12-08T08:41:23.707 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2023-12-08T08:41:23.726 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2023-12-08T08:41:23.734 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2023-12-08T08:41:23.753 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/cifs/ install 2023-12-08T08:41:23.754 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2023-12-08T08:41:23.763 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2023-12-08T08:41:23.772 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2023-12-08T08:41:23.780 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2023-12-08T08:41:23.789 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/ext4/ install 2023-12-08T08:41:23.791 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2023-12-08T08:41:23.800 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2023-12-08T08:41:23.976 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2023-12-08T08:41:23.984 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2023-12-08T08:41:24.168 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/f2fs/ install 2023-12-08T08:41:24.170 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2023-12-08T08:41:24.178 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2023-12-08T08:41:24.189 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2023-12-08T08:41:24.198 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2023-12-08T08:41:24.209 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/generic/ install 2023-12-08T08:41:24.214 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2023-12-08T08:41:24.223 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 /var/lib/xfstests/tests/generic 2023-12-08T08:41:26.080 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2023-12-08T08:41:26.088 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out /var/lib/xfstests/tests/generic 2023-12-08T08:41:27.963 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/nfs/ install 2023-12-08T08:41:27.964 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2023-12-08T08:41:27.972 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2023-12-08T08:41:27.981 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2023-12-08T08:41:27.990 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2023-12-08T08:41:27.998 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/ocfs2/ install 2023-12-08T08:41:28.000 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2023-12-08T08:41:28.009 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2023-12-08T08:41:28.017 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2023-12-08T08:41:28.026 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2023-12-08T08:41:28.035 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/overlay/ install 2023-12-08T08:41:28.037 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2023-12-08T08:41:28.045 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2023-12-08T08:41:28.304 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2023-12-08T08:41:28.313 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2023-12-08T08:41:28.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/perf/ install 2023-12-08T08:41:28.574 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2023-12-08T08:41:28.583 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2023-12-08T08:41:28.593 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2023-12-08T08:41:28.600 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2023-12-08T08:41:28.609 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/selftest/ install 2023-12-08T08:41:28.610 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2023-12-08T08:41:28.619 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2023-12-08T08:41:28.640 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2023-12-08T08:41:28.648 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2023-12-08T08:41:28.669 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/shared/ install 2023-12-08T08:41:28.671 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2023-12-08T08:41:28.679 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2023-12-08T08:41:28.693 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2023-12-08T08:41:28.702 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2023-12-08T08:41:28.715 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/tmpfs/ install 2023-12-08T08:41:28.717 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2023-12-08T08:41:28.725 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2023-12-08T08:41:28.734 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2023-12-08T08:41:28.742 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2023-12-08T08:41:28.751 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/udf/ install 2023-12-08T08:41:28.753 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2023-12-08T08:41:28.761 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2023-12-08T08:41:28.770 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2023-12-08T08:41:28.778 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2023-12-08T08:41:28.787 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.MkHH5XcxMMxfstests-dev/tests/xfs/ install 2023-12-08T08:41:28.792 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2023-12-08T08:41:28.801 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2023-12-08T08:41:30.555 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2023-12-08T08:41:30.564 INFO:teuthology.orchestra.run.smithi016.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2023-12-08T08:41:32.343 INFO:teuthology.orchestra.run.smithi016.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2023-12-08T08:41:32.351 INFO:teuthology.orchestra.run.smithi016.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2023-12-08T08:41:32.360 INFO:teuthology.orchestra.run.smithi016.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2023-12-08T08:41:32.368 INFO:teuthology.orchestra.run.smithi016.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2023-12-08T08:41:32.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T08:41:32.379 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec ls tests/generic) 2023-12-08T08:41:32.436 INFO:teuthology.orchestra.run.smithi016.stdout:001 2023-12-08T08:41:32.436 INFO:teuthology.orchestra.run.smithi016.stdout:001.out 2023-12-08T08:41:32.436 INFO:teuthology.orchestra.run.smithi016.stdout:002 2023-12-08T08:41:32.436 INFO:teuthology.orchestra.run.smithi016.stdout:002.out 2023-12-08T08:41:32.436 INFO:teuthology.orchestra.run.smithi016.stdout:003 2023-12-08T08:41:32.436 INFO:teuthology.orchestra.run.smithi016.stdout:003.out 2023-12-08T08:41:32.436 INFO:teuthology.orchestra.run.smithi016.stdout:004 2023-12-08T08:41:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:004.out 2023-12-08T08:41:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:005 2023-12-08T08:41:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:005.out 2023-12-08T08:41:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:006 2023-12-08T08:41:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:006.out 2023-12-08T08:41:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:007 2023-12-08T08:41:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:007.out 2023-12-08T08:41:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:008 2023-12-08T08:41:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:008.out 2023-12-08T08:41:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:009 2023-12-08T08:41:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:009.out 2023-12-08T08:41:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:010 2023-12-08T08:41:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:010.out 2023-12-08T08:41:32.437 INFO:teuthology.orchestra.run.smithi016.stdout:011 2023-12-08T08:41:32.438 INFO:teuthology.orchestra.run.smithi016.stdout:011.out 2023-12-08T08:41:32.438 INFO:teuthology.orchestra.run.smithi016.stdout:012 2023-12-08T08:41:32.438 INFO:teuthology.orchestra.run.smithi016.stdout:012.out 2023-12-08T08:41:32.438 INFO:teuthology.orchestra.run.smithi016.stdout:013 2023-12-08T08:41:32.438 INFO:teuthology.orchestra.run.smithi016.stdout:013.out 2023-12-08T08:41:32.438 INFO:teuthology.orchestra.run.smithi016.stdout:014 2023-12-08T08:41:32.438 INFO:teuthology.orchestra.run.smithi016.stdout:014.out 2023-12-08T08:41:32.438 INFO:teuthology.orchestra.run.smithi016.stdout:015 2023-12-08T08:41:32.438 INFO:teuthology.orchestra.run.smithi016.stdout:015.out 2023-12-08T08:41:32.438 INFO:teuthology.orchestra.run.smithi016.stdout:016 2023-12-08T08:41:32.438 INFO:teuthology.orchestra.run.smithi016.stdout:016.out 2023-12-08T08:41:32.438 INFO:teuthology.orchestra.run.smithi016.stdout:017 2023-12-08T08:41:32.438 INFO:teuthology.orchestra.run.smithi016.stdout:017.out 2023-12-08T08:41:32.438 INFO:teuthology.orchestra.run.smithi016.stdout:018 2023-12-08T08:41:32.439 INFO:teuthology.orchestra.run.smithi016.stdout:018.out 2023-12-08T08:41:32.439 INFO:teuthology.orchestra.run.smithi016.stdout:019 2023-12-08T08:41:32.439 INFO:teuthology.orchestra.run.smithi016.stdout:019.out 2023-12-08T08:41:32.439 INFO:teuthology.orchestra.run.smithi016.stdout:020 2023-12-08T08:41:32.439 INFO:teuthology.orchestra.run.smithi016.stdout:020.out 2023-12-08T08:41:32.439 INFO:teuthology.orchestra.run.smithi016.stdout:021 2023-12-08T08:41:32.439 INFO:teuthology.orchestra.run.smithi016.stdout:021.out 2023-12-08T08:41:32.439 INFO:teuthology.orchestra.run.smithi016.stdout:022 2023-12-08T08:41:32.439 INFO:teuthology.orchestra.run.smithi016.stdout:022.out 2023-12-08T08:41:32.439 INFO:teuthology.orchestra.run.smithi016.stdout:023 2023-12-08T08:41:32.439 INFO:teuthology.orchestra.run.smithi016.stdout:023.out 2023-12-08T08:41:32.439 INFO:teuthology.orchestra.run.smithi016.stdout:024 2023-12-08T08:41:32.439 INFO:teuthology.orchestra.run.smithi016.stdout:024.out 2023-12-08T08:41:32.440 INFO:teuthology.orchestra.run.smithi016.stdout:025 2023-12-08T08:41:32.440 INFO:teuthology.orchestra.run.smithi016.stdout:025.out 2023-12-08T08:41:32.440 INFO:teuthology.orchestra.run.smithi016.stdout:026 2023-12-08T08:41:32.440 INFO:teuthology.orchestra.run.smithi016.stdout:026.out 2023-12-08T08:41:32.440 INFO:teuthology.orchestra.run.smithi016.stdout:027 2023-12-08T08:41:32.440 INFO:teuthology.orchestra.run.smithi016.stdout:027.out 2023-12-08T08:41:32.440 INFO:teuthology.orchestra.run.smithi016.stdout:028 2023-12-08T08:41:32.440 INFO:teuthology.orchestra.run.smithi016.stdout:028.out 2023-12-08T08:41:32.440 INFO:teuthology.orchestra.run.smithi016.stdout:029 2023-12-08T08:41:32.440 INFO:teuthology.orchestra.run.smithi016.stdout:029.out 2023-12-08T08:41:32.440 INFO:teuthology.orchestra.run.smithi016.stdout:030 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:030.out 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:031 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:031.out 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:032 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:032.out 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:033 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:033.out 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:034 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:034.out 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:035 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:035.cfg 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:035.out.default 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:035.out.nfs 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:036 2023-12-08T08:41:32.441 INFO:teuthology.orchestra.run.smithi016.stdout:036.out 2023-12-08T08:41:32.442 INFO:teuthology.orchestra.run.smithi016.stdout:037 2023-12-08T08:41:32.442 INFO:teuthology.orchestra.run.smithi016.stdout:037.out 2023-12-08T08:41:32.442 INFO:teuthology.orchestra.run.smithi016.stdout:038 2023-12-08T08:41:32.442 INFO:teuthology.orchestra.run.smithi016.stdout:038.out 2023-12-08T08:41:32.442 INFO:teuthology.orchestra.run.smithi016.stdout:039 2023-12-08T08:41:32.442 INFO:teuthology.orchestra.run.smithi016.stdout:039.out 2023-12-08T08:41:32.442 INFO:teuthology.orchestra.run.smithi016.stdout:040 2023-12-08T08:41:32.442 INFO:teuthology.orchestra.run.smithi016.stdout:040.out 2023-12-08T08:41:32.442 INFO:teuthology.orchestra.run.smithi016.stdout:041 2023-12-08T08:41:32.442 INFO:teuthology.orchestra.run.smithi016.stdout:041.out 2023-12-08T08:41:32.442 INFO:teuthology.orchestra.run.smithi016.stdout:042 2023-12-08T08:41:32.442 INFO:teuthology.orchestra.run.smithi016.stdout:042.out 2023-12-08T08:41:32.442 INFO:teuthology.orchestra.run.smithi016.stdout:043 2023-12-08T08:41:32.442 INFO:teuthology.orchestra.run.smithi016.stdout:043.out 2023-12-08T08:41:32.443 INFO:teuthology.orchestra.run.smithi016.stdout:044 2023-12-08T08:41:32.443 INFO:teuthology.orchestra.run.smithi016.stdout:044.out 2023-12-08T08:41:32.443 INFO:teuthology.orchestra.run.smithi016.stdout:045 2023-12-08T08:41:32.443 INFO:teuthology.orchestra.run.smithi016.stdout:045.out 2023-12-08T08:41:32.443 INFO:teuthology.orchestra.run.smithi016.stdout:046 2023-12-08T08:41:32.443 INFO:teuthology.orchestra.run.smithi016.stdout:046.out 2023-12-08T08:41:32.443 INFO:teuthology.orchestra.run.smithi016.stdout:047 2023-12-08T08:41:32.443 INFO:teuthology.orchestra.run.smithi016.stdout:047.out 2023-12-08T08:41:32.443 INFO:teuthology.orchestra.run.smithi016.stdout:048 2023-12-08T08:41:32.443 INFO:teuthology.orchestra.run.smithi016.stdout:048.out 2023-12-08T08:41:32.443 INFO:teuthology.orchestra.run.smithi016.stdout:049 2023-12-08T08:41:32.443 INFO:teuthology.orchestra.run.smithi016.stdout:049.out 2023-12-08T08:41:32.443 INFO:teuthology.orchestra.run.smithi016.stdout:050 2023-12-08T08:41:32.443 INFO:teuthology.orchestra.run.smithi016.stdout:050.cfg 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:050.out.default 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:050.out.nojournal 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:050.out.xfsquota 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:051 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:051.out 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:052 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:052.out 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:053 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:053.out 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:054 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:054.out 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:055 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:055.out 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:056 2023-12-08T08:41:32.444 INFO:teuthology.orchestra.run.smithi016.stdout:056.out 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:057 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:057.out 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:058 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:058.out 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:059 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:059.out 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:060 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:060.out 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:061 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:061.out 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:062 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:062.out 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:063 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:063.out 2023-12-08T08:41:32.445 INFO:teuthology.orchestra.run.smithi016.stdout:064 2023-12-08T08:41:32.446 INFO:teuthology.orchestra.run.smithi016.stdout:064.out 2023-12-08T08:41:32.446 INFO:teuthology.orchestra.run.smithi016.stdout:065 2023-12-08T08:41:32.446 INFO:teuthology.orchestra.run.smithi016.stdout:065.out 2023-12-08T08:41:32.446 INFO:teuthology.orchestra.run.smithi016.stdout:066 2023-12-08T08:41:32.446 INFO:teuthology.orchestra.run.smithi016.stdout:066.out 2023-12-08T08:41:32.446 INFO:teuthology.orchestra.run.smithi016.stdout:067 2023-12-08T08:41:32.446 INFO:teuthology.orchestra.run.smithi016.stdout:067.out 2023-12-08T08:41:32.446 INFO:teuthology.orchestra.run.smithi016.stdout:068 2023-12-08T08:41:32.446 INFO:teuthology.orchestra.run.smithi016.stdout:068.out 2023-12-08T08:41:32.446 INFO:teuthology.orchestra.run.smithi016.stdout:069 2023-12-08T08:41:32.446 INFO:teuthology.orchestra.run.smithi016.stdout:069.out 2023-12-08T08:41:32.446 INFO:teuthology.orchestra.run.smithi016.stdout:070 2023-12-08T08:41:32.446 INFO:teuthology.orchestra.run.smithi016.stdout:070.out 2023-12-08T08:41:32.446 INFO:teuthology.orchestra.run.smithi016.stdout:071 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:071.out 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:072 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:072.out 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:073 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:073.out 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:074 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:074.out 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:075 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:075.out 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:076 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:076.out 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:077 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:077.out 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:078 2023-12-08T08:41:32.447 INFO:teuthology.orchestra.run.smithi016.stdout:078.out 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:079 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:079.out 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:080 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:080.out 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:081 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:081.out 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:082 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:082.out 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:083 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:083.out 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:084 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:084.out 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:085 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:085.out 2023-12-08T08:41:32.448 INFO:teuthology.orchestra.run.smithi016.stdout:086 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:086.out 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:087 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:087.out 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:088 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:088.out 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:089 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:089.out 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:090 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:090.out 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:091 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:091.out 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:092 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:092.out 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:093 2023-12-08T08:41:32.449 INFO:teuthology.orchestra.run.smithi016.stdout:093.out 2023-12-08T08:41:32.450 INFO:teuthology.orchestra.run.smithi016.stdout:094 2023-12-08T08:41:32.450 INFO:teuthology.orchestra.run.smithi016.stdout:094.out 2023-12-08T08:41:32.450 INFO:teuthology.orchestra.run.smithi016.stdout:095 2023-12-08T08:41:32.450 INFO:teuthology.orchestra.run.smithi016.stdout:095.out 2023-12-08T08:41:32.450 INFO:teuthology.orchestra.run.smithi016.stdout:096 2023-12-08T08:41:32.450 INFO:teuthology.orchestra.run.smithi016.stdout:096.out 2023-12-08T08:41:32.450 INFO:teuthology.orchestra.run.smithi016.stdout:097 2023-12-08T08:41:32.450 INFO:teuthology.orchestra.run.smithi016.stdout:097.out 2023-12-08T08:41:32.450 INFO:teuthology.orchestra.run.smithi016.stdout:098 2023-12-08T08:41:32.450 INFO:teuthology.orchestra.run.smithi016.stdout:098.out 2023-12-08T08:41:32.450 INFO:teuthology.orchestra.run.smithi016.stdout:099 2023-12-08T08:41:32.451 INFO:teuthology.orchestra.run.smithi016.stdout:099.out 2023-12-08T08:41:32.451 INFO:teuthology.orchestra.run.smithi016.stdout:100 2023-12-08T08:41:32.451 INFO:teuthology.orchestra.run.smithi016.stdout:100.out 2023-12-08T08:41:32.451 INFO:teuthology.orchestra.run.smithi016.stdout:101 2023-12-08T08:41:32.451 INFO:teuthology.orchestra.run.smithi016.stdout:101.out 2023-12-08T08:41:32.451 INFO:teuthology.orchestra.run.smithi016.stdout:102 2023-12-08T08:41:32.451 INFO:teuthology.orchestra.run.smithi016.stdout:102.out 2023-12-08T08:41:32.451 INFO:teuthology.orchestra.run.smithi016.stdout:103 2023-12-08T08:41:32.451 INFO:teuthology.orchestra.run.smithi016.stdout:103.out 2023-12-08T08:41:32.451 INFO:teuthology.orchestra.run.smithi016.stdout:104 2023-12-08T08:41:32.451 INFO:teuthology.orchestra.run.smithi016.stdout:104.out 2023-12-08T08:41:32.451 INFO:teuthology.orchestra.run.smithi016.stdout:105 2023-12-08T08:41:32.451 INFO:teuthology.orchestra.run.smithi016.stdout:105.out 2023-12-08T08:41:32.451 INFO:teuthology.orchestra.run.smithi016.stdout:106 2023-12-08T08:41:32.452 INFO:teuthology.orchestra.run.smithi016.stdout:106.out 2023-12-08T08:41:32.452 INFO:teuthology.orchestra.run.smithi016.stdout:107 2023-12-08T08:41:32.452 INFO:teuthology.orchestra.run.smithi016.stdout:107.out 2023-12-08T08:41:32.452 INFO:teuthology.orchestra.run.smithi016.stdout:108 2023-12-08T08:41:32.452 INFO:teuthology.orchestra.run.smithi016.stdout:108.out 2023-12-08T08:41:32.452 INFO:teuthology.orchestra.run.smithi016.stdout:109 2023-12-08T08:41:32.452 INFO:teuthology.orchestra.run.smithi016.stdout:109.out 2023-12-08T08:41:32.452 INFO:teuthology.orchestra.run.smithi016.stdout:110 2023-12-08T08:41:32.452 INFO:teuthology.orchestra.run.smithi016.stdout:110.out 2023-12-08T08:41:32.452 INFO:teuthology.orchestra.run.smithi016.stdout:111 2023-12-08T08:41:32.452 INFO:teuthology.orchestra.run.smithi016.stdout:111.out 2023-12-08T08:41:32.452 INFO:teuthology.orchestra.run.smithi016.stdout:112 2023-12-08T08:41:32.452 INFO:teuthology.orchestra.run.smithi016.stdout:112.out 2023-12-08T08:41:32.452 INFO:teuthology.orchestra.run.smithi016.stdout:113 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:113.out 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:114 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:114.out 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:115 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:115.out 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:116 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:116.out 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:117 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:117.out 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:118 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:118.out 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:119 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:119.out 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:120 2023-12-08T08:41:32.453 INFO:teuthology.orchestra.run.smithi016.stdout:120.out 2023-12-08T08:41:32.454 INFO:teuthology.orchestra.run.smithi016.stdout:121 2023-12-08T08:41:32.454 INFO:teuthology.orchestra.run.smithi016.stdout:121.out 2023-12-08T08:41:32.454 INFO:teuthology.orchestra.run.smithi016.stdout:122 2023-12-08T08:41:32.454 INFO:teuthology.orchestra.run.smithi016.stdout:122.out 2023-12-08T08:41:32.454 INFO:teuthology.orchestra.run.smithi016.stdout:123 2023-12-08T08:41:32.454 INFO:teuthology.orchestra.run.smithi016.stdout:123.out 2023-12-08T08:41:32.454 INFO:teuthology.orchestra.run.smithi016.stdout:124 2023-12-08T08:41:32.454 INFO:teuthology.orchestra.run.smithi016.stdout:124.out 2023-12-08T08:41:32.454 INFO:teuthology.orchestra.run.smithi016.stdout:125 2023-12-08T08:41:32.454 INFO:teuthology.orchestra.run.smithi016.stdout:125.out 2023-12-08T08:41:32.454 INFO:teuthology.orchestra.run.smithi016.stdout:126 2023-12-08T08:41:32.454 INFO:teuthology.orchestra.run.smithi016.stdout:126.out 2023-12-08T08:41:32.454 INFO:teuthology.orchestra.run.smithi016.stdout:127 2023-12-08T08:41:32.454 INFO:teuthology.orchestra.run.smithi016.stdout:127.out 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:128 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:128.out 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:129 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:129.out 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:130 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:130.out 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:131 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:131.out 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:132 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:132.out 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:133 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:133.out 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:134 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:134.out 2023-12-08T08:41:32.455 INFO:teuthology.orchestra.run.smithi016.stdout:135 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:135.out 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:136 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:136.out 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:137 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:137.out 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:138 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:138.out 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:139 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:139.out 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:140 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:140.out 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:141 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:141.out 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:142 2023-12-08T08:41:32.456 INFO:teuthology.orchestra.run.smithi016.stdout:142.out 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:143 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:143.out 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:144 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:144.out 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:145 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:145.out 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:146 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:146.out 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:147 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:147.out 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:148 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:148.out 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:149 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:149.out 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:150 2023-12-08T08:41:32.457 INFO:teuthology.orchestra.run.smithi016.stdout:150.out 2023-12-08T08:41:32.458 INFO:teuthology.orchestra.run.smithi016.stdout:151 2023-12-08T08:41:32.458 INFO:teuthology.orchestra.run.smithi016.stdout:151.out 2023-12-08T08:41:32.458 INFO:teuthology.orchestra.run.smithi016.stdout:152 2023-12-08T08:41:32.458 INFO:teuthology.orchestra.run.smithi016.stdout:152.out 2023-12-08T08:41:32.458 INFO:teuthology.orchestra.run.smithi016.stdout:153 2023-12-08T08:41:32.458 INFO:teuthology.orchestra.run.smithi016.stdout:153.out 2023-12-08T08:41:32.458 INFO:teuthology.orchestra.run.smithi016.stdout:154 2023-12-08T08:41:32.458 INFO:teuthology.orchestra.run.smithi016.stdout:154.out 2023-12-08T08:41:32.458 INFO:teuthology.orchestra.run.smithi016.stdout:155 2023-12-08T08:41:32.458 INFO:teuthology.orchestra.run.smithi016.stdout:155.out 2023-12-08T08:41:32.458 INFO:teuthology.orchestra.run.smithi016.stdout:156 2023-12-08T08:41:32.458 INFO:teuthology.orchestra.run.smithi016.stdout:156.out 2023-12-08T08:41:32.458 INFO:teuthology.orchestra.run.smithi016.stdout:157 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:157.out 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:158 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:158.out 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:159 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:159.out 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:160 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:160.out 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:161 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:161.out 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:162 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:162.out 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:163 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:163.out 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:164 2023-12-08T08:41:32.459 INFO:teuthology.orchestra.run.smithi016.stdout:164.out 2023-12-08T08:41:32.460 INFO:teuthology.orchestra.run.smithi016.stdout:165 2023-12-08T08:41:32.460 INFO:teuthology.orchestra.run.smithi016.stdout:165.out 2023-12-08T08:41:32.460 INFO:teuthology.orchestra.run.smithi016.stdout:166 2023-12-08T08:41:32.460 INFO:teuthology.orchestra.run.smithi016.stdout:166.out 2023-12-08T08:41:32.460 INFO:teuthology.orchestra.run.smithi016.stdout:167 2023-12-08T08:41:32.460 INFO:teuthology.orchestra.run.smithi016.stdout:167.out 2023-12-08T08:41:32.460 INFO:teuthology.orchestra.run.smithi016.stdout:168 2023-12-08T08:41:32.460 INFO:teuthology.orchestra.run.smithi016.stdout:168.out 2023-12-08T08:41:32.460 INFO:teuthology.orchestra.run.smithi016.stdout:169 2023-12-08T08:41:32.460 INFO:teuthology.orchestra.run.smithi016.stdout:169.out 2023-12-08T08:41:32.460 INFO:teuthology.orchestra.run.smithi016.stdout:170 2023-12-08T08:41:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:170.out 2023-12-08T08:41:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:171 2023-12-08T08:41:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:171.out 2023-12-08T08:41:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:172 2023-12-08T08:41:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:172.out 2023-12-08T08:41:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:173 2023-12-08T08:41:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:173.out 2023-12-08T08:41:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:174 2023-12-08T08:41:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:174.out 2023-12-08T08:41:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:175 2023-12-08T08:41:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:175.out 2023-12-08T08:41:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:176 2023-12-08T08:41:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:176.out 2023-12-08T08:41:32.461 INFO:teuthology.orchestra.run.smithi016.stdout:177 2023-12-08T08:41:32.462 INFO:teuthology.orchestra.run.smithi016.stdout:177.out 2023-12-08T08:41:32.462 INFO:teuthology.orchestra.run.smithi016.stdout:178 2023-12-08T08:41:32.462 INFO:teuthology.orchestra.run.smithi016.stdout:178.out 2023-12-08T08:41:32.462 INFO:teuthology.orchestra.run.smithi016.stdout:179 2023-12-08T08:41:32.462 INFO:teuthology.orchestra.run.smithi016.stdout:179.out 2023-12-08T08:41:32.462 INFO:teuthology.orchestra.run.smithi016.stdout:180 2023-12-08T08:41:32.462 INFO:teuthology.orchestra.run.smithi016.stdout:180.out 2023-12-08T08:41:32.462 INFO:teuthology.orchestra.run.smithi016.stdout:181 2023-12-08T08:41:32.462 INFO:teuthology.orchestra.run.smithi016.stdout:181.out 2023-12-08T08:41:32.462 INFO:teuthology.orchestra.run.smithi016.stdout:182 2023-12-08T08:41:32.462 INFO:teuthology.orchestra.run.smithi016.stdout:182.out 2023-12-08T08:41:32.462 INFO:teuthology.orchestra.run.smithi016.stdout:183 2023-12-08T08:41:32.462 INFO:teuthology.orchestra.run.smithi016.stdout:183.out 2023-12-08T08:41:32.462 INFO:teuthology.orchestra.run.smithi016.stdout:184 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:184.out 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:185 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:185.out 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:186 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:186.out 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:187 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:187.out 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:188 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:188.out 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:189 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:189.out 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:190 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:190.out 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:191 2023-12-08T08:41:32.463 INFO:teuthology.orchestra.run.smithi016.stdout:191.out 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:192 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:192.out 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:193 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:193.out 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:194 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:194.out 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:195 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:195.out 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:196 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:196.out 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:197 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:197.out 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:198 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:198.out 2023-12-08T08:41:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:199 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:199.out 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:200 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:200.out 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:201 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:201.out 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:202 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:202.out 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:203 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:203.out 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:204 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:204.out 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:205 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:205.out 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:206 2023-12-08T08:41:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:206.out 2023-12-08T08:41:32.466 INFO:teuthology.orchestra.run.smithi016.stdout:207 2023-12-08T08:41:32.466 INFO:teuthology.orchestra.run.smithi016.stdout:207.out 2023-12-08T08:41:32.466 INFO:teuthology.orchestra.run.smithi016.stdout:208 2023-12-08T08:41:32.466 INFO:teuthology.orchestra.run.smithi016.stdout:208.out 2023-12-08T08:41:32.466 INFO:teuthology.orchestra.run.smithi016.stdout:209 2023-12-08T08:41:32.466 INFO:teuthology.orchestra.run.smithi016.stdout:209.out 2023-12-08T08:41:32.466 INFO:teuthology.orchestra.run.smithi016.stdout:210 2023-12-08T08:41:32.466 INFO:teuthology.orchestra.run.smithi016.stdout:210.out 2023-12-08T08:41:32.466 INFO:teuthology.orchestra.run.smithi016.stdout:211 2023-12-08T08:41:32.466 INFO:teuthology.orchestra.run.smithi016.stdout:211.out 2023-12-08T08:41:32.466 INFO:teuthology.orchestra.run.smithi016.stdout:212 2023-12-08T08:41:32.466 INFO:teuthology.orchestra.run.smithi016.stdout:212.out 2023-12-08T08:41:32.466 INFO:teuthology.orchestra.run.smithi016.stdout:213 2023-12-08T08:41:32.466 INFO:teuthology.orchestra.run.smithi016.stdout:213.out 2023-12-08T08:41:32.467 INFO:teuthology.orchestra.run.smithi016.stdout:214 2023-12-08T08:41:32.467 INFO:teuthology.orchestra.run.smithi016.stdout:214.out 2023-12-08T08:41:32.467 INFO:teuthology.orchestra.run.smithi016.stdout:215 2023-12-08T08:41:32.467 INFO:teuthology.orchestra.run.smithi016.stdout:215.out 2023-12-08T08:41:32.467 INFO:teuthology.orchestra.run.smithi016.stdout:216 2023-12-08T08:41:32.467 INFO:teuthology.orchestra.run.smithi016.stdout:216.out 2023-12-08T08:41:32.467 INFO:teuthology.orchestra.run.smithi016.stdout:217 2023-12-08T08:41:32.467 INFO:teuthology.orchestra.run.smithi016.stdout:217.out 2023-12-08T08:41:32.467 INFO:teuthology.orchestra.run.smithi016.stdout:218 2023-12-08T08:41:32.467 INFO:teuthology.orchestra.run.smithi016.stdout:218.out 2023-12-08T08:41:32.467 INFO:teuthology.orchestra.run.smithi016.stdout:219 2023-12-08T08:41:32.467 INFO:teuthology.orchestra.run.smithi016.stdout:219.out 2023-12-08T08:41:32.467 INFO:teuthology.orchestra.run.smithi016.stdout:220 2023-12-08T08:41:32.467 INFO:teuthology.orchestra.run.smithi016.stdout:220.out 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:221 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:221.out 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:222 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:222.out 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:223 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:223.out 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:224 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:224.out 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:225 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:225.out 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:226 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:226.out 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:227 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:227.out 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:228 2023-12-08T08:41:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:228.out 2023-12-08T08:41:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:229 2023-12-08T08:41:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:229.out 2023-12-08T08:41:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:230 2023-12-08T08:41:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:230.out 2023-12-08T08:41:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:231 2023-12-08T08:41:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:231.out 2023-12-08T08:41:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:232 2023-12-08T08:41:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:232.out 2023-12-08T08:41:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:233 2023-12-08T08:41:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:233.out 2023-12-08T08:41:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:234 2023-12-08T08:41:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:234.out 2023-12-08T08:41:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:235 2023-12-08T08:41:32.469 INFO:teuthology.orchestra.run.smithi016.stdout:235.out 2023-12-08T08:41:32.470 INFO:teuthology.orchestra.run.smithi016.stdout:236 2023-12-08T08:41:32.470 INFO:teuthology.orchestra.run.smithi016.stdout:236.out 2023-12-08T08:41:32.470 INFO:teuthology.orchestra.run.smithi016.stdout:237 2023-12-08T08:41:32.470 INFO:teuthology.orchestra.run.smithi016.stdout:237.out 2023-12-08T08:41:32.470 INFO:teuthology.orchestra.run.smithi016.stdout:238 2023-12-08T08:41:32.470 INFO:teuthology.orchestra.run.smithi016.stdout:238.out 2023-12-08T08:41:32.470 INFO:teuthology.orchestra.run.smithi016.stdout:239 2023-12-08T08:41:32.470 INFO:teuthology.orchestra.run.smithi016.stdout:239.out 2023-12-08T08:41:32.470 INFO:teuthology.orchestra.run.smithi016.stdout:240 2023-12-08T08:41:32.470 INFO:teuthology.orchestra.run.smithi016.stdout:240.out 2023-12-08T08:41:32.470 INFO:teuthology.orchestra.run.smithi016.stdout:241 2023-12-08T08:41:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:241.out 2023-12-08T08:41:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:242 2023-12-08T08:41:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:242.out 2023-12-08T08:41:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:243 2023-12-08T08:41:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:243.out 2023-12-08T08:41:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:244 2023-12-08T08:41:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:244.out 2023-12-08T08:41:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:245 2023-12-08T08:41:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:245.out 2023-12-08T08:41:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:246 2023-12-08T08:41:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:246.out 2023-12-08T08:41:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:247 2023-12-08T08:41:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:247.out 2023-12-08T08:41:32.471 INFO:teuthology.orchestra.run.smithi016.stdout:248 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:248.out 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:249 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:249.out 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:250 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:250.out 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:251 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:251.out 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:252 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:252.out 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:253 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:253.out 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:254 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:254.out 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:255 2023-12-08T08:41:32.472 INFO:teuthology.orchestra.run.smithi016.stdout:255.out 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:256 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:256.out 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:257 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:257.out 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:258 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:258.out 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:259 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:259.out 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:260 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:260.out 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:261 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:261.out 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:262 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:262.out 2023-12-08T08:41:32.473 INFO:teuthology.orchestra.run.smithi016.stdout:263 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:263.out 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:264 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:264.out 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:265 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:265.out 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:266 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:266.out 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:267 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:267.out 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:268 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:268.out 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:269 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:269.out 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:270 2023-12-08T08:41:32.474 INFO:teuthology.orchestra.run.smithi016.stdout:270.out 2023-12-08T08:41:32.475 INFO:teuthology.orchestra.run.smithi016.stdout:271 2023-12-08T08:41:32.475 INFO:teuthology.orchestra.run.smithi016.stdout:271.out 2023-12-08T08:41:32.475 INFO:teuthology.orchestra.run.smithi016.stdout:272 2023-12-08T08:41:32.475 INFO:teuthology.orchestra.run.smithi016.stdout:272.out 2023-12-08T08:41:32.475 INFO:teuthology.orchestra.run.smithi016.stdout:273 2023-12-08T08:41:32.475 INFO:teuthology.orchestra.run.smithi016.stdout:273.out 2023-12-08T08:41:32.475 INFO:teuthology.orchestra.run.smithi016.stdout:274 2023-12-08T08:41:32.475 INFO:teuthology.orchestra.run.smithi016.stdout:274.out 2023-12-08T08:41:32.475 INFO:teuthology.orchestra.run.smithi016.stdout:275 2023-12-08T08:41:32.475 INFO:teuthology.orchestra.run.smithi016.stdout:275.out 2023-12-08T08:41:32.475 INFO:teuthology.orchestra.run.smithi016.stdout:276 2023-12-08T08:41:32.475 INFO:teuthology.orchestra.run.smithi016.stdout:276.out 2023-12-08T08:41:32.475 INFO:teuthology.orchestra.run.smithi016.stdout:277 2023-12-08T08:41:32.475 INFO:teuthology.orchestra.run.smithi016.stdout:277.out 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:278 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:278.out 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:279 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:279.out 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:280 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:280.out 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:281 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:281.out 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:282 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:282.out 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:283 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:283.out 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:284 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:284.out 2023-12-08T08:41:32.476 INFO:teuthology.orchestra.run.smithi016.stdout:285 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:285.out 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:286 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:286.out 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:287 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:287.out 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:288 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:288.out 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:289 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:289.out 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:290 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:290.out 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:291 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:291.out 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:292 2023-12-08T08:41:32.477 INFO:teuthology.orchestra.run.smithi016.stdout:292.out 2023-12-08T08:41:32.478 INFO:teuthology.orchestra.run.smithi016.stdout:293 2023-12-08T08:41:32.478 INFO:teuthology.orchestra.run.smithi016.stdout:293.out 2023-12-08T08:41:32.478 INFO:teuthology.orchestra.run.smithi016.stdout:294 2023-12-08T08:41:32.478 INFO:teuthology.orchestra.run.smithi016.stdout:294.out 2023-12-08T08:41:32.478 INFO:teuthology.orchestra.run.smithi016.stdout:295 2023-12-08T08:41:32.478 INFO:teuthology.orchestra.run.smithi016.stdout:295.out 2023-12-08T08:41:32.478 INFO:teuthology.orchestra.run.smithi016.stdout:296 2023-12-08T08:41:32.478 INFO:teuthology.orchestra.run.smithi016.stdout:296.out 2023-12-08T08:41:32.478 INFO:teuthology.orchestra.run.smithi016.stdout:297 2023-12-08T08:41:32.478 INFO:teuthology.orchestra.run.smithi016.stdout:297.out 2023-12-08T08:41:32.478 INFO:teuthology.orchestra.run.smithi016.stdout:298 2023-12-08T08:41:32.478 INFO:teuthology.orchestra.run.smithi016.stdout:298.out 2023-12-08T08:41:32.478 INFO:teuthology.orchestra.run.smithi016.stdout:299 2023-12-08T08:41:32.478 INFO:teuthology.orchestra.run.smithi016.stdout:299.out 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:300 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:300.out 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:301 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:301.out 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:302 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:302.out 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:303 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:303.out 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:304 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:304.out 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:305 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:305.out 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:306 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:306.out 2023-12-08T08:41:32.479 INFO:teuthology.orchestra.run.smithi016.stdout:307 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:307.out 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:308 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:308.out 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:309 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:309.out 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:310 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:310.out 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:311 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:311.out 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:312 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:312.out 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:313 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:313.out 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:314 2023-12-08T08:41:32.480 INFO:teuthology.orchestra.run.smithi016.stdout:314.out 2023-12-08T08:41:32.481 INFO:teuthology.orchestra.run.smithi016.stdout:315 2023-12-08T08:41:32.481 INFO:teuthology.orchestra.run.smithi016.stdout:315.out 2023-12-08T08:41:32.481 INFO:teuthology.orchestra.run.smithi016.stdout:316 2023-12-08T08:41:32.481 INFO:teuthology.orchestra.run.smithi016.stdout:316.out 2023-12-08T08:41:32.481 INFO:teuthology.orchestra.run.smithi016.stdout:317 2023-12-08T08:41:32.481 INFO:teuthology.orchestra.run.smithi016.stdout:317.out 2023-12-08T08:41:32.481 INFO:teuthology.orchestra.run.smithi016.stdout:318 2023-12-08T08:41:32.481 INFO:teuthology.orchestra.run.smithi016.stdout:318.out 2023-12-08T08:41:32.481 INFO:teuthology.orchestra.run.smithi016.stdout:319 2023-12-08T08:41:32.481 INFO:teuthology.orchestra.run.smithi016.stdout:319.out 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:320 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:320.out 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:321 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:321.out 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:322 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:322.out 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:323 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:323.out 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:324 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:324.out 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:325 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:325.out 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:326 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:326.out 2023-12-08T08:41:32.482 INFO:teuthology.orchestra.run.smithi016.stdout:327 2023-12-08T08:41:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:327.out 2023-12-08T08:41:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:328 2023-12-08T08:41:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:328.out 2023-12-08T08:41:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:329 2023-12-08T08:41:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:329.out 2023-12-08T08:41:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:330 2023-12-08T08:41:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:330.out 2023-12-08T08:41:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:331 2023-12-08T08:41:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:331.out 2023-12-08T08:41:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:332 2023-12-08T08:41:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:332.out 2023-12-08T08:41:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:333 2023-12-08T08:41:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:333.out 2023-12-08T08:41:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:334 2023-12-08T08:41:32.484 INFO:teuthology.orchestra.run.smithi016.stdout:334.out 2023-12-08T08:41:32.484 INFO:teuthology.orchestra.run.smithi016.stdout:335 2023-12-08T08:41:32.484 INFO:teuthology.orchestra.run.smithi016.stdout:335.out 2023-12-08T08:41:32.484 INFO:teuthology.orchestra.run.smithi016.stdout:336 2023-12-08T08:41:32.484 INFO:teuthology.orchestra.run.smithi016.stdout:336.out 2023-12-08T08:41:32.484 INFO:teuthology.orchestra.run.smithi016.stdout:337 2023-12-08T08:41:32.484 INFO:teuthology.orchestra.run.smithi016.stdout:337.out 2023-12-08T08:41:32.484 INFO:teuthology.orchestra.run.smithi016.stdout:338 2023-12-08T08:41:32.484 INFO:teuthology.orchestra.run.smithi016.stdout:338.out 2023-12-08T08:41:32.484 INFO:teuthology.orchestra.run.smithi016.stdout:339 2023-12-08T08:41:32.484 INFO:teuthology.orchestra.run.smithi016.stdout:339.out 2023-12-08T08:41:32.484 INFO:teuthology.orchestra.run.smithi016.stdout:340 2023-12-08T08:41:32.484 INFO:teuthology.orchestra.run.smithi016.stdout:340.out 2023-12-08T08:41:32.484 INFO:teuthology.orchestra.run.smithi016.stdout:341 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:341.out 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:342 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:342.out 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:343 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:343.out 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:344 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:344.out 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:345 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:345.out 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:346 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:346.out 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:347 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:347.out 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:348 2023-12-08T08:41:32.485 INFO:teuthology.orchestra.run.smithi016.stdout:348.out 2023-12-08T08:41:32.486 INFO:teuthology.orchestra.run.smithi016.stdout:349 2023-12-08T08:41:32.486 INFO:teuthology.orchestra.run.smithi016.stdout:349.out 2023-12-08T08:41:32.486 INFO:teuthology.orchestra.run.smithi016.stdout:350 2023-12-08T08:41:32.486 INFO:teuthology.orchestra.run.smithi016.stdout:350.out 2023-12-08T08:41:32.486 INFO:teuthology.orchestra.run.smithi016.stdout:351 2023-12-08T08:41:32.486 INFO:teuthology.orchestra.run.smithi016.stdout:351.out 2023-12-08T08:41:32.486 INFO:teuthology.orchestra.run.smithi016.stdout:352 2023-12-08T08:41:32.486 INFO:teuthology.orchestra.run.smithi016.stdout:352.out 2023-12-08T08:41:32.486 INFO:teuthology.orchestra.run.smithi016.stdout:353 2023-12-08T08:41:32.486 INFO:teuthology.orchestra.run.smithi016.stdout:353.out 2023-12-08T08:41:32.486 INFO:teuthology.orchestra.run.smithi016.stdout:354 2023-12-08T08:41:32.486 INFO:teuthology.orchestra.run.smithi016.stdout:354.out 2023-12-08T08:41:32.486 INFO:teuthology.orchestra.run.smithi016.stdout:355 2023-12-08T08:41:32.486 INFO:teuthology.orchestra.run.smithi016.stdout:355.out 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:356 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:356.out 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:357 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:357.out 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:358 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:358.out 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:359 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:359.out 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:360 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:360.out 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:361 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:361.out 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:362 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:362.out 2023-12-08T08:41:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:363 2023-12-08T08:41:32.488 INFO:teuthology.orchestra.run.smithi016.stdout:363.out 2023-12-08T08:41:32.488 INFO:teuthology.orchestra.run.smithi016.stdout:364 2023-12-08T08:41:32.488 INFO:teuthology.orchestra.run.smithi016.stdout:364.out 2023-12-08T08:41:32.488 INFO:teuthology.orchestra.run.smithi016.stdout:365 2023-12-08T08:41:32.488 INFO:teuthology.orchestra.run.smithi016.stdout:365.out 2023-12-08T08:41:32.488 INFO:teuthology.orchestra.run.smithi016.stdout:366 2023-12-08T08:41:32.488 INFO:teuthology.orchestra.run.smithi016.stdout:366.out 2023-12-08T08:41:32.488 INFO:teuthology.orchestra.run.smithi016.stdout:367 2023-12-08T08:41:32.488 INFO:teuthology.orchestra.run.smithi016.stdout:367.out 2023-12-08T08:41:32.488 INFO:teuthology.orchestra.run.smithi016.stdout:368 2023-12-08T08:41:32.488 INFO:teuthology.orchestra.run.smithi016.stdout:368.out 2023-12-08T08:41:32.488 INFO:teuthology.orchestra.run.smithi016.stdout:369 2023-12-08T08:41:32.488 INFO:teuthology.orchestra.run.smithi016.stdout:369.out 2023-12-08T08:41:32.488 INFO:teuthology.orchestra.run.smithi016.stdout:370 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:370.out 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:371 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:371.out 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:372 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:372.out 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:373 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:373.out 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:374 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:374.out 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:375 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:375.out 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:376 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:376.out 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:377 2023-12-08T08:41:32.489 INFO:teuthology.orchestra.run.smithi016.stdout:377.out 2023-12-08T08:41:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:378 2023-12-08T08:41:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:378.out 2023-12-08T08:41:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:379 2023-12-08T08:41:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:379.out 2023-12-08T08:41:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:380 2023-12-08T08:41:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:380.out 2023-12-08T08:41:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:381 2023-12-08T08:41:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:381.out 2023-12-08T08:41:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:382 2023-12-08T08:41:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:382.out 2023-12-08T08:41:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:383 2023-12-08T08:41:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:383.out 2023-12-08T08:41:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:384 2023-12-08T08:41:32.490 INFO:teuthology.orchestra.run.smithi016.stdout:384.out 2023-12-08T08:41:32.491 INFO:teuthology.orchestra.run.smithi016.stdout:385 2023-12-08T08:41:32.491 INFO:teuthology.orchestra.run.smithi016.stdout:385.out 2023-12-08T08:41:32.491 INFO:teuthology.orchestra.run.smithi016.stdout:386 2023-12-08T08:41:32.491 INFO:teuthology.orchestra.run.smithi016.stdout:386.out 2023-12-08T08:41:32.491 INFO:teuthology.orchestra.run.smithi016.stdout:387 2023-12-08T08:41:32.491 INFO:teuthology.orchestra.run.smithi016.stdout:387.out 2023-12-08T08:41:32.491 INFO:teuthology.orchestra.run.smithi016.stdout:388 2023-12-08T08:41:32.491 INFO:teuthology.orchestra.run.smithi016.stdout:388.out 2023-12-08T08:41:32.491 INFO:teuthology.orchestra.run.smithi016.stdout:389 2023-12-08T08:41:32.491 INFO:teuthology.orchestra.run.smithi016.stdout:389.out 2023-12-08T08:41:32.491 INFO:teuthology.orchestra.run.smithi016.stdout:390 2023-12-08T08:41:32.492 INFO:teuthology.orchestra.run.smithi016.stdout:390.out 2023-12-08T08:41:32.492 INFO:teuthology.orchestra.run.smithi016.stdout:391 2023-12-08T08:41:32.492 INFO:teuthology.orchestra.run.smithi016.stdout:391.out 2023-12-08T08:41:32.492 INFO:teuthology.orchestra.run.smithi016.stdout:392 2023-12-08T08:41:32.492 INFO:teuthology.orchestra.run.smithi016.stdout:392.out 2023-12-08T08:41:32.492 INFO:teuthology.orchestra.run.smithi016.stdout:393 2023-12-08T08:41:32.492 INFO:teuthology.orchestra.run.smithi016.stdout:393.out 2023-12-08T08:41:32.492 INFO:teuthology.orchestra.run.smithi016.stdout:394 2023-12-08T08:41:32.492 INFO:teuthology.orchestra.run.smithi016.stdout:394.out 2023-12-08T08:41:32.492 INFO:teuthology.orchestra.run.smithi016.stdout:395 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:395.out 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:396 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:396.out 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:397 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:397.out 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:398 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:398.out 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:399 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:399.out 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:400 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:400.out 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:401 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:401.out 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:402 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:402.out 2023-12-08T08:41:32.493 INFO:teuthology.orchestra.run.smithi016.stdout:403 2023-12-08T08:41:32.494 INFO:teuthology.orchestra.run.smithi016.stdout:403.out 2023-12-08T08:41:32.494 INFO:teuthology.orchestra.run.smithi016.stdout:404 2023-12-08T08:41:32.494 INFO:teuthology.orchestra.run.smithi016.stdout:404.out 2023-12-08T08:41:32.494 INFO:teuthology.orchestra.run.smithi016.stdout:405 2023-12-08T08:41:32.494 INFO:teuthology.orchestra.run.smithi016.stdout:405.out 2023-12-08T08:41:32.494 INFO:teuthology.orchestra.run.smithi016.stdout:406 2023-12-08T08:41:32.494 INFO:teuthology.orchestra.run.smithi016.stdout:406.out 2023-12-08T08:41:32.494 INFO:teuthology.orchestra.run.smithi016.stdout:407 2023-12-08T08:41:32.494 INFO:teuthology.orchestra.run.smithi016.stdout:407.out 2023-12-08T08:41:32.494 INFO:teuthology.orchestra.run.smithi016.stdout:408 2023-12-08T08:41:32.494 INFO:teuthology.orchestra.run.smithi016.stdout:408.out 2023-12-08T08:41:32.494 INFO:teuthology.orchestra.run.smithi016.stdout:409 2023-12-08T08:41:32.494 INFO:teuthology.orchestra.run.smithi016.stdout:409.out 2023-12-08T08:41:32.494 INFO:teuthology.orchestra.run.smithi016.stdout:410 2023-12-08T08:41:32.495 INFO:teuthology.orchestra.run.smithi016.stdout:410.out 2023-12-08T08:41:32.495 INFO:teuthology.orchestra.run.smithi016.stdout:411 2023-12-08T08:41:32.495 INFO:teuthology.orchestra.run.smithi016.stdout:411.out 2023-12-08T08:41:32.495 INFO:teuthology.orchestra.run.smithi016.stdout:412 2023-12-08T08:41:32.495 INFO:teuthology.orchestra.run.smithi016.stdout:412.out 2023-12-08T08:41:32.495 INFO:teuthology.orchestra.run.smithi016.stdout:413 2023-12-08T08:41:32.495 INFO:teuthology.orchestra.run.smithi016.stdout:413.out 2023-12-08T08:41:32.495 INFO:teuthology.orchestra.run.smithi016.stdout:414 2023-12-08T08:41:32.495 INFO:teuthology.orchestra.run.smithi016.stdout:414.out 2023-12-08T08:41:32.495 INFO:teuthology.orchestra.run.smithi016.stdout:415 2023-12-08T08:41:32.495 INFO:teuthology.orchestra.run.smithi016.stdout:415.out 2023-12-08T08:41:32.495 INFO:teuthology.orchestra.run.smithi016.stdout:416 2023-12-08T08:41:32.495 INFO:teuthology.orchestra.run.smithi016.stdout:416.out 2023-12-08T08:41:32.495 INFO:teuthology.orchestra.run.smithi016.stdout:417 2023-12-08T08:41:32.496 INFO:teuthology.orchestra.run.smithi016.stdout:417.out 2023-12-08T08:41:32.496 INFO:teuthology.orchestra.run.smithi016.stdout:418 2023-12-08T08:41:32.496 INFO:teuthology.orchestra.run.smithi016.stdout:418.out 2023-12-08T08:41:32.496 INFO:teuthology.orchestra.run.smithi016.stdout:419 2023-12-08T08:41:32.496 INFO:teuthology.orchestra.run.smithi016.stdout:419.out 2023-12-08T08:41:32.496 INFO:teuthology.orchestra.run.smithi016.stdout:420 2023-12-08T08:41:32.496 INFO:teuthology.orchestra.run.smithi016.stdout:420.out 2023-12-08T08:41:32.496 INFO:teuthology.orchestra.run.smithi016.stdout:421 2023-12-08T08:41:32.496 INFO:teuthology.orchestra.run.smithi016.stdout:421.out 2023-12-08T08:41:32.496 INFO:teuthology.orchestra.run.smithi016.stdout:422 2023-12-08T08:41:32.496 INFO:teuthology.orchestra.run.smithi016.stdout:422.out 2023-12-08T08:41:32.496 INFO:teuthology.orchestra.run.smithi016.stdout:423 2023-12-08T08:41:32.496 INFO:teuthology.orchestra.run.smithi016.stdout:423.out 2023-12-08T08:41:32.496 INFO:teuthology.orchestra.run.smithi016.stdout:424 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:424.out 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:425 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:425.out 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:426 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:426.out 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:427 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:427.out 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:428 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:428.out 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:429 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:429.out 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:430 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:430.out 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:431 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:431.out 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:432 2023-12-08T08:41:32.497 INFO:teuthology.orchestra.run.smithi016.stdout:432.out 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:433 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:433.out 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:434 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:434.out 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:435 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:435.out 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:436 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:436.out 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:437 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:437.out 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:438 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:438.out 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:439 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:439.out 2023-12-08T08:41:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:440 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:440.out 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:441 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:441.out 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:442 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:442.out 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:443 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:443.out 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:444 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:444.out 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:445 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:445.out 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:446 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:446.out 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:447 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:447.out 2023-12-08T08:41:32.499 INFO:teuthology.orchestra.run.smithi016.stdout:448 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:448.out 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:449 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:449.out 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:450 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:450.out 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:451 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:451.out 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:452 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:452.out 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:453 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:453.out 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:454 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:454.out 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:455 2023-12-08T08:41:32.500 INFO:teuthology.orchestra.run.smithi016.stdout:455.out 2023-12-08T08:41:32.501 INFO:teuthology.orchestra.run.smithi016.stdout:456 2023-12-08T08:41:32.501 INFO:teuthology.orchestra.run.smithi016.stdout:456.out 2023-12-08T08:41:32.501 INFO:teuthology.orchestra.run.smithi016.stdout:457 2023-12-08T08:41:32.501 INFO:teuthology.orchestra.run.smithi016.stdout:457.out 2023-12-08T08:41:32.501 INFO:teuthology.orchestra.run.smithi016.stdout:458 2023-12-08T08:41:32.501 INFO:teuthology.orchestra.run.smithi016.stdout:458.out 2023-12-08T08:41:32.501 INFO:teuthology.orchestra.run.smithi016.stdout:459 2023-12-08T08:41:32.501 INFO:teuthology.orchestra.run.smithi016.stdout:459.out 2023-12-08T08:41:32.501 INFO:teuthology.orchestra.run.smithi016.stdout:460 2023-12-08T08:41:32.501 INFO:teuthology.orchestra.run.smithi016.stdout:460.out 2023-12-08T08:41:32.501 INFO:teuthology.orchestra.run.smithi016.stdout:461 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:461.out 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:462 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:462.out 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:463 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:463.out 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:464 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:464.out 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:465 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:465.out 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:466 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:466.out 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:467 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:467.out 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:468 2023-12-08T08:41:32.502 INFO:teuthology.orchestra.run.smithi016.stdout:468.out 2023-12-08T08:41:32.503 INFO:teuthology.orchestra.run.smithi016.stdout:469 2023-12-08T08:41:32.503 INFO:teuthology.orchestra.run.smithi016.stdout:469.out 2023-12-08T08:41:32.503 INFO:teuthology.orchestra.run.smithi016.stdout:470 2023-12-08T08:41:32.503 INFO:teuthology.orchestra.run.smithi016.stdout:470.out 2023-12-08T08:41:32.503 INFO:teuthology.orchestra.run.smithi016.stdout:471 2023-12-08T08:41:32.503 INFO:teuthology.orchestra.run.smithi016.stdout:471.out 2023-12-08T08:41:32.503 INFO:teuthology.orchestra.run.smithi016.stdout:472 2023-12-08T08:41:32.503 INFO:teuthology.orchestra.run.smithi016.stdout:472.out 2023-12-08T08:41:32.503 INFO:teuthology.orchestra.run.smithi016.stdout:473 2023-12-08T08:41:32.503 INFO:teuthology.orchestra.run.smithi016.stdout:473.out 2023-12-08T08:41:32.503 INFO:teuthology.orchestra.run.smithi016.stdout:474 2023-12-08T08:41:32.503 INFO:teuthology.orchestra.run.smithi016.stdout:474.out 2023-12-08T08:41:32.503 INFO:teuthology.orchestra.run.smithi016.stdout:475 2023-12-08T08:41:32.503 INFO:teuthology.orchestra.run.smithi016.stdout:475.out 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:476 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:476.out 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:477 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:477.out 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:478 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:478.out 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:479 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:479.out 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:480 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:480.out 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:481 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:481.out 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:482 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:482.out 2023-12-08T08:41:32.504 INFO:teuthology.orchestra.run.smithi016.stdout:483 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:483.out 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:484 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:484.out 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:485 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:485.out 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:486 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:486.out 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:487 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:487.out 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:488 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:488.out 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:489 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:489.out 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:490 2023-12-08T08:41:32.505 INFO:teuthology.orchestra.run.smithi016.stdout:490.out 2023-12-08T08:41:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:491 2023-12-08T08:41:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:491.out 2023-12-08T08:41:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:492 2023-12-08T08:41:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:492.out 2023-12-08T08:41:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:493 2023-12-08T08:41:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:493.out 2023-12-08T08:41:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:494 2023-12-08T08:41:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:494.out 2023-12-08T08:41:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:495 2023-12-08T08:41:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:495.out 2023-12-08T08:41:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:496 2023-12-08T08:41:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:496.out 2023-12-08T08:41:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:497 2023-12-08T08:41:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:497.out 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:498 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:498.out 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:499 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:499.out 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:500 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:500.out 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:501 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:501.out 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:502 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:502.out 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:503 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:503.out 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:504 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:504.out 2023-12-08T08:41:32.507 INFO:teuthology.orchestra.run.smithi016.stdout:505 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:505.out 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:506 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:506.out 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:507 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:507.out 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:508 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:508.out 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:509 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:509.out 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:510 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:510.out 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:511 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:511.out 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:512 2023-12-08T08:41:32.508 INFO:teuthology.orchestra.run.smithi016.stdout:512.out 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:513 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:513.out 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:514 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:514.out 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:515 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:515.out 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:516 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:516.out 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:517 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:517.out 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:518 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:518.out 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:519 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:519.out 2023-12-08T08:41:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:520 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:520.out 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:521 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:521.out 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:522 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:522.out 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:523 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:523.out 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:524 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:524.out 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:525 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:525.out 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:526 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:526.out 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:527 2023-12-08T08:41:32.510 INFO:teuthology.orchestra.run.smithi016.stdout:527.out 2023-12-08T08:41:32.511 INFO:teuthology.orchestra.run.smithi016.stdout:528 2023-12-08T08:41:32.511 INFO:teuthology.orchestra.run.smithi016.stdout:528.out 2023-12-08T08:41:32.511 INFO:teuthology.orchestra.run.smithi016.stdout:529 2023-12-08T08:41:32.511 INFO:teuthology.orchestra.run.smithi016.stdout:529.out 2023-12-08T08:41:32.511 INFO:teuthology.orchestra.run.smithi016.stdout:530 2023-12-08T08:41:32.511 INFO:teuthology.orchestra.run.smithi016.stdout:530.out 2023-12-08T08:41:32.511 INFO:teuthology.orchestra.run.smithi016.stdout:531 2023-12-08T08:41:32.511 INFO:teuthology.orchestra.run.smithi016.stdout:531.out 2023-12-08T08:41:32.511 INFO:teuthology.orchestra.run.smithi016.stdout:532 2023-12-08T08:41:32.511 INFO:teuthology.orchestra.run.smithi016.stdout:532.out 2023-12-08T08:41:32.511 INFO:teuthology.orchestra.run.smithi016.stdout:533 2023-12-08T08:41:32.511 INFO:teuthology.orchestra.run.smithi016.stdout:533.out 2023-12-08T08:41:32.511 INFO:teuthology.orchestra.run.smithi016.stdout:534 2023-12-08T08:41:32.511 INFO:teuthology.orchestra.run.smithi016.stdout:534.out 2023-12-08T08:41:32.512 INFO:teuthology.orchestra.run.smithi016.stdout:535 2023-12-08T08:41:32.512 INFO:teuthology.orchestra.run.smithi016.stdout:535.out 2023-12-08T08:41:32.512 INFO:teuthology.orchestra.run.smithi016.stdout:536 2023-12-08T08:41:32.512 INFO:teuthology.orchestra.run.smithi016.stdout:536.out 2023-12-08T08:41:32.512 INFO:teuthology.orchestra.run.smithi016.stdout:537 2023-12-08T08:41:32.512 INFO:teuthology.orchestra.run.smithi016.stdout:537.out 2023-12-08T08:41:32.512 INFO:teuthology.orchestra.run.smithi016.stdout:538 2023-12-08T08:41:32.512 INFO:teuthology.orchestra.run.smithi016.stdout:538.out 2023-12-08T08:41:32.512 INFO:teuthology.orchestra.run.smithi016.stdout:539 2023-12-08T08:41:32.512 INFO:teuthology.orchestra.run.smithi016.stdout:539.out 2023-12-08T08:41:32.512 INFO:teuthology.orchestra.run.smithi016.stdout:540 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:540.out 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:541 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:541.out 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:542 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:542.out 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:543 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:543.out 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:544 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:544.out 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:545 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:545.out 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:546 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:546.out 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:547 2023-12-08T08:41:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:547.out 2023-12-08T08:41:32.514 INFO:teuthology.orchestra.run.smithi016.stdout:548 2023-12-08T08:41:32.514 INFO:teuthology.orchestra.run.smithi016.stdout:548.out 2023-12-08T08:41:32.514 INFO:teuthology.orchestra.run.smithi016.stdout:549 2023-12-08T08:41:32.514 INFO:teuthology.orchestra.run.smithi016.stdout:549.out 2023-12-08T08:41:32.514 INFO:teuthology.orchestra.run.smithi016.stdout:550 2023-12-08T08:41:32.514 INFO:teuthology.orchestra.run.smithi016.stdout:550.out 2023-12-08T08:41:32.514 INFO:teuthology.orchestra.run.smithi016.stdout:551 2023-12-08T08:41:32.514 INFO:teuthology.orchestra.run.smithi016.stdout:551.out 2023-12-08T08:41:32.514 INFO:teuthology.orchestra.run.smithi016.stdout:552 2023-12-08T08:41:32.514 INFO:teuthology.orchestra.run.smithi016.stdout:552.out 2023-12-08T08:41:32.514 INFO:teuthology.orchestra.run.smithi016.stdout:553 2023-12-08T08:41:32.514 INFO:teuthology.orchestra.run.smithi016.stdout:553.out 2023-12-08T08:41:32.514 INFO:teuthology.orchestra.run.smithi016.stdout:554 2023-12-08T08:41:32.514 INFO:teuthology.orchestra.run.smithi016.stdout:554.out 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:555 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:555.out 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:556 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:556.out 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:557 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:557.out 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:558 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:558.out 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:559 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:559.out 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:560 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:560.out 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:561 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:561.out 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:562 2023-12-08T08:41:32.515 INFO:teuthology.orchestra.run.smithi016.stdout:562.out 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:563 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:563.out 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:564 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:564.out 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:565 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:565.out 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:566 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:566.out 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:567 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:567.out 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:568 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:568.out 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:569 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:569.out 2023-12-08T08:41:32.516 INFO:teuthology.orchestra.run.smithi016.stdout:570 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:570.out 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:571 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:571.out 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:572 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:572.out 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:573 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:573.out 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:574 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:574.out 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:575 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:575.out 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:576 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:576.out 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:577 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:577.out 2023-12-08T08:41:32.517 INFO:teuthology.orchestra.run.smithi016.stdout:578 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:578.out 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:579 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:579.out 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:580 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:580.out 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:581 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:581.out 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:582 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:582.out 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:583 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:583.out 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:584 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:584.out 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:585 2023-12-08T08:41:32.518 INFO:teuthology.orchestra.run.smithi016.stdout:585.out 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:586 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:586.out 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:587 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:587.out 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:588 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:588.out 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:589 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:589.out 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:590 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:590.out 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:591 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:591.out 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:592 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:592.out 2023-12-08T08:41:32.519 INFO:teuthology.orchestra.run.smithi016.stdout:593 2023-12-08T08:41:32.520 INFO:teuthology.orchestra.run.smithi016.stdout:593.out 2023-12-08T08:41:32.520 INFO:teuthology.orchestra.run.smithi016.stdout:594 2023-12-08T08:41:32.520 INFO:teuthology.orchestra.run.smithi016.stdout:594.out 2023-12-08T08:41:32.520 INFO:teuthology.orchestra.run.smithi016.stdout:595 2023-12-08T08:41:32.520 INFO:teuthology.orchestra.run.smithi016.stdout:595.out 2023-12-08T08:41:32.520 INFO:teuthology.orchestra.run.smithi016.stdout:596 2023-12-08T08:41:32.520 INFO:teuthology.orchestra.run.smithi016.stdout:596.out 2023-12-08T08:41:32.520 INFO:teuthology.orchestra.run.smithi016.stdout:597 2023-12-08T08:41:32.520 INFO:teuthology.orchestra.run.smithi016.stdout:597.out 2023-12-08T08:41:32.520 INFO:teuthology.orchestra.run.smithi016.stdout:598 2023-12-08T08:41:32.520 INFO:teuthology.orchestra.run.smithi016.stdout:598.out 2023-12-08T08:41:32.520 INFO:teuthology.orchestra.run.smithi016.stdout:599 2023-12-08T08:41:32.520 INFO:teuthology.orchestra.run.smithi016.stdout:599.out 2023-12-08T08:41:32.520 INFO:teuthology.orchestra.run.smithi016.stdout:600 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:600.out 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:601 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:601.out 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:602 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:602.out 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:603 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:603.out 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:604 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:604.out 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:605 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:605.out 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:606 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:606.out 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:607 2023-12-08T08:41:32.521 INFO:teuthology.orchestra.run.smithi016.stdout:607.out 2023-12-08T08:41:32.522 INFO:teuthology.orchestra.run.smithi016.stdout:608 2023-12-08T08:41:32.522 INFO:teuthology.orchestra.run.smithi016.stdout:608.out 2023-12-08T08:41:32.522 INFO:teuthology.orchestra.run.smithi016.stdout:609 2023-12-08T08:41:32.522 INFO:teuthology.orchestra.run.smithi016.stdout:609.out 2023-12-08T08:41:32.522 INFO:teuthology.orchestra.run.smithi016.stdout:610 2023-12-08T08:41:32.522 INFO:teuthology.orchestra.run.smithi016.stdout:610.out 2023-12-08T08:41:32.522 INFO:teuthology.orchestra.run.smithi016.stdout:611 2023-12-08T08:41:32.522 INFO:teuthology.orchestra.run.smithi016.stdout:611.out 2023-12-08T08:41:32.522 INFO:teuthology.orchestra.run.smithi016.stdout:612 2023-12-08T08:41:32.522 INFO:teuthology.orchestra.run.smithi016.stdout:612.out 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:613 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:613.out 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:614 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:614.out 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:615 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:615.out 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:616 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:616.out 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:617 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:617.out 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:618 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:618.out 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:619 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:619.out 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:620 2023-12-08T08:41:32.523 INFO:teuthology.orchestra.run.smithi016.stdout:620.out 2023-12-08T08:41:32.524 INFO:teuthology.orchestra.run.smithi016.stdout:621 2023-12-08T08:41:32.524 INFO:teuthology.orchestra.run.smithi016.stdout:621.out 2023-12-08T08:41:32.524 INFO:teuthology.orchestra.run.smithi016.stdout:622 2023-12-08T08:41:32.524 INFO:teuthology.orchestra.run.smithi016.stdout:622.out 2023-12-08T08:41:32.524 INFO:teuthology.orchestra.run.smithi016.stdout:623 2023-12-08T08:41:32.524 INFO:teuthology.orchestra.run.smithi016.stdout:623.out 2023-12-08T08:41:32.524 INFO:teuthology.orchestra.run.smithi016.stdout:624 2023-12-08T08:41:32.524 INFO:teuthology.orchestra.run.smithi016.stdout:624.out 2023-12-08T08:41:32.524 INFO:teuthology.orchestra.run.smithi016.stdout:625 2023-12-08T08:41:32.524 INFO:teuthology.orchestra.run.smithi016.stdout:625.out 2023-12-08T08:41:32.524 INFO:teuthology.orchestra.run.smithi016.stdout:626 2023-12-08T08:41:32.524 INFO:teuthology.orchestra.run.smithi016.stdout:626.out 2023-12-08T08:41:32.524 INFO:teuthology.orchestra.run.smithi016.stdout:627 2023-12-08T08:41:32.525 INFO:teuthology.orchestra.run.smithi016.stdout:627.out 2023-12-08T08:41:32.525 INFO:teuthology.orchestra.run.smithi016.stdout:628 2023-12-08T08:41:32.525 INFO:teuthology.orchestra.run.smithi016.stdout:628.out 2023-12-08T08:41:32.525 INFO:teuthology.orchestra.run.smithi016.stdout:629 2023-12-08T08:41:32.525 INFO:teuthology.orchestra.run.smithi016.stdout:629.out 2023-12-08T08:41:32.525 INFO:teuthology.orchestra.run.smithi016.stdout:630 2023-12-08T08:41:32.525 INFO:teuthology.orchestra.run.smithi016.stdout:630.out 2023-12-08T08:41:32.525 INFO:teuthology.orchestra.run.smithi016.stdout:631 2023-12-08T08:41:32.525 INFO:teuthology.orchestra.run.smithi016.stdout:631.out 2023-12-08T08:41:32.526 INFO:teuthology.orchestra.run.smithi016.stdout:632 2023-12-08T08:41:32.526 INFO:teuthology.orchestra.run.smithi016.stdout:632.out 2023-12-08T08:41:32.526 INFO:teuthology.orchestra.run.smithi016.stdout:633 2023-12-08T08:41:32.526 INFO:teuthology.orchestra.run.smithi016.stdout:633.out 2023-12-08T08:41:32.526 INFO:teuthology.orchestra.run.smithi016.stdout:634 2023-12-08T08:41:32.526 INFO:teuthology.orchestra.run.smithi016.stdout:634.out 2023-12-08T08:41:32.526 INFO:teuthology.orchestra.run.smithi016.stdout:635 2023-12-08T08:41:32.526 INFO:teuthology.orchestra.run.smithi016.stdout:635.out 2023-12-08T08:41:32.526 INFO:teuthology.orchestra.run.smithi016.stdout:636 2023-12-08T08:41:32.526 INFO:teuthology.orchestra.run.smithi016.stdout:636.out 2023-12-08T08:41:32.526 INFO:teuthology.orchestra.run.smithi016.stdout:637 2023-12-08T08:41:32.526 INFO:teuthology.orchestra.run.smithi016.stdout:637.out 2023-12-08T08:41:32.526 INFO:teuthology.orchestra.run.smithi016.stdout:638 2023-12-08T08:41:32.526 INFO:teuthology.orchestra.run.smithi016.stdout:638.out 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:639 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:639.out 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:640 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:640.out 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:641 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:641.out 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:642 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:642.out 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:643 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:643.out 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:644 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:644.out 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:645 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:645.out 2023-12-08T08:41:32.527 INFO:teuthology.orchestra.run.smithi016.stdout:646 2023-12-08T08:41:32.528 INFO:teuthology.orchestra.run.smithi016.stdout:646.out 2023-12-08T08:41:32.528 INFO:teuthology.orchestra.run.smithi016.stdout:647 2023-12-08T08:41:32.528 INFO:teuthology.orchestra.run.smithi016.stdout:647.out 2023-12-08T08:41:32.528 INFO:teuthology.orchestra.run.smithi016.stdout:648 2023-12-08T08:41:32.528 INFO:teuthology.orchestra.run.smithi016.stdout:648.out 2023-12-08T08:41:32.528 INFO:teuthology.orchestra.run.smithi016.stdout:649 2023-12-08T08:41:32.528 INFO:teuthology.orchestra.run.smithi016.stdout:649.out 2023-12-08T08:41:32.528 INFO:teuthology.orchestra.run.smithi016.stdout:650 2023-12-08T08:41:32.529 INFO:teuthology.orchestra.run.smithi016.stdout:650.out 2023-12-08T08:41:32.529 INFO:teuthology.orchestra.run.smithi016.stdout:651 2023-12-08T08:41:32.529 INFO:teuthology.orchestra.run.smithi016.stdout:651.out 2023-12-08T08:41:32.529 INFO:teuthology.orchestra.run.smithi016.stdout:652 2023-12-08T08:41:32.529 INFO:teuthology.orchestra.run.smithi016.stdout:652.out 2023-12-08T08:41:32.529 INFO:teuthology.orchestra.run.smithi016.stdout:653 2023-12-08T08:41:32.529 INFO:teuthology.orchestra.run.smithi016.stdout:653.out 2023-12-08T08:41:32.529 INFO:teuthology.orchestra.run.smithi016.stdout:654 2023-12-08T08:41:32.529 INFO:teuthology.orchestra.run.smithi016.stdout:654.out 2023-12-08T08:41:32.529 INFO:teuthology.orchestra.run.smithi016.stdout:655 2023-12-08T08:41:32.529 INFO:teuthology.orchestra.run.smithi016.stdout:655.out 2023-12-08T08:41:32.529 INFO:teuthology.orchestra.run.smithi016.stdout:656 2023-12-08T08:41:32.529 INFO:teuthology.orchestra.run.smithi016.stdout:656.out 2023-12-08T08:41:32.529 INFO:teuthology.orchestra.run.smithi016.stdout:657 2023-12-08T08:41:32.530 INFO:teuthology.orchestra.run.smithi016.stdout:657.out 2023-12-08T08:41:32.530 INFO:teuthology.orchestra.run.smithi016.stdout:658 2023-12-08T08:41:32.530 INFO:teuthology.orchestra.run.smithi016.stdout:658.out 2023-12-08T08:41:32.530 INFO:teuthology.orchestra.run.smithi016.stdout:659 2023-12-08T08:41:32.530 INFO:teuthology.orchestra.run.smithi016.stdout:659.out 2023-12-08T08:41:32.530 INFO:teuthology.orchestra.run.smithi016.stdout:660 2023-12-08T08:41:32.530 INFO:teuthology.orchestra.run.smithi016.stdout:660.out 2023-12-08T08:41:32.530 INFO:teuthology.orchestra.run.smithi016.stdout:661 2023-12-08T08:41:32.530 INFO:teuthology.orchestra.run.smithi016.stdout:661.out 2023-12-08T08:41:32.530 INFO:teuthology.orchestra.run.smithi016.stdout:662 2023-12-08T08:41:32.530 INFO:teuthology.orchestra.run.smithi016.stdout:662.out 2023-12-08T08:41:32.530 INFO:teuthology.orchestra.run.smithi016.stdout:663 2023-12-08T08:41:32.530 INFO:teuthology.orchestra.run.smithi016.stdout:663.out 2023-12-08T08:41:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:664 2023-12-08T08:41:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:664.out 2023-12-08T08:41:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:665 2023-12-08T08:41:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:665.out 2023-12-08T08:41:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:666 2023-12-08T08:41:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:666.out 2023-12-08T08:41:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:667 2023-12-08T08:41:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:667.out 2023-12-08T08:41:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:668 2023-12-08T08:41:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:668.out 2023-12-08T08:41:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:669 2023-12-08T08:41:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:669.out 2023-12-08T08:41:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:670 2023-12-08T08:41:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:670.out 2023-12-08T08:41:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:671 2023-12-08T08:41:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:671.out 2023-12-08T08:41:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:672 2023-12-08T08:41:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:672.out 2023-12-08T08:41:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:673 2023-12-08T08:41:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:673.out 2023-12-08T08:41:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:674 2023-12-08T08:41:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:674.out 2023-12-08T08:41:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:675 2023-12-08T08:41:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:675.out 2023-12-08T08:41:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:676 2023-12-08T08:41:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:676.out 2023-12-08T08:41:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:677 2023-12-08T08:41:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:677.out 2023-12-08T08:41:32.533 INFO:teuthology.orchestra.run.smithi016.stdout:678 2023-12-08T08:41:32.533 INFO:teuthology.orchestra.run.smithi016.stdout:678.out 2023-12-08T08:41:32.533 INFO:teuthology.orchestra.run.smithi016.stdout:679 2023-12-08T08:41:32.533 INFO:teuthology.orchestra.run.smithi016.stdout:679.out 2023-12-08T08:41:32.533 INFO:teuthology.orchestra.run.smithi016.stdout:680 2023-12-08T08:41:32.533 INFO:teuthology.orchestra.run.smithi016.stdout:680.out 2023-12-08T08:41:32.533 INFO:teuthology.orchestra.run.smithi016.stdout:681 2023-12-08T08:41:32.533 INFO:teuthology.orchestra.run.smithi016.stdout:681.out 2023-12-08T08:41:32.533 INFO:teuthology.orchestra.run.smithi016.stdout:682 2023-12-08T08:41:32.533 INFO:teuthology.orchestra.run.smithi016.stdout:682.out 2023-12-08T08:41:32.533 INFO:teuthology.orchestra.run.smithi016.stdout:683 2023-12-08T08:41:32.533 INFO:teuthology.orchestra.run.smithi016.stdout:683.out 2023-12-08T08:41:32.533 INFO:teuthology.orchestra.run.smithi016.stdout:684 2023-12-08T08:41:32.533 INFO:teuthology.orchestra.run.smithi016.stdout:684.out 2023-12-08T08:41:32.534 INFO:teuthology.orchestra.run.smithi016.stdout:685 2023-12-08T08:41:32.534 INFO:teuthology.orchestra.run.smithi016.stdout:685.out 2023-12-08T08:41:32.534 INFO:teuthology.orchestra.run.smithi016.stdout:686 2023-12-08T08:41:32.534 INFO:teuthology.orchestra.run.smithi016.stdout:686.out 2023-12-08T08:41:32.534 INFO:teuthology.orchestra.run.smithi016.stdout:687 2023-12-08T08:41:32.534 INFO:teuthology.orchestra.run.smithi016.stdout:687.out 2023-12-08T08:41:32.534 INFO:teuthology.orchestra.run.smithi016.stdout:688 2023-12-08T08:41:32.534 INFO:teuthology.orchestra.run.smithi016.stdout:688.out 2023-12-08T08:41:32.534 INFO:teuthology.orchestra.run.smithi016.stdout:689 2023-12-08T08:41:32.534 INFO:teuthology.orchestra.run.smithi016.stdout:689.out 2023-12-08T08:41:32.534 INFO:teuthology.orchestra.run.smithi016.stdout:690 2023-12-08T08:41:32.534 INFO:teuthology.orchestra.run.smithi016.stdout:690.out 2023-12-08T08:41:32.534 INFO:teuthology.orchestra.run.smithi016.stdout:691 2023-12-08T08:41:32.534 INFO:teuthology.orchestra.run.smithi016.stdout:691.out 2023-12-08T08:41:32.535 INFO:teuthology.orchestra.run.smithi016.stdout:692 2023-12-08T08:41:32.535 INFO:teuthology.orchestra.run.smithi016.stdout:692.out 2023-12-08T08:41:32.535 INFO:teuthology.orchestra.run.smithi016.stdout:693 2023-12-08T08:41:32.535 INFO:teuthology.orchestra.run.smithi016.stdout:693.out 2023-12-08T08:41:32.535 INFO:teuthology.orchestra.run.smithi016.stdout:694 2023-12-08T08:41:32.535 INFO:teuthology.orchestra.run.smithi016.stdout:694.out 2023-12-08T08:41:32.535 INFO:teuthology.orchestra.run.smithi016.stdout:695 2023-12-08T08:41:32.535 INFO:teuthology.orchestra.run.smithi016.stdout:695.out 2023-12-08T08:41:32.535 INFO:teuthology.orchestra.run.smithi016.stdout:696 2023-12-08T08:41:32.535 INFO:teuthology.orchestra.run.smithi016.stdout:696.out 2023-12-08T08:41:32.535 INFO:teuthology.orchestra.run.smithi016.stdout:697 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:697.out 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:698 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:698.out 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:699 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:699.out 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:700 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:700.out 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:701 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:701.out 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:702 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:702.out 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:703 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:703.out 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:704 2023-12-08T08:41:32.536 INFO:teuthology.orchestra.run.smithi016.stdout:704.out 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:705 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:705.out 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:706 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:706.out 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:707 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:707.out 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:708 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:708.out 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:709 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:709.out 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:710 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:710.out 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:711 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:711.out 2023-12-08T08:41:32.537 INFO:teuthology.orchestra.run.smithi016.stdout:712 2023-12-08T08:41:32.538 INFO:teuthology.orchestra.run.smithi016.stdout:712.out 2023-12-08T08:41:32.538 INFO:teuthology.orchestra.run.smithi016.stdout:713 2023-12-08T08:41:32.538 INFO:teuthology.orchestra.run.smithi016.stdout:713.out 2023-12-08T08:41:32.538 INFO:teuthology.orchestra.run.smithi016.stdout:714 2023-12-08T08:41:32.538 INFO:teuthology.orchestra.run.smithi016.stdout:714.out 2023-12-08T08:41:32.538 INFO:teuthology.orchestra.run.smithi016.stdout:715 2023-12-08T08:41:32.538 INFO:teuthology.orchestra.run.smithi016.stdout:715.out 2023-12-08T08:41:32.538 INFO:teuthology.orchestra.run.smithi016.stdout:716 2023-12-08T08:41:32.538 INFO:teuthology.orchestra.run.smithi016.stdout:716.out 2023-12-08T08:41:32.538 INFO:teuthology.orchestra.run.smithi016.stdout:717 2023-12-08T08:41:32.538 INFO:teuthology.orchestra.run.smithi016.stdout:717.out 2023-12-08T08:41:32.538 INFO:teuthology.orchestra.run.smithi016.stdout:718 2023-12-08T08:41:32.538 INFO:teuthology.orchestra.run.smithi016.stdout:718.out 2023-12-08T08:41:32.538 INFO:teuthology.orchestra.run.smithi016.stdout:719 2023-12-08T08:41:32.539 INFO:teuthology.orchestra.run.smithi016.stdout:719.out 2023-12-08T08:41:32.539 INFO:teuthology.orchestra.run.smithi016.stdout:720 2023-12-08T08:41:32.539 INFO:teuthology.orchestra.run.smithi016.stdout:720.out 2023-12-08T08:41:32.539 INFO:teuthology.orchestra.run.smithi016.stdout:721 2023-12-08T08:41:32.539 INFO:teuthology.orchestra.run.smithi016.stdout:721.out 2023-12-08T08:41:32.539 INFO:teuthology.orchestra.run.smithi016.stdout:722 2023-12-08T08:41:32.539 INFO:teuthology.orchestra.run.smithi016.stdout:722.out 2023-12-08T08:41:32.539 INFO:teuthology.orchestra.run.smithi016.stdout:723 2023-12-08T08:41:32.539 INFO:teuthology.orchestra.run.smithi016.stdout:723.out 2023-12-08T08:41:32.539 INFO:teuthology.orchestra.run.smithi016.stdout:724 2023-12-08T08:41:32.539 INFO:teuthology.orchestra.run.smithi016.stdout:724.out 2023-12-08T08:41:32.539 INFO:teuthology.orchestra.run.smithi016.stdout:725 2023-12-08T08:41:32.539 INFO:teuthology.orchestra.run.smithi016.stdout:725.out 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:726 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:726.out 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:727 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:727.out 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:728 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:728.out 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:729 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:729.out 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:730 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:730.out 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:731 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:731.out 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:732 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:732.out 2023-12-08T08:41:32.540 INFO:teuthology.orchestra.run.smithi016.stdout:group.list 2023-12-08T08:41:32.541 INFO:teuthology.orchestra.run.smithi016.stdout:Makefile 2023-12-08T08:41:32.541 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2023-12-08T08:41:32.688 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:41:32.692 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:41:32.692 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:41:34.650 INFO:teuthology.orchestra.run.smithi016.stdout:generic/001 2s 2023-12-08T08:41:34.655 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/001 2023-12-08T08:41:34.655 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:41:34.658 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:41:34.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi016.stderr:ceph-fuse[105452]: fuse finished with error 0 and tester_r 0 2023-12-08T08:41:34.667 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:41:34.667 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2023-12-08T08:41:34.839 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:41:34.843 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:41:34.843 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:41:35.265 INFO:teuthology.orchestra.run.smithi016.stdout:generic/002 1s 2023-12-08T08:41:35.270 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/002 2023-12-08T08:41:35.270 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:41:35.273 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:41:35.281 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:41:35.281 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2023-12-08T08:41:35.469 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:41:35.473 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:41:35.473 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:41:35.586 INFO:teuthology.orchestra.run.smithi016.stdout:generic/003 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:41:35.589 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/003 2023-12-08T08:41:35.589 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/003 2023-12-08T08:41:35.589 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:41:35.592 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:41:35.600 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:41:35.600 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2023-12-08T08:41:35.772 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:41:35.776 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:41:35.776 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:41:35.892 INFO:teuthology.orchestra.run.smithi016.stdout:generic/004 [not run] O_TMPFILE is not supported 2023-12-08T08:41:35.896 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/004 2023-12-08T08:41:35.896 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/004 2023-12-08T08:41:35.896 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:41:35.899 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:41:35.907 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:41:35.907 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2023-12-08T08:41:36.076 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:41:36.080 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:41:36.080 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:41:36.577 INFO:teuthology.orchestra.run.smithi016.stdout:generic/005 0s 2023-12-08T08:41:36.582 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/005 2023-12-08T08:41:36.582 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:41:36.585 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:41:36.593 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:41:36.593 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2023-12-08T08:41:36.787 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:41:36.791 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:41:36.791 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:41:41.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:41:41.256 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:42:08.133 INFO:teuthology.orchestra.run.smithi016.stdout:generic/006 32s 2023-12-08T08:42:08.139 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/006 2023-12-08T08:42:08.139 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:42:08.141 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:42:08.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:42:08.150 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2023-12-08T08:42:08.344 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:42:08.348 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:42:08.348 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:42:11.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:42:11.596 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:42:41.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:42:41.678 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:43:11.389 INFO:teuthology.orchestra.run.smithi016.stdout:generic/007 63s 2023-12-08T08:43:11.397 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/007 2023-12-08T08:43:11.398 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:43:11.402 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:43:11.415 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:43:11.416 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2023-12-08T08:43:11.642 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:43:11.646 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:43:11.646 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:43:11.759 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:43:11.761 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:43:12.527 INFO:teuthology.orchestra.run.smithi016.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-12-08T08:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/008 2023-12-08T08:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/008 2023-12-08T08:43:12.531 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:43:12.533 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:43:12.541 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:43:12.542 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2023-12-08T08:43:12.710 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:43:12.714 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:43:12.714 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:43:13.550 INFO:teuthology.orchestra.run.smithi016.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-12-08T08:43:13.553 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/009 2023-12-08T08:43:13.553 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/009 2023-12-08T08:43:13.553 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:43:13.556 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:43:13.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:43:13.564 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2023-12-08T08:43:13.732 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:43:13.736 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:43:13.736 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:43:14.355 INFO:teuthology.orchestra.run.smithi016.stdout:generic/010 1s 2023-12-08T08:43:14.434 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/010 2023-12-08T08:43:14.434 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:43:14.436 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:43:14.447 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:43:14.447 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2023-12-08T08:43:14.640 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:43:14.644 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:43:14.644 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:43:41.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:43:41.834 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:43:47.983 INFO:teuthology.orchestra.run.smithi016.stdout:generic/011 33s 2023-12-08T08:43:47.988 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/011 2023-12-08T08:43:47.988 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:43:47.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:43:48.000 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:43:48.000 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2023-12-08T08:43:48.198 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:43:48.202 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:43:48.203 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:43:49.145 INFO:teuthology.orchestra.run.smithi016.stdout:generic/012 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-12-08T08:43:49.148 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/012 2023-12-08T08:43:49.148 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/012 2023-12-08T08:43:49.148 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:43:49.151 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:43:49.173 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:43:49.173 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2023-12-08T08:43:49.355 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:43:49.359 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:43:49.359 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:44:12.219 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:44:12.222 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:44:42.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:44:42.771 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:45:06.485 INFO:teuthology.orchestra.run.smithi016.stdout:generic/013 77s 2023-12-08T08:45:06.492 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/013 2023-12-08T08:45:06.493 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:45:06.497 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:45:06.507 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:45:06.508 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2023-12-08T08:45:06.780 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:45:06.787 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:45:06.788 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:45:13.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:45:13.101 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:45:43.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:45:43.410 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:45:43.478 INFO:tasks.ceph.mds.b.smithi112.stderr:2023-12-08T08:45:43.476+0000 7f0cd7b54640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 96998) UID: 0 2023-12-08T08:45:43.479 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-12-08T08:45:43.476+0000 7f52ecb1a640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 96998) UID: 0 2023-12-08T08:48:51.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:48:51.901 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:49:20.549 INFO:teuthology.orchestra.run.smithi016.stdout:generic/014 254s 2023-12-08T08:49:20.555 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/014 2023-12-08T08:49:20.555 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:20.559 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:20.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:20.574 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2023-12-08T08:49:20.806 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:20.811 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:20.811 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:20.958 INFO:teuthology.orchestra.run.smithi016.stdout:generic/015 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:49:20.962 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/015 2023-12-08T08:49:20.962 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/015 2023-12-08T08:49:20.962 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:20.965 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:20.976 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:20.977 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2023-12-08T08:49:21.215 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:21.220 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:21.220 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:22.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:49:22.187 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:49:22.598 INFO:teuthology.orchestra.run.smithi016.stdout:generic/016 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-12-08T08:49:22.603 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/016 2023-12-08T08:49:22.603 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/016 2023-12-08T08:49:22.603 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:22.607 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:22.618 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:22.618 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2023-12-08T08:49:22.979 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:22.984 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:22.984 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:23.129 INFO:teuthology.orchestra.run.smithi016.stdout:generic/017 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:49:23.133 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/017 2023-12-08T08:49:23.133 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/017 2023-12-08T08:49:23.133 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:23.136 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:23.151 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:23.152 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2023-12-08T08:49:23.379 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:23.393 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:23.393 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:23.534 INFO:teuthology.orchestra.run.smithi016.stdout:generic/018 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:49:23.538 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/018 2023-12-08T08:49:23.538 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/018 2023-12-08T08:49:23.538 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:23.540 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:23.552 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:23.552 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2023-12-08T08:49:23.797 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:23.801 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:23.801 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:23.949 INFO:teuthology.orchestra.run.smithi016.stdout:generic/019 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:49:23.953 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/019 2023-12-08T08:49:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/019 2023-12-08T08:49:23.954 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:23.956 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:23.967 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:23.968 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2023-12-08T08:49:24.223 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:24.228 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:24.228 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:32.726 INFO:teuthology.orchestra.run.smithi016.stdout:generic/020 - output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/020.out.bad) 2023-12-08T08:49:32.726 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/020.out 2023-12-08 08:39:53.749918085 +0000 2023-12-08T08:49:32.726 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/020.out.bad 2023-12-08 08:49:32.718593658 +0000 2023-12-08T08:49:32.726 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -47,9 +47,13 @@ 2023-12-08T08:49:32.726 INFO:teuthology.orchestra.run.smithi016.stdout: user.snrub="fish2\012" 2023-12-08T08:49:32.726 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:32.726 INFO:teuthology.orchestra.run.smithi016.stdout: *** really long value 2023-12-08T08:49:32.726 INFO:teuthology.orchestra.run.smithi016.stdout: -0000000 00 2023-12-08T08:49:32.727 INFO:teuthology.orchestra.run.smithi016.stdout: -* 2023-12-08T08:49:32.727 INFO:teuthology.orchestra.run.smithi016.stdout: -ATTRSIZE 2023-12-08T08:49:32.727 INFO:teuthology.orchestra.run.smithi016.stdout: +attr_set: No space left on device 2023-12-08T08:49:32.727 INFO:teuthology.orchestra.run.smithi016.stdout: +Could not set "long_attr" for 2023-12-08T08:49:32.727 INFO:teuthology.orchestra.run.smithi016.stdout: +attr_get: No data available 2023-12-08T08:49:32.727 INFO:teuthology.orchestra.run.smithi016.stdout: +Could not get "long_attr" for 2023-12-08T08:49:32.727 INFO:teuthology.orchestra.run.smithi016.stdout: +0000000 2023-12-08T08:49:32.727 INFO:teuthology.orchestra.run.smithi016.stdout: +attr_remove: No data available 2023-12-08T08:49:32.727 INFO:teuthology.orchestra.run.smithi016.stdout: +Could not remove "long_attr" for 2023-12-08T08:49:32.727 INFO:teuthology.orchestra.run.smithi016.stdout: *** set/get/remove really long names (expect failure) 2023-12-08T08:49:32.727 INFO:teuthology.orchestra.run.smithi016.stdout: attr_set: Invalid argument 2023-12-08T08:49:32.727 INFO:teuthology.orchestra.run.smithi016.stdout: Could not set "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" for 2023-12-08T08:49:32.728 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/020 2023-12-08T08:49:32.728 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/020 2023-12-08T08:49:32.729 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T08:49:32.731 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:32.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T08:49:32.743 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T08:49:32.743 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T08:49:32.743 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2023-12-08T08:49:33.124 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:33.128 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:33.129 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:34.494 INFO:teuthology.orchestra.run.smithi016.stdout:generic/021 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-12-08T08:49:34.498 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/021 2023-12-08T08:49:34.498 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/021 2023-12-08T08:49:34.498 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:34.501 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:34.511 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:34.511 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2023-12-08T08:49:34.918 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:34.924 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:34.924 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:36.276 INFO:teuthology.orchestra.run.smithi016.stdout:generic/022 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-12-08T08:49:36.280 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/022 2023-12-08T08:49:36.281 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/022 2023-12-08T08:49:36.281 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:36.283 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:36.297 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:36.297 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2023-12-08T08:49:36.709 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:36.714 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:36.714 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:38.312 INFO:teuthology.orchestra.run.smithi016.stdout:generic/023 2s 2023-12-08T08:49:38.319 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/023 2023-12-08T08:49:38.319 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:38.322 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:38.333 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:38.333 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2023-12-08T08:49:38.752 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:38.757 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:38.757 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:38.935 INFO:teuthology.orchestra.run.smithi016.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2023-12-08T08:49:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/024 2023-12-08T08:49:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/024 2023-12-08T08:49:38.940 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:38.943 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:38.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:38.954 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2023-12-08T08:49:39.353 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:39.359 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:39.359 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:39.560 INFO:teuthology.orchestra.run.smithi016.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2023-12-08T08:49:39.565 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/025 2023-12-08T08:49:39.565 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/025 2023-12-08T08:49:39.565 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:39.568 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:39.596 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:39.596 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2023-12-08T08:49:39.971 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:39.976 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:39.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:40.166 INFO:teuthology.orchestra.run.smithi016.stdout:generic/026 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-12-08T08:49:40.172 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/026 2023-12-08T08:49:40.172 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/026 2023-12-08T08:49:40.172 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:40.175 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:40.186 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:40.186 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2023-12-08T08:49:40.418 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:40.423 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:40.423 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:40.575 INFO:teuthology.orchestra.run.smithi016.stdout:generic/027 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:49:40.579 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/027 2023-12-08T08:49:40.579 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/027 2023-12-08T08:49:40.579 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:40.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:40.602 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:40.603 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2023-12-08T08:49:40.847 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:40.852 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:40.852 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:46.022 INFO:teuthology.orchestra.run.smithi016.stdout:generic/028 6s 2023-12-08T08:49:46.071 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/028 2023-12-08T08:49:46.071 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:46.074 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:46.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:46.089 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2023-12-08T08:49:46.320 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:46.325 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:46.325 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:46.483 INFO:teuthology.orchestra.run.smithi016.stdout:generic/029 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:49:46.487 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/029 2023-12-08T08:49:46.487 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/029 2023-12-08T08:49:46.487 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:46.491 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:46.504 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:46.505 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2023-12-08T08:49:46.743 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:46.749 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:46.749 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:46.894 INFO:teuthology.orchestra.run.smithi016.stdout:generic/030 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:49:46.901 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/030 2023-12-08T08:49:46.901 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/030 2023-12-08T08:49:46.901 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:46.902 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:46.912 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:46.912 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2023-12-08T08:49:47.143 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:47.148 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:47.148 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:47.300 INFO:teuthology.orchestra.run.smithi016.stdout:generic/031 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:49:47.304 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/031 2023-12-08T08:49:47.305 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/031 2023-12-08T08:49:47.305 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:47.307 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:47.319 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:47.319 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2023-12-08T08:49:47.552 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:47.557 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:47.558 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:47.703 INFO:teuthology.orchestra.run.smithi016.stdout:generic/032 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:49:47.707 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/032 2023-12-08T08:49:47.707 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/032 2023-12-08T08:49:47.708 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:47.710 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:47.722 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:47.722 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2023-12-08T08:49:47.960 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:47.969 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:47.969 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:48.117 INFO:teuthology.orchestra.run.smithi016.stdout:generic/033 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:49:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/033 2023-12-08T08:49:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/033 2023-12-08T08:49:48.122 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:48.125 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:48.135 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:48.135 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2023-12-08T08:49:48.380 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:48.385 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:48.385 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:48.544 INFO:teuthology.orchestra.run.smithi016.stdout:generic/034 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:49:48.548 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/034 2023-12-08T08:49:48.549 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/034 2023-12-08T08:49:48.549 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:48.551 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:48.565 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:48.565 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2023-12-08T08:49:48.845 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:48.849 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:48.850 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:49.085 INFO:teuthology.orchestra.run.smithi016.stdout:generic/035 1s 2023-12-08T08:49:49.097 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/035 2023-12-08T08:49:49.097 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:49.100 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:49.113 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:49.113 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2023-12-08T08:49:49.501 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:49:49.506 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:49:49.506 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:52.291 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:49:52.294 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:49:59.780 INFO:teuthology.orchestra.run.smithi016.stdout:generic/036 10s 2023-12-08T08:49:59.789 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/036 2023-12-08T08:49:59.790 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:49:59.792 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:49:59.803 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:49:59.803 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2023-12-08T08:50:00.202 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:00.208 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:00.208 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:00.355 INFO:teuthology.orchestra.run.smithi016.stdout:generic/037 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:00.362 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/037 2023-12-08T08:50:00.362 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/037 2023-12-08T08:50:00.362 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:00.364 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:00.375 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:00.375 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2023-12-08T08:50:00.622 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:00.627 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:00.627 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:00.775 INFO:teuthology.orchestra.run.smithi016.stdout:generic/038 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:00.779 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/038 2023-12-08T08:50:00.779 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/038 2023-12-08T08:50:00.779 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:00.782 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:00.792 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:00.793 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2023-12-08T08:50:01.041 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:01.048 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:01.207 INFO:teuthology.orchestra.run.smithi016.stdout:generic/039 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:01.212 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/039 2023-12-08T08:50:01.212 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/039 2023-12-08T08:50:01.212 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:01.215 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:01.227 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:01.227 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2023-12-08T08:50:01.456 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:01.460 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:01.460 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:01.618 INFO:teuthology.orchestra.run.smithi016.stdout:generic/040 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:01.623 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/040 2023-12-08T08:50:01.623 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/040 2023-12-08T08:50:01.623 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:01.626 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:01.637 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:01.637 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2023-12-08T08:50:01.879 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:01.881 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:01.881 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:02.038 INFO:teuthology.orchestra.run.smithi016.stdout:generic/041 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:02.043 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/041 2023-12-08T08:50:02.043 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/041 2023-12-08T08:50:02.043 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:02.046 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:02.057 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:02.057 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2023-12-08T08:50:02.308 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:02.313 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:02.313 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:02.468 INFO:teuthology.orchestra.run.smithi016.stdout:generic/042 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:02.473 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/042 2023-12-08T08:50:02.473 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/042 2023-12-08T08:50:02.474 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:02.476 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:02.487 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:02.487 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2023-12-08T08:50:02.726 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:02.731 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:02.731 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:02.879 INFO:teuthology.orchestra.run.smithi016.stdout:generic/043 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:02.883 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/043 2023-12-08T08:50:02.883 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/043 2023-12-08T08:50:02.883 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:02.886 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:02.898 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:02.898 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2023-12-08T08:50:03.137 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:03.144 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:03.144 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:03.286 INFO:teuthology.orchestra.run.smithi016.stdout:generic/044 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:03.291 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/044 2023-12-08T08:50:03.291 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/044 2023-12-08T08:50:03.291 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:03.294 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:03.305 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:03.305 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2023-12-08T08:50:03.528 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:03.533 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:03.533 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:03.686 INFO:teuthology.orchestra.run.smithi016.stdout:generic/045 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:03.690 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/045 2023-12-08T08:50:03.691 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/045 2023-12-08T08:50:03.691 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:03.694 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:03.704 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:03.704 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2023-12-08T08:50:03.940 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:03.946 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:03.946 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:04.102 INFO:teuthology.orchestra.run.smithi016.stdout:generic/046 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:04.106 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/046 2023-12-08T08:50:04.106 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/046 2023-12-08T08:50:04.106 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:04.109 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:04.120 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:04.121 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2023-12-08T08:50:04.368 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:04.373 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:04.373 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:04.525 INFO:teuthology.orchestra.run.smithi016.stdout:generic/047 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:04.529 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/047 2023-12-08T08:50:04.530 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/047 2023-12-08T08:50:04.530 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:04.533 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:04.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:04.543 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2023-12-08T08:50:04.775 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:04.781 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:04.781 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:04.936 INFO:teuthology.orchestra.run.smithi016.stdout:generic/048 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:04.940 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/048 2023-12-08T08:50:04.940 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/048 2023-12-08T08:50:04.940 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:04.943 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:04.953 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:04.953 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2023-12-08T08:50:05.196 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:05.201 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:05.201 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:05.352 INFO:teuthology.orchestra.run.smithi016.stdout:generic/049 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:05.356 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/049 2023-12-08T08:50:05.357 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/049 2023-12-08T08:50:05.357 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:05.360 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:05.371 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:05.371 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2023-12-08T08:50:05.605 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:05.609 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:05.609 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:05.771 INFO:teuthology.orchestra.run.smithi016.stdout:generic/050 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:05.775 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/050 2023-12-08T08:50:05.775 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/050 2023-12-08T08:50:05.775 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:05.777 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:05.788 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:05.789 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2023-12-08T08:50:06.024 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:06.029 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:06.029 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:06.172 INFO:teuthology.orchestra.run.smithi016.stdout:generic/051 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/051 2023-12-08T08:50:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/051 2023-12-08T08:50:06.176 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:06.179 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:06.189 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:06.189 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2023-12-08T08:50:06.431 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:06.437 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:06.437 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:06.591 INFO:teuthology.orchestra.run.smithi016.stdout:generic/052 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:06.596 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/052 2023-12-08T08:50:06.596 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/052 2023-12-08T08:50:06.596 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:06.599 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:06.612 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:06.612 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2023-12-08T08:50:06.866 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:06.872 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:06.872 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:07.033 INFO:teuthology.orchestra.run.smithi016.stdout:generic/053 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:07.038 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/053 2023-12-08T08:50:07.038 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/053 2023-12-08T08:50:07.038 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:07.041 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:07.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:07.053 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2023-12-08T08:50:07.297 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:07.301 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:07.302 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:07.444 INFO:teuthology.orchestra.run.smithi016.stdout:generic/054 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:07.449 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/054 2023-12-08T08:50:07.449 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/054 2023-12-08T08:50:07.449 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:07.452 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:07.464 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:07.464 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2023-12-08T08:50:07.696 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:07.701 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:07.701 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:07.841 INFO:teuthology.orchestra.run.smithi016.stdout:generic/055 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:07.844 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/055 2023-12-08T08:50:07.844 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/055 2023-12-08T08:50:07.845 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:07.847 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:07.859 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:07.859 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2023-12-08T08:50:08.090 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:08.095 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:08.095 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:08.260 INFO:teuthology.orchestra.run.smithi016.stdout:generic/056 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:08.265 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/056 2023-12-08T08:50:08.265 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/056 2023-12-08T08:50:08.265 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:08.268 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:08.278 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:08.278 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2023-12-08T08:50:08.513 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:08.519 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:08.519 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:08.675 INFO:teuthology.orchestra.run.smithi016.stdout:generic/057 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:08.679 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/057 2023-12-08T08:50:08.679 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/057 2023-12-08T08:50:08.679 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:08.682 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:08.694 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:08.694 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2023-12-08T08:50:08.939 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:08.943 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:08.944 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:10.291 INFO:teuthology.orchestra.run.smithi016.stdout:generic/058 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-12-08T08:50:10.295 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/058 2023-12-08T08:50:10.295 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/058 2023-12-08T08:50:10.295 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:10.298 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:10.317 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:10.317 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2023-12-08T08:50:10.722 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:10.726 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:10.726 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:10.894 INFO:teuthology.orchestra.run.smithi016.stdout:generic/059 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:10.899 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/059 2023-12-08T08:50:10.899 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/059 2023-12-08T08:50:10.899 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:10.903 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:10.917 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:10.917 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2023-12-08T08:50:11.163 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:11.167 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:11.168 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:12.530 INFO:teuthology.orchestra.run.smithi016.stdout:generic/060 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-12-08T08:50:12.534 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/060 2023-12-08T08:50:12.534 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/060 2023-12-08T08:50:12.535 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:12.537 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:12.566 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:12.566 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2023-12-08T08:50:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:12.962 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:12.962 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:14.347 INFO:teuthology.orchestra.run.smithi016.stdout:generic/061 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-12-08T08:50:14.353 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/061 2023-12-08T08:50:14.353 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/061 2023-12-08T08:50:14.354 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:14.355 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:14.377 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:14.377 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2023-12-08T08:50:14.758 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:14.765 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:14.765 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:14.907 INFO:teuthology.orchestra.run.smithi016.stdout:generic/062 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:14.912 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/062 2023-12-08T08:50:14.912 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/062 2023-12-08T08:50:14.912 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:14.914 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:14.925 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:14.925 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2023-12-08T08:50:15.158 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:15.163 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:15.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:16.521 INFO:teuthology.orchestra.run.smithi016.stdout:generic/063 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-12-08T08:50:16.525 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/063 2023-12-08T08:50:16.525 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/063 2023-12-08T08:50:16.525 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:16.529 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:16.547 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:16.547 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2023-12-08T08:50:16.942 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:16.946 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:16.946 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:17.098 INFO:teuthology.orchestra.run.smithi016.stdout:generic/064 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:17.103 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/064 2023-12-08T08:50:17.103 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/064 2023-12-08T08:50:17.103 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:17.106 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:17.116 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:17.117 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2023-12-08T08:50:17.358 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:17.363 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:17.363 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:17.524 INFO:teuthology.orchestra.run.smithi016.stdout:generic/065 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:17.529 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/065 2023-12-08T08:50:17.529 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/065 2023-12-08T08:50:17.529 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:17.532 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:17.547 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:17.547 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2023-12-08T08:50:17.785 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:17.789 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:17.790 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:17.955 INFO:teuthology.orchestra.run.smithi016.stdout:generic/066 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:17.960 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/066 2023-12-08T08:50:17.960 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/066 2023-12-08T08:50:17.960 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:17.963 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:17.975 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:17.975 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2023-12-08T08:50:18.208 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:18.213 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:18.213 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:18.393 INFO:teuthology.orchestra.run.smithi016.stdout:generic/067 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:18.397 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/067 2023-12-08T08:50:18.398 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/067 2023-12-08T08:50:18.398 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:18.401 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:18.410 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:18.411 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2023-12-08T08:50:18.670 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:18.676 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:18.676 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:18.831 INFO:teuthology.orchestra.run.smithi016.stdout:generic/068 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:18.835 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/068 2023-12-08T08:50:18.835 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/068 2023-12-08T08:50:18.835 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:18.839 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:18.850 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:18.851 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2023-12-08T08:50:19.079 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:19.084 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:19.084 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:19.241 INFO:teuthology.orchestra.run.smithi016.stdout:generic/069 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:50:19.245 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/069 2023-12-08T08:50:19.245 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/069 2023-12-08T08:50:19.245 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:50:19.248 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:19.258 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:50:19.258 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2023-12-08T08:50:19.492 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:50:19.497 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:50:19.497 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:50:22.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:50:22.425 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:50:53.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:50:53.040 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:51:23.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:51:23.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:51:26.151 INFO:teuthology.orchestra.run.smithi016.stdout:generic/070 67s 2023-12-08T08:51:26.157 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/070 2023-12-08T08:51:26.158 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:51:26.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:51:26.173 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:51:26.174 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2023-12-08T08:51:26.645 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:51:26.649 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:51:26.649 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:51:26.790 INFO:teuthology.orchestra.run.smithi016.stdout:generic/071 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:51:26.794 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/071 2023-12-08T08:51:26.794 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/071 2023-12-08T08:51:26.795 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:51:26.797 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:51:26.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:51:26.809 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2023-12-08T08:51:27.049 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:51:27.058 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:51:27.058 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:51:28.363 INFO:teuthology.orchestra.run.smithi016.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2023-12-08T08:51:28.368 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/072 2023-12-08T08:51:28.368 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/072 2023-12-08T08:51:28.368 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:51:28.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:51:28.383 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:51:28.383 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2023-12-08T08:51:28.619 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:51:28.624 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:51:28.624 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:51:28.788 INFO:teuthology.orchestra.run.smithi016.stdout:generic/073 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T08:51:28.795 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/073 2023-12-08T08:51:28.795 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/073 2023-12-08T08:51:28.795 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T08:51:28.798 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:51:28.811 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T08:51:28.811 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2023-12-08T08:51:29.060 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T08:51:29.066 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T08:51:29.066 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T08:51:53.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:51:53.198 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:52:23.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:52:23.582 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:52:53.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:52:53.838 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:53:27.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:53:27.735 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:53:59.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:53:59.419 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:54:29.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:54:29.867 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:55:00.073 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:55:00.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:55:30.552 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:55:30.556 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:56:00.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:56:00.628 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:56:30.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:56:30.769 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:57:00.847 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:57:00.850 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:57:30.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:57:30.924 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:58:01.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:58:01.232 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T08:58:01.296 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T08:58:01.294+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 97359) UID: 0 2023-12-08T08:58:01.296 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T08:58:01.294+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 97359) UID: 0 2023-12-08T08:58:01.297 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T08:58:01.294+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 97359) UID: 0 2023-12-08T08:58:01.297 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T08:58:01.294+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 97359) UID: 0 2023-12-08T09:01:32.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:01:32.139 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:01:32.388 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T09:01:32.386+0000 7ff25129f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 199098) UID: 0 2023-12-08T09:01:32.388 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T09:01:32.386+0000 7eff8ec05640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 199098) UID: 0 2023-12-08T09:01:32.388 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T09:01:32.386+0000 7f6d0f17a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 199098) UID: 0 2023-12-08T09:01:32.400 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T09:01:32.388+0000 7fdb0d8f4640 -1 Fail to open '/proc/199098/cmdline' error = (2) No such file or directory 2023-12-08T09:01:32.401 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T09:01:32.388+0000 7fdb0d8f4640 -1 received signal: Hangup from (PID: 199098) UID: 0 2023-12-08T09:06:51.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:06:51.950 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:06:52.085 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T09:06:52.083+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 97454) UID: 0 2023-12-08T09:06:52.085 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T09:06:52.083+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 97454) UID: 0 2023-12-08T09:06:52.086 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T09:06:52.083+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 97454) UID: 0 2023-12-08T09:06:52.086 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T09:06:52.083+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 97454) UID: 0 2023-12-08T09:06:53.531 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T09:06:53.529+0000 7eff8ec05640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 199312) UID: 0 2023-12-08T09:06:53.531 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T09:06:53.529+0000 7f6d0f17a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 199312) UID: 0 2023-12-08T09:06:53.531 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T09:06:53.529+0000 7ff25129f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 199312) UID: 0 2023-12-08T09:06:53.532 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T09:06:53.529+0000 7fdb0d8f4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 199312) UID: 0 2023-12-08T09:19:14.649 INFO:teuthology.orchestra.run.smithi016.stdout:generic/074 1665s 2023-12-08T09:19:14.742 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/074 2023-12-08T09:19:14.742 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:19:14.744 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:19:14.856 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:19:14.856 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2023-12-08T09:19:16.172 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:19:16.176 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:19:16.176 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:19:53.327 INFO:teuthology.orchestra.run.smithi016.stdout:generic/075 37s 2023-12-08T09:19:53.332 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/075 2023-12-08T09:19:53.333 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:19:53.335 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:19:53.346 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:19:53.346 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2023-12-08T09:19:53.823 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:19:53.827 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:19:53.827 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:19:53.962 INFO:teuthology.orchestra.run.smithi016.stdout:generic/076 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:19:53.966 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/076 2023-12-08T09:19:53.966 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/076 2023-12-08T09:19:53.966 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:19:53.969 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:19:53.978 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:19:53.978 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2023-12-08T09:19:54.151 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:19:54.155 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:19:54.155 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:00.674 INFO:teuthology.orchestra.run.smithi016.stdout:generic/077 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:20:00.678 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/077 2023-12-08T09:20:00.678 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/077 2023-12-08T09:20:00.678 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:20:00.680 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:00.701 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:20:00.701 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2023-12-08T09:20:00.980 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:20:00.984 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:20:00.984 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:01.228 INFO:teuthology.orchestra.run.smithi016.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2023-12-08T09:20:01.232 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/078 2023-12-08T09:20:01.232 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/078 2023-12-08T09:20:01.232 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:20:01.235 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:01.244 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:20:01.244 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2023-12-08T09:20:01.732 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:20:01.739 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:20:01.739 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:01.939 INFO:teuthology.orchestra.run.smithi016.stdout:generic/079 [not run] file system doesn't support chattr +ia 2023-12-08T09:20:01.942 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/079 2023-12-08T09:20:01.943 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/079 2023-12-08T09:20:01.943 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:20:01.945 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:01.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:20:01.955 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2023-12-08T09:20:02.128 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:20:02.132 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:20:02.133 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:04.301 INFO:teuthology.orchestra.run.smithi016.stdout:generic/080 2s 2023-12-08T09:20:04.306 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/080 2023-12-08T09:20:04.306 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:20:04.309 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:04.318 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:20:04.318 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2023-12-08T09:20:04.497 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:20:04.501 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:20:04.501 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:04.618 INFO:teuthology.orchestra.run.smithi016.stdout:generic/081 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:20:04.622 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/081 2023-12-08T09:20:04.622 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/081 2023-12-08T09:20:04.622 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:20:04.624 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:04.633 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:20:04.633 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2023-12-08T09:20:04.809 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:20:04.813 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:20:04.813 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:04.939 INFO:teuthology.orchestra.run.smithi016.stdout:generic/082 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:20:04.943 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/082 2023-12-08T09:20:04.943 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/082 2023-12-08T09:20:04.944 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:20:04.945 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:04.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:20:04.954 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2023-12-08T09:20:05.127 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:20:05.132 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:20:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:05.241 INFO:teuthology.orchestra.run.smithi016.stdout:generic/083 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:20:05.245 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/083 2023-12-08T09:20:05.245 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/083 2023-12-08T09:20:05.245 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:20:05.248 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:05.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:20:05.267 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2023-12-08T09:20:05.444 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:20:05.448 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:20:05.448 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:05.560 INFO:teuthology.orchestra.run.smithi016.stdout:generic/084 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:20:05.564 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/084 2023-12-08T09:20:05.564 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/084 2023-12-08T09:20:05.564 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:20:05.565 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:05.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:20:05.576 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2023-12-08T09:20:05.749 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:20:05.754 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:20:05.754 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:05.980 INFO:teuthology.orchestra.run.smithi016.stdout:generic/085 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:20:05.983 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/085 2023-12-08T09:20:05.984 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/085 2023-12-08T09:20:05.984 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:20:05.986 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:05.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:20:05.995 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2023-12-08T09:20:06.167 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:20:06.171 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:20:06.171 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:12.886 INFO:teuthology.orchestra.run.smithi016.stdout:generic/086 6s 2023-12-08T09:20:13.033 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/086 2023-12-08T09:20:13.033 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:20:13.033 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:13.317 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:20:13.317 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2023-12-08T09:20:15.670 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:20:15.674 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:20:15.674 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:16.197 INFO:teuthology.orchestra.run.smithi016.stdout:generic/087 1s 2023-12-08T09:20:16.203 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/087 2023-12-08T09:20:16.203 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:20:16.205 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:16.214 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:20:16.214 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2023-12-08T09:20:16.677 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:20:16.681 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:20:16.682 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:16.945 INFO:teuthology.orchestra.run.smithi016.stdout:generic/088 0s 2023-12-08T09:20:16.950 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/088 2023-12-08T09:20:16.951 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:20:16.953 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:20:16.961 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:20:16.962 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2023-12-08T09:20:17.440 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:20:17.444 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:20:17.444 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:21:53.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:21:53.508 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:21:53.595 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T09:21:53.593+0000 7eff8ec05640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 207725) UID: 0 2023-12-08T09:21:53.595 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T09:21:53.593+0000 7ff25129f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 207725) UID: 0 2023-12-08T09:21:53.595 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T09:21:53.593+0000 7fdb0d8f4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 207725) UID: 0 2023-12-08T09:21:53.595 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T09:21:53.593+0000 7f6d0f17a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 207725) UID: 0 2023-12-08T09:21:55.783 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T09:21:55.781+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 97960) UID: 0 2023-12-08T09:21:55.783 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T09:21:55.781+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 97960) UID: 0 2023-12-08T09:21:55.784 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T09:21:55.781+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 97960) UID: 0 2023-12-08T09:21:55.784 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T09:21:55.781+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 97960) UID: 0 2023-12-08T09:28:26.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:28:26.905 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:28:27.146 INFO:tasks.ceph.mds.b.smithi112.stderr:2023-12-08T09:28:27.144+0000 7f0cd7b54640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 97987) UID: 0 2023-12-08T09:28:27.147 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-12-08T09:28:27.144+0000 7f52ecb1a640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 97987) UID: 0 2023-12-08T09:34:56.092 INFO:teuthology.orchestra.run.smithi016.stdout:generic/089 879s 2023-12-08T09:34:56.097 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/089 2023-12-08T09:34:56.097 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:34:56.099 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:34:56.109 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:34:56.110 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/090) 2023-12-08T09:34:56.560 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:34:56.565 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:34:56.566 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:34:57.511 INFO:teuthology.orchestra.run.smithi016.stdout:generic/090 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:34:57.515 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/090 2023-12-08T09:34:57.516 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/090 2023-12-08T09:34:57.516 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:34:57.518 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:34:57.530 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:34:57.530 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/091) 2023-12-08T09:34:57.711 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:34:57.716 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:34:57.716 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:02.892 INFO:teuthology.orchestra.run.smithi016.stdout:generic/091 125s 2023-12-08T09:37:02.897 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/091 2023-12-08T09:37:02.897 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:02.908 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:02.908 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/092) 2023-12-08T09:37:03.324 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:03.328 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:03.328 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:03.501 INFO:teuthology.orchestra.run.smithi016.stdout:generic/092 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-12-08T09:37:03.504 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/092 2023-12-08T09:37:03.504 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/092 2023-12-08T09:37:03.504 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:03.507 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:03.515 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:03.516 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/093) 2023-12-08T09:37:03.955 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:03.961 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:03.961 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:04.701 INFO:teuthology.orchestra.run.smithi016.stdout:generic/093 1s 2023-12-08T09:37:04.706 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/093 2023-12-08T09:37:04.707 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:04.709 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:04.718 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:04.718 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/094) 2023-12-08T09:37:05.150 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:05.154 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:05.154 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:05.263 INFO:teuthology.orchestra.run.smithi016.stdout:generic/094 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:37:05.266 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/094 2023-12-08T09:37:05.266 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/094 2023-12-08T09:37:05.266 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:05.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:05.278 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:05.278 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/095) 2023-12-08T09:37:05.452 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:05.456 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:05.456 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:05.565 INFO:teuthology.orchestra.run.smithi016.stdout:generic/095 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:37:05.568 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/095 2023-12-08T09:37:05.568 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/095 2023-12-08T09:37:05.569 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:05.571 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:05.579 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:05.579 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/096) 2023-12-08T09:37:05.752 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:05.756 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:05.756 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:05.864 INFO:teuthology.orchestra.run.smithi016.stdout:generic/096 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:37:05.867 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/096 2023-12-08T09:37:05.868 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/096 2023-12-08T09:37:05.868 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:05.870 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:05.879 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:05.879 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/097) 2023-12-08T09:37:06.050 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:06.054 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:06.054 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:06.659 INFO:teuthology.orchestra.run.smithi016.stdout:generic/097 0s 2023-12-08T09:37:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/097 2023-12-08T09:37:06.665 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:06.668 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:06.675 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:06.676 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/098) 2023-12-08T09:37:06.848 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:06.852 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:06.852 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:06.961 INFO:teuthology.orchestra.run.smithi016.stdout:generic/098 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:37:06.965 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/098 2023-12-08T09:37:06.965 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/098 2023-12-08T09:37:06.965 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:06.967 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:06.975 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:06.976 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2023-12-08T09:37:07.148 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:07.153 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:07.153 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:07.309 INFO:teuthology.orchestra.run.smithi016.stdout:generic/099 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-12-08T09:37:07.312 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/099 2023-12-08T09:37:07.312 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/099 2023-12-08T09:37:07.312 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:07.315 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:07.323 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:07.323 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/100) 2023-12-08T09:37:07.497 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:07.502 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:07.502 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:17.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:37:17.909 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:37:18.134 INFO:tasks.ceph.mds.b.smithi112.stderr:2023-12-08T09:37:18.132+0000 7f0cd7b54640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 98016) UID: 0 2023-12-08T09:37:18.134 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-12-08T09:37:18.132+0000 7f52ecb1a640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 98016) UID: 0 2023-12-08T09:37:34.996 INFO:teuthology.orchestra.run.smithi016.stdout:generic/100 27s 2023-12-08T09:37:35.002 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/100 2023-12-08T09:37:35.002 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:35.004 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:35.019 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:35.020 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/101) 2023-12-08T09:37:35.355 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:35.375 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:35.375 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:35.596 INFO:teuthology.orchestra.run.smithi016.stdout:generic/101 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:37:35.601 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/101 2023-12-08T09:37:35.601 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/101 2023-12-08T09:37:35.601 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:35.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:35.612 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:35.612 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/102) 2023-12-08T09:37:35.809 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:35.813 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:35.813 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:35.926 INFO:teuthology.orchestra.run.smithi016.stdout:generic/102 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:37:35.930 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/102 2023-12-08T09:37:35.930 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/102 2023-12-08T09:37:35.930 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:35.932 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:35.941 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:35.941 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/103) 2023-12-08T09:37:36.120 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:36.124 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:36.125 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:36.254 INFO:teuthology.orchestra.run.smithi016.stdout:generic/103 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:37:36.258 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/103 2023-12-08T09:37:36.258 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/103 2023-12-08T09:37:36.258 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:36.260 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:36.269 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:36.269 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/104) 2023-12-08T09:37:36.444 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:36.448 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:36.448 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:36.570 INFO:teuthology.orchestra.run.smithi016.stdout:generic/104 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:37:36.574 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/104 2023-12-08T09:37:36.574 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/104 2023-12-08T09:37:36.574 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:36.576 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:36.585 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:36.585 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/105) 2023-12-08T09:37:36.759 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:36.763 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:36.763 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:36.872 INFO:teuthology.orchestra.run.smithi016.stdout:generic/105 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:37:36.875 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/105 2023-12-08T09:37:36.875 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/105 2023-12-08T09:37:36.875 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:36.877 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:36.886 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:36.886 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/106) 2023-12-08T09:37:37.061 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:37.065 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:37.065 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:37.187 INFO:teuthology.orchestra.run.smithi016.stdout:generic/106 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:37:37.191 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/106 2023-12-08T09:37:37.191 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/106 2023-12-08T09:37:37.191 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:37.193 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:37.202 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:37.202 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/107) 2023-12-08T09:37:37.376 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:37.380 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:37.380 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:37.499 INFO:teuthology.orchestra.run.smithi016.stdout:generic/107 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:37:37.502 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/107 2023-12-08T09:37:37.503 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/107 2023-12-08T09:37:37.503 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:37.505 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:37.513 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:37.513 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/108) 2023-12-08T09:37:37.685 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:37.689 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:37.689 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:37.946 INFO:teuthology.orchestra.run.smithi016.stdout:generic/108 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:37:37.949 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/108 2023-12-08T09:37:37.950 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/108 2023-12-08T09:37:37.950 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:37.952 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:37.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:37.960 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/109) 2023-12-08T09:37:38.132 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:38.136 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:38.136 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:38.243 INFO:teuthology.orchestra.run.smithi016.stdout:generic/109 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:37:38.247 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/109 2023-12-08T09:37:38.247 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/109 2023-12-08T09:37:38.247 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:38.249 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:38.258 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:38.258 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/110) 2023-12-08T09:37:38.430 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:38.434 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:38.434 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:38.586 INFO:teuthology.orchestra.run.smithi016.stdout:generic/110 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:37:38.590 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/110 2023-12-08T09:37:38.590 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/110 2023-12-08T09:37:38.590 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:38.592 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:38.601 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:38.601 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/111) 2023-12-08T09:37:39.031 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:39.035 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:39.035 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:39.195 INFO:teuthology.orchestra.run.smithi016.stdout:generic/111 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:37:39.198 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/111 2023-12-08T09:37:39.198 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/111 2023-12-08T09:37:39.198 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:37:39.200 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:37:39.209 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:37:39.210 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/112) 2023-12-08T09:37:39.645 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:37:39.649 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:37:39.649 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:38:23.048 INFO:teuthology.orchestra.run.smithi016.stdout:generic/112 44s 2023-12-08T09:38:23.052 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/112 2023-12-08T09:38:23.052 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:38:23.054 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:38:23.068 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:38:23.069 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/113) 2023-12-08T09:38:23.535 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:38:23.539 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:38:23.539 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:05.777 INFO:teuthology.orchestra.run.smithi016.stdout:generic/113 42s 2023-12-08T09:39:05.782 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/113 2023-12-08T09:39:05.782 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:39:05.784 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:05.794 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:39:05.795 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/114) 2023-12-08T09:39:06.226 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:39:06.230 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:39:06.230 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:06.414 INFO:teuthology.orchestra.run.smithi016.stdout:generic/114 [not run] device block size: 4096 greater than 512 2023-12-08T09:39:06.418 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/114 2023-12-08T09:39:06.418 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/114 2023-12-08T09:39:06.418 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:39:06.420 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:06.429 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:39:06.430 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/115) 2023-12-08T09:39:06.601 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:39:06.605 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:39:06.605 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:06.796 INFO:teuthology.orchestra.run.smithi016.stdout:generic/115 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:39:06.800 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/115 2023-12-08T09:39:06.800 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/115 2023-12-08T09:39:06.800 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:39:06.802 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:06.811 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:39:06.811 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/116) 2023-12-08T09:39:07.249 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:39:07.253 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:39:07.253 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:07.410 INFO:teuthology.orchestra.run.smithi016.stdout:generic/116 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:39:07.414 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/116 2023-12-08T09:39:07.414 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/116 2023-12-08T09:39:07.414 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:39:07.416 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:07.426 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:39:07.426 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/117) 2023-12-08T09:39:07.862 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:39:07.866 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:39:07.866 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:07.976 INFO:teuthology.orchestra.run.smithi016.stdout:generic/117 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:39:07.979 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/117 2023-12-08T09:39:07.979 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/117 2023-12-08T09:39:07.980 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:39:07.982 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:07.990 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:39:07.991 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/118) 2023-12-08T09:39:08.166 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:39:08.170 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:39:08.170 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:08.322 INFO:teuthology.orchestra.run.smithi016.stdout:generic/118 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:39:08.325 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/118 2023-12-08T09:39:08.325 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/118 2023-12-08T09:39:08.325 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:39:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:08.336 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:39:08.337 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/119) 2023-12-08T09:39:08.772 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:39:08.776 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:39:08.776 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:08.931 INFO:teuthology.orchestra.run.smithi016.stdout:generic/119 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:39:08.935 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/119 2023-12-08T09:39:08.935 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/119 2023-12-08T09:39:08.935 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:39:08.937 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:08.946 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:39:08.947 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/120) 2023-12-08T09:39:09.390 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:39:09.394 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:39:09.395 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:09.504 INFO:teuthology.orchestra.run.smithi016.stdout:generic/120 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:39:09.507 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/120 2023-12-08T09:39:09.507 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/120 2023-12-08T09:39:09.507 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:39:09.510 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:09.519 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:39:09.519 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/121) 2023-12-08T09:39:09.697 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:39:09.701 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:39:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:09.862 INFO:teuthology.orchestra.run.smithi016.stdout:generic/121 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-12-08T09:39:09.866 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/121 2023-12-08T09:39:09.866 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/121 2023-12-08T09:39:09.866 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:39:09.868 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:09.877 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:39:09.877 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/122) 2023-12-08T09:39:10.302 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:39:10.306 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:39:10.306 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:10.466 INFO:teuthology.orchestra.run.smithi016.stdout:generic/122 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-12-08T09:39:10.469 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/122 2023-12-08T09:39:10.470 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/122 2023-12-08T09:39:10.470 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:39:10.472 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:10.481 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:39:10.481 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/123) 2023-12-08T09:39:10.925 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:39:10.929 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:39:10.930 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:11.376 INFO:teuthology.orchestra.run.smithi016.stdout:generic/123 1s 2023-12-08T09:39:11.381 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/123 2023-12-08T09:39:11.382 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:39:11.384 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:11.393 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:39:11.394 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/124) 2023-12-08T09:39:11.828 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:39:11.832 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:39:11.832 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:11.945 INFO:teuthology.orchestra.run.smithi016.stdout:generic/124 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:39:11.949 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/124 2023-12-08T09:39:11.949 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/124 2023-12-08T09:39:11.949 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:39:11.951 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:39:11.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:39:11.961 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/125) 2023-12-08T09:39:12.133 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:39:12.137 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:39:12.138 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:40:13.486 INFO:teuthology.orchestra.run.smithi016.stdout:generic/125 61s 2023-12-08T09:40:13.492 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/125 2023-12-08T09:40:13.492 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:40:13.494 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:40:13.526 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:40:13.527 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/126) 2023-12-08T09:40:13.953 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:40:13.957 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:40:13.957 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:40:14.500 INFO:teuthology.orchestra.run.smithi016.stdout:generic/126 - output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/126.out.bad) 2023-12-08T09:40:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/126.out 2023-12-08 08:39:53.756917984 +0000 2023-12-08T09:40:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/126.out.bad 2023-12-08 09:40:14.497292032 +0000 2023-12-08T09:40:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -1,7 +1,7 @@ 2023-12-08T09:40:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: QA output created by 126 2023-12-08T09:40:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: -x a 001 file owned by (99/99) as user/group(12/100) PASS 2023-12-08T09:40:14.502 INFO:teuthology.orchestra.run.smithi016.stdout: -x a 010 file owned by (99/99) as user/group(200/99) PASS 2023-12-08T09:40:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: -x a 100 file owned by (99/99) as user/group(99/500) PASS 2023-12-08T09:40:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: +x a 001 file owned by (99/99) as user/group(12/100) FAIL 2023-12-08T09:40:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: +x a 010 file owned by (99/99) as user/group(200/99) FAIL 2023-12-08T09:40:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: +x a 100 file owned by (99/99) as user/group(99/500) FAIL 2023-12-08T09:40:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: w a 002 file owned by (99/99) as user/group(12/100) PASS 2023-12-08T09:40:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: w a 020 file owned by (99/99) as user/group(200/99) PASS 2023-12-08T09:40:14.503 INFO:teuthology.orchestra.run.smithi016.stdout: w a 200 file owned by (99/99) as user/group(99/500) PASS 2023-12-08T09:40:14.505 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/126 2023-12-08T09:40:14.505 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/126 2023-12-08T09:40:14.505 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T09:40:14.507 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:40:14.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T09:40:14.517 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T09:40:14.517 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T09:40:14.517 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/127) 2023-12-08T09:40:14.971 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:40:14.975 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:40:14.975 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:41:23.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:41:23.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:41:54.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:41:54.069 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:42:24.123 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:42:24.126 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:42:54.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:42:54.181 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:43:24.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:43:24.495 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:43:54.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:43:54.610 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:44:24.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:44:24.709 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:44:55.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:44:55.089 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:45:25.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:45:25.154 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:45:55.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:45:55.383 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:46:25.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:46:25.467 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:46:55.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:46:55.534 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:47:25.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:47:25.590 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:47:55.833 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:47:55.836 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:48:26.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:48:26.146 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:48:56.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:48:56.201 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:48:56.233 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T09:48:56.231+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98364) UID: 0 2023-12-08T09:48:56.233 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T09:48:56.231+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98364) UID: 0 2023-12-08T09:48:56.233 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T09:48:56.231+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98364) UID: 0 2023-12-08T09:48:56.234 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T09:48:56.231+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98364) UID: 0 2023-12-08T09:51:47.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:51:47.802 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:51:47.872 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T09:51:47.870+0000 7f6d0f17a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 241586) UID: 0 2023-12-08T09:51:47.872 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T09:51:47.870+0000 7ff25129f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 241586) UID: 0 2023-12-08T09:51:47.872 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T09:51:47.870+0000 7eff8ec05640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 241586) UID: 0 2023-12-08T09:51:47.873 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T09:51:47.870+0000 7fdb0d8f4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 241586) UID: 0 2023-12-08T09:52:20.544 INFO:teuthology.orchestra.run.smithi016.stdout:generic/127 726s 2023-12-08T09:52:20.793 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/127 2023-12-08T09:52:20.793 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:52:20.808 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:52:20.900 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:52:20.900 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/128) 2023-12-08T09:52:21.675 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:52:21.679 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:52:21.679 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:52:21.855 INFO:teuthology.orchestra.run.smithi016.stdout:generic/128 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:52:21.859 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/128 2023-12-08T09:52:21.859 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/128 2023-12-08T09:52:21.859 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:52:21.861 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:52:21.870 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:52:21.870 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/129) 2023-12-08T09:52:22.045 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:52:22.049 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:52:22.049 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:52:22.158 INFO:teuthology.orchestra.run.smithi016.stdout:generic/129 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:52:22.162 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/129 2023-12-08T09:52:22.162 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/129 2023-12-08T09:52:22.162 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:52:22.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:52:22.173 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:52:22.173 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/130) 2023-12-08T09:52:22.350 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:52:22.353 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:52:22.354 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:52:22.466 INFO:teuthology.orchestra.run.smithi016.stdout:generic/130 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:52:22.470 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/130 2023-12-08T09:52:22.470 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/130 2023-12-08T09:52:22.470 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:52:22.472 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:52:22.481 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:52:22.481 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/131) 2023-12-08T09:52:22.658 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:52:22.662 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:52:22.663 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:52:24.047 INFO:teuthology.orchestra.run.smithi016.stdout:generic/131 2s 2023-12-08T09:52:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/131 2023-12-08T09:52:24.052 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:52:24.055 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:52:24.064 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:52:24.064 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/132) 2023-12-08T09:52:24.553 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:52:24.557 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:52:24.557 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:52:24.668 INFO:teuthology.orchestra.run.smithi016.stdout:generic/132 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:52:24.671 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/132 2023-12-08T09:52:24.671 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/132 2023-12-08T09:52:24.671 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:52:24.674 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:52:24.683 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:52:24.683 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/133) 2023-12-08T09:52:24.858 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:52:24.863 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:52:24.863 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:54:54.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:54:54.867 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:55:24.928 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:55:24.932 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:55:55.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:55:55.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:56:19.192 INFO:teuthology.orchestra.run.smithi016.stdout:generic/133 235s 2023-12-08T09:56:19.197 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/133 2023-12-08T09:56:19.197 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:19.200 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:19.208 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:19.209 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/134) 2023-12-08T09:56:19.636 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:19.641 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:19.641 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:19.812 INFO:teuthology.orchestra.run.smithi016.stdout:generic/134 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:19.815 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/134 2023-12-08T09:56:19.815 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/134 2023-12-08T09:56:19.815 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:19.818 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:19.826 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:19.827 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/135) 2023-12-08T09:56:20.311 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:20.318 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:20.318 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:20.487 INFO:teuthology.orchestra.run.smithi016.stdout:generic/135 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:20.490 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/135 2023-12-08T09:56:20.491 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/135 2023-12-08T09:56:20.491 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:20.493 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:20.502 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:20.502 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/136) 2023-12-08T09:56:20.676 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:20.680 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:20.680 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:20.836 INFO:teuthology.orchestra.run.smithi016.stdout:generic/136 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:20.839 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/136 2023-12-08T09:56:20.839 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/136 2023-12-08T09:56:20.839 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:20.841 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:20.850 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:20.850 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/137) 2023-12-08T09:56:21.282 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:21.286 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:21.287 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:21.441 INFO:teuthology.orchestra.run.smithi016.stdout:generic/137 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:21.445 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/137 2023-12-08T09:56:21.445 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/137 2023-12-08T09:56:21.445 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:21.447 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:21.456 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:21.456 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/138) 2023-12-08T09:56:21.904 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:21.908 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:21.908 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:22.070 INFO:teuthology.orchestra.run.smithi016.stdout:generic/138 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:22.074 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/138 2023-12-08T09:56:22.074 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/138 2023-12-08T09:56:22.074 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:22.076 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:22.085 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:22.085 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/139) 2023-12-08T09:56:22.514 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:22.519 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:22.519 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:22.677 INFO:teuthology.orchestra.run.smithi016.stdout:generic/139 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:22.681 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/139 2023-12-08T09:56:22.681 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/139 2023-12-08T09:56:22.681 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:22.683 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:22.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:22.694 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/140) 2023-12-08T09:56:23.120 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:23.125 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:23.125 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:23.277 INFO:teuthology.orchestra.run.smithi016.stdout:generic/140 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:23.281 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/140 2023-12-08T09:56:23.281 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/140 2023-12-08T09:56:23.281 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:23.284 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:23.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:23.292 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/141) 2023-12-08T09:56:23.711 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:23.716 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:23.716 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:23.826 INFO:teuthology.orchestra.run.smithi016.stdout:generic/141 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/141 2023-12-08T09:56:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/141 2023-12-08T09:56:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:23.841 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:23.841 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/142) 2023-12-08T09:56:24.013 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:24.018 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:24.018 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:24.171 INFO:teuthology.orchestra.run.smithi016.stdout:generic/142 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:24.175 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/142 2023-12-08T09:56:24.175 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/142 2023-12-08T09:56:24.175 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:24.181 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:24.185 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:24.186 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/143) 2023-12-08T09:56:24.615 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:24.619 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:24.619 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:24.774 INFO:teuthology.orchestra.run.smithi016.stdout:generic/143 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:24.777 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/143 2023-12-08T09:56:24.777 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/143 2023-12-08T09:56:24.777 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:24.780 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:24.789 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:24.789 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/144) 2023-12-08T09:56:25.219 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:25.223 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:25.224 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:25.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:56:25.236 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:56:25.377 INFO:teuthology.orchestra.run.smithi016.stdout:generic/144 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:25.381 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/144 2023-12-08T09:56:25.381 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/144 2023-12-08T09:56:25.381 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:25.383 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:25.392 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:25.392 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/145) 2023-12-08T09:56:25.810 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:25.815 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:25.815 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:25.974 INFO:teuthology.orchestra.run.smithi016.stdout:generic/145 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:25.978 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/145 2023-12-08T09:56:25.978 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/145 2023-12-08T09:56:25.978 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:25.980 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:25.989 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:25.989 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/146) 2023-12-08T09:56:26.426 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:26.430 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:26.430 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:26.585 INFO:teuthology.orchestra.run.smithi016.stdout:generic/146 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:26.588 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/146 2023-12-08T09:56:26.588 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/146 2023-12-08T09:56:26.588 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:26.591 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:26.599 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:26.600 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/147) 2023-12-08T09:56:27.017 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:27.022 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:27.022 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:27.178 INFO:teuthology.orchestra.run.smithi016.stdout:generic/147 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:27.181 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/147 2023-12-08T09:56:27.182 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/147 2023-12-08T09:56:27.182 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:27.184 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:27.192 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:27.193 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/148) 2023-12-08T09:56:27.627 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:27.631 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:27.631 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:27.786 INFO:teuthology.orchestra.run.smithi016.stdout:generic/148 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:27.790 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/148 2023-12-08T09:56:27.790 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/148 2023-12-08T09:56:27.790 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:27.792 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:27.801 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:27.801 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/149) 2023-12-08T09:56:28.225 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:28.229 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:28.229 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:28.384 INFO:teuthology.orchestra.run.smithi016.stdout:generic/149 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:28.389 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/149 2023-12-08T09:56:28.389 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/149 2023-12-08T09:56:28.389 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:28.390 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:28.440 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:28.440 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/150) 2023-12-08T09:56:28.831 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:28.835 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:28.835 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:28.990 INFO:teuthology.orchestra.run.smithi016.stdout:generic/150 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:28.993 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/150 2023-12-08T09:56:28.993 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/150 2023-12-08T09:56:28.994 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:28.997 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:29.015 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:29.016 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/151) 2023-12-08T09:56:29.439 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:29.443 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:29.443 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:29.597 INFO:teuthology.orchestra.run.smithi016.stdout:generic/151 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:29.600 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/151 2023-12-08T09:56:29.601 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/151 2023-12-08T09:56:29.601 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:29.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:29.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:29.612 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/152) 2023-12-08T09:56:30.041 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:30.045 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:30.045 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:30.197 INFO:teuthology.orchestra.run.smithi016.stdout:generic/152 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:30.201 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/152 2023-12-08T09:56:30.201 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/152 2023-12-08T09:56:30.201 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:30.203 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:30.211 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:30.212 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/153) 2023-12-08T09:56:30.650 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:30.653 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:30.654 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:30.806 INFO:teuthology.orchestra.run.smithi016.stdout:generic/153 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:30.810 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/153 2023-12-08T09:56:30.810 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/153 2023-12-08T09:56:30.810 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:30.812 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:30.821 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:30.821 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/154) 2023-12-08T09:56:31.258 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:31.262 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:31.262 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:31.415 INFO:teuthology.orchestra.run.smithi016.stdout:generic/154 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:31.419 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/154 2023-12-08T09:56:31.419 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/154 2023-12-08T09:56:31.419 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:31.421 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:31.429 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:31.430 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/155) 2023-12-08T09:56:31.864 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:31.868 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:31.868 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:32.022 INFO:teuthology.orchestra.run.smithi016.stdout:generic/155 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:32.025 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/155 2023-12-08T09:56:32.025 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/155 2023-12-08T09:56:32.025 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:32.027 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:32.082 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:32.083 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/156) 2023-12-08T09:56:32.464 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:32.468 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:32.468 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:32.622 INFO:teuthology.orchestra.run.smithi016.stdout:generic/156 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/156 2023-12-08T09:56:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/156 2023-12-08T09:56:32.625 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:32.628 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:32.674 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:32.674 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/157) 2023-12-08T09:56:33.069 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:33.073 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:33.385 INFO:teuthology.orchestra.run.smithi016.stdout:generic/157 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:33.388 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/157 2023-12-08T09:56:33.388 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/157 2023-12-08T09:56:33.389 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:33.399 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:33.399 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/158) 2023-12-08T09:56:33.833 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:33.837 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:34.013 INFO:teuthology.orchestra.run.smithi016.stdout:generic/158 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/158 2023-12-08T09:56:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/158 2023-12-08T09:56:34.017 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:34.027 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:34.027 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/159) 2023-12-08T09:56:34.452 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:34.456 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:34.457 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:34.634 INFO:teuthology.orchestra.run.smithi016.stdout:generic/159 [not run] file system doesn't support chattr +i 2023-12-08T09:56:34.638 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/159 2023-12-08T09:56:34.638 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/159 2023-12-08T09:56:34.638 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:34.640 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:34.657 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:34.657 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/160) 2023-12-08T09:56:34.830 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:34.834 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:34.834 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:34.979 INFO:teuthology.orchestra.run.smithi016.stdout:generic/160 [not run] file system doesn't support chattr +i 2023-12-08T09:56:34.983 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/160 2023-12-08T09:56:34.983 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/160 2023-12-08T09:56:34.983 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:34.985 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:34.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:34.994 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/161) 2023-12-08T09:56:35.168 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:35.172 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:35.173 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:35.283 INFO:teuthology.orchestra.run.smithi016.stdout:generic/161 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:35.286 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/161 2023-12-08T09:56:35.286 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/161 2023-12-08T09:56:35.286 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:35.289 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:35.297 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:35.298 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/162) 2023-12-08T09:56:35.472 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:35.476 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:35.476 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:35.586 INFO:teuthology.orchestra.run.smithi016.stdout:generic/162 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:35.590 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/162 2023-12-08T09:56:35.590 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/162 2023-12-08T09:56:35.590 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:35.592 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:35.610 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:35.611 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/163) 2023-12-08T09:56:35.784 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:35.788 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:35.788 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:35.897 INFO:teuthology.orchestra.run.smithi016.stdout:generic/163 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:35.901 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/163 2023-12-08T09:56:35.901 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/163 2023-12-08T09:56:35.901 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:35.903 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:35.912 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:35.912 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/164) 2023-12-08T09:56:36.084 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:36.088 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:36.088 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:36.203 INFO:teuthology.orchestra.run.smithi016.stdout:generic/164 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:36.206 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/164 2023-12-08T09:56:36.206 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/164 2023-12-08T09:56:36.206 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:36.208 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:36.217 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:36.217 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/165) 2023-12-08T09:56:36.390 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:36.394 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:36.394 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:36.503 INFO:teuthology.orchestra.run.smithi016.stdout:generic/165 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:36.507 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/165 2023-12-08T09:56:36.507 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/165 2023-12-08T09:56:36.507 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:36.509 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:36.518 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:36.518 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/166) 2023-12-08T09:56:36.692 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:36.696 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:36.696 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:36.811 INFO:teuthology.orchestra.run.smithi016.stdout:generic/166 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:36.815 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/166 2023-12-08T09:56:36.815 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/166 2023-12-08T09:56:36.815 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:36.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:36.825 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:36.826 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/167) 2023-12-08T09:56:37.001 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:37.005 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:37.005 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:37.114 INFO:teuthology.orchestra.run.smithi016.stdout:generic/167 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:37.118 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/167 2023-12-08T09:56:37.118 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/167 2023-12-08T09:56:37.118 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:37.120 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:37.143 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:37.143 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/168) 2023-12-08T09:56:37.315 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:37.320 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:37.320 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:37.430 INFO:teuthology.orchestra.run.smithi016.stdout:generic/168 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:37.434 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/168 2023-12-08T09:56:37.434 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/168 2023-12-08T09:56:37.434 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:37.436 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:37.445 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:37.445 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/169) 2023-12-08T09:56:37.620 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:37.626 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:37.626 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:37.738 INFO:teuthology.orchestra.run.smithi016.stdout:generic/169 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:37.742 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/169 2023-12-08T09:56:37.742 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/169 2023-12-08T09:56:37.742 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:37.744 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:37.753 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:37.753 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/170) 2023-12-08T09:56:37.928 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:37.932 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:37.932 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:38.041 INFO:teuthology.orchestra.run.smithi016.stdout:generic/170 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/170 2023-12-08T09:56:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/170 2023-12-08T09:56:38.045 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:38.047 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:38.056 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:38.056 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/171) 2023-12-08T09:56:38.230 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:38.234 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:38.234 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:38.377 INFO:teuthology.orchestra.run.smithi016.stdout:generic/171 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:38.381 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/171 2023-12-08T09:56:38.381 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/171 2023-12-08T09:56:38.381 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:38.384 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:38.392 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:38.392 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/172) 2023-12-08T09:56:38.566 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:38.571 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:38.571 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:38.684 INFO:teuthology.orchestra.run.smithi016.stdout:generic/172 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:38.688 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/172 2023-12-08T09:56:38.688 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/172 2023-12-08T09:56:38.688 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:38.690 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:38.699 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:38.699 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/173) 2023-12-08T09:56:38.877 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:38.881 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:38.881 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:39.001 INFO:teuthology.orchestra.run.smithi016.stdout:generic/173 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:39.005 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/173 2023-12-08T09:56:39.005 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/173 2023-12-08T09:56:39.005 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:39.007 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:39.016 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:39.016 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/174) 2023-12-08T09:56:39.190 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:39.194 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:39.194 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:39.346 INFO:teuthology.orchestra.run.smithi016.stdout:generic/174 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/174 2023-12-08T09:56:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/174 2023-12-08T09:56:39.350 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:39.352 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:39.361 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:39.361 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/175) 2023-12-08T09:56:39.535 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:39.539 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:39.539 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:39.650 INFO:teuthology.orchestra.run.smithi016.stdout:generic/175 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:39.654 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/175 2023-12-08T09:56:39.654 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/175 2023-12-08T09:56:39.654 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:39.656 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:39.678 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:39.679 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/176) 2023-12-08T09:56:39.851 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:39.855 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:39.855 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:39.973 INFO:teuthology.orchestra.run.smithi016.stdout:generic/176 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:39.977 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/176 2023-12-08T09:56:39.977 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/176 2023-12-08T09:56:39.977 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:39.979 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:39.987 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:39.988 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/177) 2023-12-08T09:56:40.161 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:40.165 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:40.166 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:40.322 INFO:teuthology.orchestra.run.smithi016.stdout:generic/177 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:40.326 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/177 2023-12-08T09:56:40.326 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/177 2023-12-08T09:56:40.326 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:40.329 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:40.337 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:40.338 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/178) 2023-12-08T09:56:40.510 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:40.514 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:40.515 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:40.675 INFO:teuthology.orchestra.run.smithi016.stdout:generic/178 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:40.679 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/178 2023-12-08T09:56:40.679 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/178 2023-12-08T09:56:40.679 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:40.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:40.689 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:40.690 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/179) 2023-12-08T09:56:41.122 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:41.126 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:41.126 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:41.278 INFO:teuthology.orchestra.run.smithi016.stdout:generic/179 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:41.281 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/179 2023-12-08T09:56:41.282 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/179 2023-12-08T09:56:41.282 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:41.284 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:41.315 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:41.316 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/180) 2023-12-08T09:56:41.722 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:41.724 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:41.724 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:41.887 INFO:teuthology.orchestra.run.smithi016.stdout:generic/180 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:41.890 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/180 2023-12-08T09:56:41.890 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/180 2023-12-08T09:56:41.891 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:41.893 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:41.901 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:41.901 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/181) 2023-12-08T09:56:42.339 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:42.340 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:42.340 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:42.493 INFO:teuthology.orchestra.run.smithi016.stdout:generic/181 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:42.497 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/181 2023-12-08T09:56:42.497 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/181 2023-12-08T09:56:42.497 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:42.500 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:42.508 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:42.508 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/182) 2023-12-08T09:56:42.924 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:42.928 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:42.928 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:43.083 INFO:teuthology.orchestra.run.smithi016.stdout:generic/182 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-12-08T09:56:43.088 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/182 2023-12-08T09:56:43.088 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/182 2023-12-08T09:56:43.088 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:43.089 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:43.098 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:43.099 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/183) 2023-12-08T09:56:43.529 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:43.533 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:43.533 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:43.643 INFO:teuthology.orchestra.run.smithi016.stdout:generic/183 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:43.646 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/183 2023-12-08T09:56:43.646 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/183 2023-12-08T09:56:43.646 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:43.649 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:43.657 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:43.658 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/184) 2023-12-08T09:56:43.831 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:43.835 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:43.835 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:44.003 INFO:teuthology.orchestra.run.smithi016.stdout:generic/184 [failed, exit status 1]- output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/184.out.bad) 2023-12-08T09:56:44.005 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/184.out 2023-12-08 08:39:53.761917912 +0000 2023-12-08T09:56:44.005 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/184.out.bad 2023-12-08 09:56:44.000023455 +0000 2023-12-08T09:56:44.005 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -1 +1,2 @@ 2023-12-08T09:56:44.005 INFO:teuthology.orchestra.run.smithi016.stdout: QA output created by 184 2023-12-08T09:56:44.005 INFO:teuthology.orchestra.run.smithi016.stdout: +/tmp/tmp.MkHH5XcxMMxfstests-dev/tests/generic/184: line 23: /tmp/tmp.hY9PAzp4zMtest/null: Permission denied 2023-12-08T09:56:44.007 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/184 2023-12-08T09:56:44.007 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/184 2023-12-08T09:56:44.007 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T09:56:44.010 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:44.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T09:56:44.018 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T09:56:44.019 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T09:56:44.019 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/185) 2023-12-08T09:56:44.192 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:44.196 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:44.196 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:44.305 INFO:teuthology.orchestra.run.smithi016.stdout:generic/185 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:44.310 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/185 2023-12-08T09:56:44.310 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/185 2023-12-08T09:56:44.310 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:44.311 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:44.341 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:44.342 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/186) 2023-12-08T09:56:44.517 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:44.521 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:44.521 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:44.629 INFO:teuthology.orchestra.run.smithi016.stdout:generic/186 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/186 2023-12-08T09:56:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/186 2023-12-08T09:56:44.633 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:44.635 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:44.644 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:44.644 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/187) 2023-12-08T09:56:44.817 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:44.821 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:44.821 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:44.930 INFO:teuthology.orchestra.run.smithi016.stdout:generic/187 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:44.933 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/187 2023-12-08T09:56:44.933 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/187 2023-12-08T09:56:44.934 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:44.936 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:44.944 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:44.944 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/188) 2023-12-08T09:56:45.119 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:45.123 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:45.123 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:45.241 INFO:teuthology.orchestra.run.smithi016.stdout:generic/188 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:45.244 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/188 2023-12-08T09:56:45.244 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/188 2023-12-08T09:56:45.244 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:45.247 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:45.255 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:45.255 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/189) 2023-12-08T09:56:45.431 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:45.435 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:45.435 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:45.544 INFO:teuthology.orchestra.run.smithi016.stdout:generic/189 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:45.548 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/189 2023-12-08T09:56:45.548 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/189 2023-12-08T09:56:45.548 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:45.550 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:45.558 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:45.559 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/190) 2023-12-08T09:56:45.732 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:45.736 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:45.736 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:45.846 INFO:teuthology.orchestra.run.smithi016.stdout:generic/190 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:45.850 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/190 2023-12-08T09:56:45.850 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/190 2023-12-08T09:56:45.850 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:45.852 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:45.872 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:45.873 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/191) 2023-12-08T09:56:46.046 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:46.049 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:46.049 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:46.157 INFO:teuthology.orchestra.run.smithi016.stdout:generic/191 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/191 2023-12-08T09:56:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/191 2023-12-08T09:56:46.161 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:46.163 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:46.172 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:46.172 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/192) 2023-12-08T09:56:46.346 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:46.350 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:46.350 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:46.467 INFO:teuthology.orchestra.run.smithi016.stdout:generic/192 [not run] atime not maintained by ceph-fuse 2023-12-08T09:56:46.470 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/192 2023-12-08T09:56:46.470 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/192 2023-12-08T09:56:46.470 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:46.472 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:46.481 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:46.481 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/193) 2023-12-08T09:56:46.658 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:46.662 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:46.662 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:47.503 INFO:teuthology.orchestra.run.smithi016.stdout:generic/193 - output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/193.out.bad) 2023-12-08T09:56:47.505 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/193.out 2023-12-08 08:39:53.761917912 +0000 2023-12-08T09:56:47.506 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/193.out.bad 2023-12-08 09:56:47.500973372 +0000 2023-12-08T09:56:47.506 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -24,7 +24,6 @@ 2023-12-08T09:56:47.506 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:47.506 INFO:teuthology.orchestra.run.smithi016.stdout: user: chmod a+r on qa_user owned file (should succeed) 2023-12-08T09:56:47.506 INFO:teuthology.orchestra.run.smithi016.stdout: user: chmod a+r on root owned file (should fail) 2023-12-08T09:56:47.506 INFO:teuthology.orchestra.run.smithi016.stdout: -chmod: changing permissions of 'test.root': Operation not permitted 2023-12-08T09:56:47.506 INFO:teuthology.orchestra.run.smithi016.stdout: check that the sgid bit is cleared 2023-12-08T09:56:47.506 INFO:teuthology.orchestra.run.smithi016.stdout: -rw-rw-rw- 2023-12-08T09:56:47.506 INFO:teuthology.orchestra.run.smithi016.stdout: check that suid bit is not cleared 2023-12-08T09:56:47.508 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/193 2023-12-08T09:56:47.508 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/193 2023-12-08T09:56:47.508 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T09:56:47.510 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:47.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T09:56:47.519 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T09:56:47.519 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T09:56:47.520 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/194) 2023-12-08T09:56:47.935 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:47.939 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:47.940 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:48.050 INFO:teuthology.orchestra.run.smithi016.stdout:generic/194 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:48.053 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/194 2023-12-08T09:56:48.053 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/194 2023-12-08T09:56:48.053 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:48.055 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:48.064 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:48.064 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/195) 2023-12-08T09:56:48.242 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:48.247 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:48.247 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:48.356 INFO:teuthology.orchestra.run.smithi016.stdout:generic/195 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:48.359 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/195 2023-12-08T09:56:48.359 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/195 2023-12-08T09:56:48.359 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:48.362 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:48.380 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:48.380 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/196) 2023-12-08T09:56:48.554 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:48.558 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:48.558 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:48.668 INFO:teuthology.orchestra.run.smithi016.stdout:generic/196 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:48.672 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/196 2023-12-08T09:56:48.672 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/196 2023-12-08T09:56:48.672 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:48.674 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:48.683 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:48.684 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/197) 2023-12-08T09:56:48.856 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:48.861 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:48.861 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:48.970 INFO:teuthology.orchestra.run.smithi016.stdout:generic/197 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:48.974 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/197 2023-12-08T09:56:48.974 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/197 2023-12-08T09:56:48.974 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:48.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:48.985 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:48.985 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/198) 2023-12-08T09:56:49.159 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:49.164 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:49.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:49.905 INFO:teuthology.orchestra.run.smithi016.stdout:generic/198 0s 2023-12-08T09:56:49.916 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/198 2023-12-08T09:56:49.916 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:49.918 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:49.927 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:49.927 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/199) 2023-12-08T09:56:50.366 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:50.371 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:50.371 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:50.523 INFO:teuthology.orchestra.run.smithi016.stdout:generic/199 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:50.527 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/199 2023-12-08T09:56:50.527 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/199 2023-12-08T09:56:50.527 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:50.529 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:50.548 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:50.549 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/200) 2023-12-08T09:56:50.725 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:50.729 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:50.729 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:50.840 INFO:teuthology.orchestra.run.smithi016.stdout:generic/200 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:50.843 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/200 2023-12-08T09:56:50.844 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/200 2023-12-08T09:56:50.844 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:50.846 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:50.854 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:50.854 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/201) 2023-12-08T09:56:51.026 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:51.030 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:51.030 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:51.139 INFO:teuthology.orchestra.run.smithi016.stdout:generic/201 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:51.143 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/201 2023-12-08T09:56:51.143 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/201 2023-12-08T09:56:51.143 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:51.146 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:51.153 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:51.154 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/202) 2023-12-08T09:56:51.326 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:51.330 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:51.330 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:51.439 INFO:teuthology.orchestra.run.smithi016.stdout:generic/202 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:51.443 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/202 2023-12-08T09:56:51.443 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/202 2023-12-08T09:56:51.443 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:51.445 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:51.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:51.454 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/203) 2023-12-08T09:56:51.627 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:51.631 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:51.631 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:51.741 INFO:teuthology.orchestra.run.smithi016.stdout:generic/203 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:51.745 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/203 2023-12-08T09:56:51.745 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/203 2023-12-08T09:56:51.745 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:51.747 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:51.755 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:51.756 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/204) 2023-12-08T09:56:51.929 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:51.933 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:51.934 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:53.087 INFO:teuthology.orchestra.run.smithi016.stdout:generic/204 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:53.091 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/204 2023-12-08T09:56:53.091 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/204 2023-12-08T09:56:53.091 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:53.094 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:53.116 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:53.116 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/205) 2023-12-08T09:56:53.298 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:53.302 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:53.302 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:53.430 INFO:teuthology.orchestra.run.smithi016.stdout:generic/205 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:53.434 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/205 2023-12-08T09:56:53.434 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/205 2023-12-08T09:56:53.434 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:53.436 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:53.445 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:53.446 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/206) 2023-12-08T09:56:53.622 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:53.627 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:53.627 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:53.737 INFO:teuthology.orchestra.run.smithi016.stdout:generic/206 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T09:56:53.741 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/206 2023-12-08T09:56:53.741 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/206 2023-12-08T09:56:53.741 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:56:53.743 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:53.752 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:56:53.753 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/207) 2023-12-08T09:56:53.928 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:56:53.932 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:56:53.932 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:56:55.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:56:55.334 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:57:08.321 INFO:teuthology.orchestra.run.smithi016.stdout:generic/207 15s 2023-12-08T09:57:08.327 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/207 2023-12-08T09:57:08.327 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T09:57:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:57:08.338 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T09:57:08.339 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/208) 2023-12-08T09:57:08.755 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T09:57:08.758 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T09:57:08.758 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T09:57:25.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:57:25.541 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:57:56.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:57:56.054 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:58:26.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:58:26.119 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T09:58:26.155 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T09:58:26.154+0000 7eff8ec05640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 276852) UID: 0 2023-12-08T09:58:26.156 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T09:58:26.154+0000 7f6d0f17a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 276852) UID: 0 2023-12-08T09:58:26.156 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T09:58:26.154+0000 7ff25129f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 276852) UID: 0 2023-12-08T09:58:26.156 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T09:58:26.154+0000 7fdb0d8f4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 276852) UID: 0 2023-12-08T10:00:29.043 INFO:teuthology.orchestra.run.smithi016.stdout:generic/208 201s 2023-12-08T10:00:29.048 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/208 2023-12-08T10:00:29.048 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:29.051 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:29.061 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:29.062 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/209) 2023-12-08T10:00:29.548 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:29.552 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:29.552 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:38.904 INFO:teuthology.orchestra.run.smithi016.stdout:generic/209 [failed, exit status 1]- output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/209.out.bad) 2023-12-08T10:00:38.919 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/209.out 2023-12-08 08:39:53.762917897 +0000 2023-12-08T10:00:38.919 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/209.out.bad 2023-12-08 10:00:38.900667748 +0000 2023-12-08T10:00:38.919 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -1,2 +1,2 @@ 2023-12-08T10:00:38.919 INFO:teuthology.orchestra.run.smithi016.stdout: QA output created by 209 2023-12-08T10:00:38.919 INFO:teuthology.orchestra.run.smithi016.stdout: -test ran for 30 seconds without error 2023-12-08T10:00:38.920 INFO:teuthology.orchestra.run.smithi016.stdout: +reader found old byte at pos 4096 2023-12-08T10:00:38.920 INFO:teuthology.orchestra.run.smithi016.stdout: \ No newline at end of file 2023-12-08T10:00:38.929 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/209 2023-12-08T10:00:38.929 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/209 2023-12-08T10:00:38.929 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T10:00:38.933 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:38.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T10:00:38.945 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T10:00:38.945 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T10:00:38.946 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/210) 2023-12-08T10:00:39.371 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:39.376 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:39.376 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:39.618 INFO:teuthology.orchestra.run.smithi016.stdout:generic/210 0s 2023-12-08T10:00:39.623 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/210 2023-12-08T10:00:39.623 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:39.625 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:39.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:39.635 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/211) 2023-12-08T10:00:40.098 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:40.102 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:40.102 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:40.397 INFO:teuthology.orchestra.run.smithi016.stdout:generic/211 0s 2023-12-08T10:00:40.402 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/211 2023-12-08T10:00:40.402 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:40.405 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:40.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:40.414 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/212) 2023-12-08T10:00:40.588 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:40.592 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:40.592 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:40.871 INFO:teuthology.orchestra.run.smithi016.stdout:generic/212 0s 2023-12-08T10:00:40.877 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/212 2023-12-08T10:00:40.878 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:40.880 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:40.888 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:40.888 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/213) 2023-12-08T10:00:41.062 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:41.066 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:41.066 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:41.598 INFO:teuthology.orchestra.run.smithi016.stdout:generic/213 - output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/213.out.bad) 2023-12-08T10:00:41.600 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/213.out 2023-12-08 08:39:53.763917883 +0000 2023-12-08T10:00:41.600 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/213.out.bad 2023-12-08 10:00:41.502630624 +0000 2023-12-08T10:00:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -1,4 +1,3 @@ 2023-12-08T10:00:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: QA output created by 213 2023-12-08T10:00:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: We should get: fallocate: No space left on device 2023-12-08T10:00:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: Strangely, xfs_io sometimes says "Success" when something went wrong, FYI 2023-12-08T10:00:41.601 INFO:teuthology.orchestra.run.smithi016.stdout: -fallocate: No space left on device 2023-12-08T10:00:41.603 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/213 2023-12-08T10:00:41.603 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/213 2023-12-08T10:00:41.603 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T10:00:41.605 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:41.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T10:00:41.615 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T10:00:41.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T10:00:41.615 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/214) 2023-12-08T10:00:42.079 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:42.083 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:42.083 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:43.074 INFO:teuthology.orchestra.run.smithi016.stdout:generic/214 1s 2023-12-08T10:00:43.083 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/214 2023-12-08T10:00:43.084 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:43.088 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:43.102 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:43.102 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/215) 2023-12-08T10:00:43.609 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:43.616 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:43.616 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:46.786 INFO:teuthology.orchestra.run.smithi016.stdout:generic/215 3s 2023-12-08T10:00:46.794 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/215 2023-12-08T10:00:46.794 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:46.798 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:46.812 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:46.812 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/216) 2023-12-08T10:00:47.080 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:47.085 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:47.085 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:47.270 INFO:teuthology.orchestra.run.smithi016.stdout:generic/216 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:47.274 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/216 2023-12-08T10:00:47.274 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/216 2023-12-08T10:00:47.275 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:47.277 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:47.289 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:47.289 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/217) 2023-12-08T10:00:47.552 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:47.561 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:47.562 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:47.733 INFO:teuthology.orchestra.run.smithi016.stdout:generic/217 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:47.738 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/217 2023-12-08T10:00:47.738 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/217 2023-12-08T10:00:47.738 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:47.741 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:47.752 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:47.753 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/218) 2023-12-08T10:00:48.018 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:48.024 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:48.024 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:48.196 INFO:teuthology.orchestra.run.smithi016.stdout:generic/218 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:48.200 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/218 2023-12-08T10:00:48.200 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/218 2023-12-08T10:00:48.200 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:48.204 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:48.215 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:48.216 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/219) 2023-12-08T10:00:48.497 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:48.503 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:48.503 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:48.928 INFO:teuthology.orchestra.run.smithi016.stdout:generic/219 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:48.934 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/219 2023-12-08T10:00:48.935 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/219 2023-12-08T10:00:48.935 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:48.937 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:48.948 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:48.948 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/220) 2023-12-08T10:00:49.223 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:49.228 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:49.228 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:49.432 INFO:teuthology.orchestra.run.smithi016.stdout:generic/220 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:49.436 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/220 2023-12-08T10:00:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/220 2023-12-08T10:00:49.437 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:49.439 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:49.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:49.454 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/221) 2023-12-08T10:00:49.712 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:49.720 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:49.720 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:50.977 INFO:teuthology.orchestra.run.smithi016.stdout:generic/221 1s 2023-12-08T10:00:50.987 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/221 2023-12-08T10:00:50.987 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:50.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:51.004 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:51.005 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/222) 2023-12-08T10:00:51.290 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:51.295 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:51.295 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:51.474 INFO:teuthology.orchestra.run.smithi016.stdout:generic/222 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:51.479 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/222 2023-12-08T10:00:51.480 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/222 2023-12-08T10:00:51.480 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:51.484 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:51.497 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:51.497 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/223) 2023-12-08T10:00:51.784 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:51.789 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:51.790 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:51.953 INFO:teuthology.orchestra.run.smithi016.stdout:generic/223 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:51.957 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/223 2023-12-08T10:00:51.957 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/223 2023-12-08T10:00:51.957 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:51.960 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:51.969 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:51.969 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/224) 2023-12-08T10:00:52.248 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:52.253 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:52.253 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:52.428 INFO:teuthology.orchestra.run.smithi016.stdout:generic/224 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:52.433 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/224 2023-12-08T10:00:52.433 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/224 2023-12-08T10:00:52.433 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:52.436 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:52.450 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:52.450 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/225) 2023-12-08T10:00:52.723 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:52.730 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:52.730 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:52.880 INFO:teuthology.orchestra.run.smithi016.stdout:generic/225 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:52.884 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/225 2023-12-08T10:00:52.884 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/225 2023-12-08T10:00:52.884 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:52.900 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:52.901 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/226) 2023-12-08T10:00:53.176 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:53.181 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:53.181 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:53.363 INFO:teuthology.orchestra.run.smithi016.stdout:generic/226 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:53.368 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/226 2023-12-08T10:00:53.368 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/226 2023-12-08T10:00:53.368 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:53.372 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:53.385 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:53.386 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/227) 2023-12-08T10:00:53.663 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:53.671 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:53.671 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:53.848 INFO:teuthology.orchestra.run.smithi016.stdout:generic/227 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:53.854 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/227 2023-12-08T10:00:53.854 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/227 2023-12-08T10:00:53.854 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:53.857 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:53.869 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:53.869 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/228) 2023-12-08T10:00:54.145 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:54.150 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:54.150 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:54.466 INFO:teuthology.orchestra.run.smithi016.stdout:generic/228 0s 2023-12-08T10:00:54.474 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/228 2023-12-08T10:00:54.474 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:54.490 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:54.490 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/229) 2023-12-08T10:00:54.995 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:55.001 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:55.002 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:55.173 INFO:teuthology.orchestra.run.smithi016.stdout:generic/229 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:55.179 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/229 2023-12-08T10:00:55.179 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/229 2023-12-08T10:00:55.179 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:55.183 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:55.193 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:55.193 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/230) 2023-12-08T10:00:55.460 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:55.464 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:55.464 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:55.629 INFO:teuthology.orchestra.run.smithi016.stdout:generic/230 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:55.638 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/230 2023-12-08T10:00:55.638 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/230 2023-12-08T10:00:55.638 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:55.641 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:55.652 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:55.652 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/231) 2023-12-08T10:00:55.923 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:55.933 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:55.933 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:56.120 INFO:teuthology.orchestra.run.smithi016.stdout:generic/231 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:56.125 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/231 2023-12-08T10:00:56.125 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/231 2023-12-08T10:00:56.126 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:56.143 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:56.143 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/232) 2023-12-08T10:00:56.419 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:56.425 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:56.425 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:56.619 INFO:teuthology.orchestra.run.smithi016.stdout:generic/232 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:56.624 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/232 2023-12-08T10:00:56.624 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/232 2023-12-08T10:00:56.624 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:56.627 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:56.639 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:56.639 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/233) 2023-12-08T10:00:56.937 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:56.942 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:56.942 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:57.104 INFO:teuthology.orchestra.run.smithi016.stdout:generic/233 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:57.108 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/233 2023-12-08T10:00:57.109 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/233 2023-12-08T10:00:57.109 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:57.112 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:57.123 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:57.123 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/234) 2023-12-08T10:00:57.392 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:57.396 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:57.396 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:57.549 INFO:teuthology.orchestra.run.smithi016.stdout:generic/234 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:57.554 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/234 2023-12-08T10:00:57.555 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/234 2023-12-08T10:00:57.555 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:57.558 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:57.568 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:57.568 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/235) 2023-12-08T10:00:57.816 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:57.822 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:57.822 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:58.020 INFO:teuthology.orchestra.run.smithi016.stdout:generic/235 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:00:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/235 2023-12-08T10:00:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/235 2023-12-08T10:00:58.026 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:58.032 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:58.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:58.051 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/236) 2023-12-08T10:00:58.333 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:00:58.340 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:00:58.341 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:59.639 INFO:teuthology.orchestra.run.smithi016.stdout:generic/236 1s 2023-12-08T10:00:59.646 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/236 2023-12-08T10:00:59.646 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:00:59.649 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:00:59.660 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:00:59.660 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/237) 2023-12-08T10:01:00.095 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:01:00.101 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:01:00.101 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:01:00.321 INFO:teuthology.orchestra.run.smithi016.stdout:generic/237 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-12-08T10:01:00.326 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/237 2023-12-08T10:01:00.326 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/237 2023-12-08T10:01:00.326 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:01:00.329 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:01:00.342 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:01:00.343 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/238) 2023-12-08T10:01:00.584 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:01:00.590 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:01:00.590 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:01:00.741 INFO:teuthology.orchestra.run.smithi016.stdout:generic/238 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:01:00.745 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/238 2023-12-08T10:01:00.745 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/238 2023-12-08T10:01:00.745 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:01:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:01:00.758 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:01:00.758 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/239) 2023-12-08T10:01:00.998 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:01:01.003 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:01:01.003 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:01:29.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:01:29.438 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:01:29.491 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T10:01:29.488+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98572) UID: 0 2023-12-08T10:01:29.494 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T10:01:29.492+0000 7fb32d740640 -1 Fail to open '/proc/98572/cmdline' error = (2) No such file or directory 2023-12-08T10:01:29.494 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T10:01:29.492+0000 7fb32d740640 -1 received signal: Hangup from (PID: 98572) UID: 0 2023-12-08T10:01:29.500 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T10:01:29.490+0000 7f0ae298c640 -1 Fail to open '/proc/98572/cmdline' error = (2) No such file or directory 2023-12-08T10:01:29.500 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T10:01:29.490+0000 7f0ae298c640 -1 received signal: Hangup from (PID: 98572) UID: 0 2023-12-08T10:01:29.501 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T10:01:29.490+0000 7f7f9cd1e640 -1 Fail to open '/proc/98572/cmdline' error = (2) No such file or directory 2023-12-08T10:01:29.501 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T10:01:29.490+0000 7f7f9cd1e640 -1 received signal: Hangup from (PID: 98572) UID: 0 2023-12-08T10:03:30.008 INFO:teuthology.orchestra.run.smithi016.stdout:generic/239 148s 2023-12-08T10:03:30.032 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/239 2023-12-08T10:03:30.032 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:03:30.034 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:03:30.045 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:03:30.046 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/240) 2023-12-08T10:03:30.477 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:03:30.481 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:03:30.481 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:03:33.654 INFO:teuthology.orchestra.run.smithi016.stdout:generic/240 3s 2023-12-08T10:03:33.661 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/240 2023-12-08T10:03:33.661 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:03:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:03:33.675 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:03:33.676 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/241) 2023-12-08T10:03:34.079 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:03:34.083 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:03:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:04:47.456 INFO:teuthology.orchestra.run.smithi016.stdout:generic/241 73s 2023-12-08T10:04:47.463 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/241 2023-12-08T10:04:47.463 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:04:47.466 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:04:47.481 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:04:47.481 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/242) 2023-12-08T10:04:47.926 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:04:47.930 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:04:47.931 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:04:48.068 INFO:teuthology.orchestra.run.smithi016.stdout:generic/242 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:04:48.072 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/242 2023-12-08T10:04:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/242 2023-12-08T10:04:48.073 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:04:48.076 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:04:48.087 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:04:48.087 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/243) 2023-12-08T10:04:48.315 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:04:48.319 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:04:48.319 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:04:48.467 INFO:teuthology.orchestra.run.smithi016.stdout:generic/243 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:04:48.471 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/243 2023-12-08T10:04:48.471 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/243 2023-12-08T10:04:48.472 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:04:48.474 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:04:48.485 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:04:48.485 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/244) 2023-12-08T10:04:48.727 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:04:48.732 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:04:48.732 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:04:48.883 INFO:teuthology.orchestra.run.smithi016.stdout:generic/244 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2023-12-08T10:04:48.889 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/244 2023-12-08T10:04:48.889 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/244 2023-12-08T10:04:48.889 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:04:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:04:48.908 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:04:48.908 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/245) 2023-12-08T10:04:49.162 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:04:49.168 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:04:49.168 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:04:49.466 INFO:teuthology.orchestra.run.smithi016.stdout:generic/245 0s 2023-12-08T10:04:49.472 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/245 2023-12-08T10:04:49.472 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:04:49.476 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:04:49.488 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:04:49.488 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/246) 2023-12-08T10:04:50.415 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:04:50.420 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:04:50.421 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:04:50.642 INFO:teuthology.orchestra.run.smithi016.stdout:generic/246 0s 2023-12-08T10:04:50.648 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/246 2023-12-08T10:04:50.649 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:04:50.651 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:04:50.662 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:04:50.662 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/247) 2023-12-08T10:04:51.092 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:04:51.096 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:04:51.096 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:10.861 INFO:teuthology.orchestra.run.smithi112.stderr:error: Compressing program wrote following message to stderr when compressing log /var/log/ceph/ceph-osd.4.log.1: 2023-12-08T10:05:10.861 INFO:teuthology.orchestra.run.smithi112.stderr:gzip: stdin: file size changed while zipping 2023-12-08T10:05:36.926 INFO:teuthology.orchestra.run.smithi016.stdout:generic/247 45s 2023-12-08T10:05:36.933 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/247 2023-12-08T10:05:36.933 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:36.937 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:36.948 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:36.948 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/248) 2023-12-08T10:05:37.179 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:37.185 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:37.185 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:37.395 INFO:teuthology.orchestra.run.smithi016.stdout:generic/248 0s 2023-12-08T10:05:37.402 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/248 2023-12-08T10:05:37.402 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:37.405 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:37.418 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:37.418 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/249) 2023-12-08T10:05:37.650 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:37.656 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:37.656 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:39.851 INFO:teuthology.orchestra.run.smithi016.stdout:generic/249 2s 2023-12-08T10:05:39.858 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/249 2023-12-08T10:05:39.858 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:39.861 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:39.876 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:39.876 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/250) 2023-12-08T10:05:40.342 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:40.346 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:40.346 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:40.566 INFO:teuthology.orchestra.run.smithi016.stdout:generic/250 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:05:40.570 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/250 2023-12-08T10:05:40.571 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/250 2023-12-08T10:05:40.571 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:40.574 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:40.583 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:40.584 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/251) 2023-12-08T10:05:40.841 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:40.845 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:40.846 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:41.029 INFO:teuthology.orchestra.run.smithi016.stdout:generic/251 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:05:41.033 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/251 2023-12-08T10:05:41.033 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/251 2023-12-08T10:05:41.033 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:41.037 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:41.049 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:41.049 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/252) 2023-12-08T10:05:41.285 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:41.290 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:41.290 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:41.452 INFO:teuthology.orchestra.run.smithi016.stdout:generic/252 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:05:41.457 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/252 2023-12-08T10:05:41.457 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/252 2023-12-08T10:05:41.457 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:41.462 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:41.473 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:41.474 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/253) 2023-12-08T10:05:41.719 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:41.725 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:41.726 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:41.887 INFO:teuthology.orchestra.run.smithi016.stdout:generic/253 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:05:41.892 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/253 2023-12-08T10:05:41.892 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/253 2023-12-08T10:05:41.892 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:41.895 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:41.907 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:41.907 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/254) 2023-12-08T10:05:42.150 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:42.155 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:42.155 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:42.307 INFO:teuthology.orchestra.run.smithi016.stdout:generic/254 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:05:42.312 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/254 2023-12-08T10:05:42.312 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/254 2023-12-08T10:05:42.312 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:42.315 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:42.324 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:42.324 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/255) 2023-12-08T10:05:42.566 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:42.572 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:42.572 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:42.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:05:42.610 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:05:42.662 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T10:05:42.660+0000 7ff25129f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 299298) UID: 0 2023-12-08T10:05:42.662 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T10:05:42.660+0000 7fdb0d8f4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 299298) UID: 0 2023-12-08T10:05:42.663 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T10:05:42.661+0000 7f6d0f17a640 -1 Fail to open '/proc/299298/cmdline' error = (2) No such file or directory 2023-12-08T10:05:42.663 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T10:05:42.661+0000 7f6d0f17a640 -1 received signal: Hangup from (PID: 299298) UID: 0 2023-12-08T10:05:42.663 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T10:05:42.661+0000 7eff8ec05640 -1 Fail to open '/proc/299298/cmdline' error = (2) No such file or directory 2023-12-08T10:05:42.663 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T10:05:42.661+0000 7eff8ec05640 -1 received signal: Hangup from (PID: 299298) UID: 0 2023-12-08T10:05:42.866 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T10:05:42.865+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98614) UID: 0 2023-12-08T10:05:42.867 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T10:05:42.865+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98614) UID: 0 2023-12-08T10:05:42.867 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T10:05:42.865+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98614) UID: 0 2023-12-08T10:05:42.867 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T10:05:42.865+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98614) UID: 0 2023-12-08T10:05:44.043 INFO:teuthology.orchestra.run.smithi016.stdout:generic/255 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-12-08T10:05:44.048 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/255 2023-12-08T10:05:44.048 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/255 2023-12-08T10:05:44.048 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:44.054 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:44.068 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:44.069 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/256) 2023-12-08T10:05:44.594 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:44.601 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:44.601 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:45.976 INFO:teuthology.orchestra.run.smithi016.stdout:generic/256 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:05:45.980 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/256 2023-12-08T10:05:45.981 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/256 2023-12-08T10:05:45.981 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:45.984 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:45.999 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:46.000 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/257) 2023-12-08T10:05:46.282 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:46.287 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:46.287 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:48.133 INFO:teuthology.orchestra.run.smithi016.stdout:generic/257 2s 2023-12-08T10:05:48.141 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/257 2023-12-08T10:05:48.141 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:48.144 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:48.154 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:48.154 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/258) 2023-12-08T10:05:48.743 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:48.749 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:48.749 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:48.955 INFO:teuthology.orchestra.run.smithi016.stdout:generic/258 [failed, exit status 1]- output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/258.out.bad) 2023-12-08T10:05:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/258.out 2023-12-08 08:39:53.766917840 +0000 2023-12-08T10:05:48.958 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/258.out.bad 2023-12-08 10:05:48.952244423 +0000 2023-12-08T10:05:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -1,5 +1,6 @@ 2023-12-08T10:05:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: QA output created by 258 2023-12-08T10:05:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: Creating file with timestamp of Jan 1, 1960 2023-12-08T10:05:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: Testing for negative seconds since epoch 2023-12-08T10:05:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: -Remounting to flush cache 2023-12-08T10:05:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: -Testing for negative seconds since epoch 2023-12-08T10:05:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: +Timestamp wrapped: 3979351756 2023-12-08T10:05:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: +Timestamp wrapped 2023-12-08T10:05:48.959 INFO:teuthology.orchestra.run.smithi016.stdout: +(see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/258.full for details) 2023-12-08T10:05:48.962 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/258 2023-12-08T10:05:48.962 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/258 2023-12-08T10:05:48.962 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T10:05:48.965 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:48.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T10:05:48.975 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T10:05:48.975 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T10:05:48.976 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/259) 2023-12-08T10:05:49.251 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:49.257 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:49.257 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:49.420 INFO:teuthology.orchestra.run.smithi016.stdout:generic/259 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:05:49.424 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/259 2023-12-08T10:05:49.425 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/259 2023-12-08T10:05:49.425 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:49.427 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:49.444 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:49.444 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/260) 2023-12-08T10:05:49.724 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:49.728 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:49.728 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:49.889 INFO:teuthology.orchestra.run.smithi016.stdout:generic/260 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:05:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/260 2023-12-08T10:05:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/260 2023-12-08T10:05:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:49.897 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:49.907 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:49.907 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/261) 2023-12-08T10:05:50.207 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:50.212 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:50.212 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:50.381 INFO:teuthology.orchestra.run.smithi016.stdout:generic/261 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:05:50.386 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/261 2023-12-08T10:05:50.386 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/261 2023-12-08T10:05:50.386 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:50.389 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:50.401 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:50.401 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/262) 2023-12-08T10:05:50.674 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:50.682 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:50.682 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:50.848 INFO:teuthology.orchestra.run.smithi016.stdout:generic/262 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:05:50.853 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/262 2023-12-08T10:05:50.854 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/262 2023-12-08T10:05:50.854 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:05:50.857 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:05:50.872 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:05:50.872 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/263) 2023-12-08T10:05:51.141 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:05:51.145 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:05:51.145 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:57.273 INFO:teuthology.orchestra.run.smithi016.stdout:generic/263 66s 2023-12-08T10:06:57.278 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/263 2023-12-08T10:06:57.278 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:06:57.280 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:57.289 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:06:57.290 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/264) 2023-12-08T10:06:57.790 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:06:57.794 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:06:57.794 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:57.939 INFO:teuthology.orchestra.run.smithi016.stdout:generic/264 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:06:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/264 2023-12-08T10:06:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/264 2023-12-08T10:06:57.943 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:06:57.945 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:57.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:06:57.954 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/265) 2023-12-08T10:06:58.141 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:06:58.145 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:06:58.145 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:58.325 INFO:teuthology.orchestra.run.smithi016.stdout:generic/265 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:06:58.328 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/265 2023-12-08T10:06:58.329 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/265 2023-12-08T10:06:58.329 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:06:58.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:58.340 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:06:58.340 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/266) 2023-12-08T10:06:58.511 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:06:58.515 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:06:58.515 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:58.637 INFO:teuthology.orchestra.run.smithi016.stdout:generic/266 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:06:58.641 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/266 2023-12-08T10:06:58.641 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/266 2023-12-08T10:06:58.641 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:06:58.643 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:58.651 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:06:58.651 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/267) 2023-12-08T10:06:58.867 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:06:58.871 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:06:58.871 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:59.009 INFO:teuthology.orchestra.run.smithi016.stdout:generic/267 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:06:59.012 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/267 2023-12-08T10:06:59.013 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/267 2023-12-08T10:06:59.013 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:06:59.015 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:59.023 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:06:59.023 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/268) 2023-12-08T10:06:59.196 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:06:59.200 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:06:59.200 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:59.323 INFO:teuthology.orchestra.run.smithi016.stdout:generic/268 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:06:59.327 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/268 2023-12-08T10:06:59.327 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/268 2023-12-08T10:06:59.327 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:06:59.329 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:59.337 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:06:59.337 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/269) 2023-12-08T10:06:59.510 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:06:59.514 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:06:59.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:59.626 INFO:teuthology.orchestra.run.smithi016.stdout:generic/269 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:06:59.629 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/269 2023-12-08T10:06:59.630 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/269 2023-12-08T10:06:59.630 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:06:59.632 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:59.640 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:06:59.640 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/270) 2023-12-08T10:06:59.815 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:06:59.819 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:06:59.819 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:59.947 INFO:teuthology.orchestra.run.smithi016.stdout:generic/270 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2023-12-08T10:06:59.950 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/270 2023-12-08T10:06:59.950 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/270 2023-12-08T10:06:59.950 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:06:59.953 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:06:59.963 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:06:59.964 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/271) 2023-12-08T10:07:00.137 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:00.140 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:00.141 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:00.299 INFO:teuthology.orchestra.run.smithi016.stdout:generic/271 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:00.303 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/271 2023-12-08T10:07:00.303 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/271 2023-12-08T10:07:00.303 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:00.305 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:00.313 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:00.314 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/272) 2023-12-08T10:07:00.486 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:00.490 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:00.491 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:00.618 INFO:teuthology.orchestra.run.smithi016.stdout:generic/272 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:00.622 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/272 2023-12-08T10:07:00.622 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/272 2023-12-08T10:07:00.622 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:00.624 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:00.633 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:00.633 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/273) 2023-12-08T10:07:00.807 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:00.811 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:00.812 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:00.931 INFO:teuthology.orchestra.run.smithi016.stdout:generic/273 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/273 2023-12-08T10:07:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/273 2023-12-08T10:07:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:00.936 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:00.945 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:00.945 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/274) 2023-12-08T10:07:01.117 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:01.121 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:01.121 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:01.238 INFO:teuthology.orchestra.run.smithi016.stdout:generic/274 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:01.242 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/274 2023-12-08T10:07:01.242 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/274 2023-12-08T10:07:01.242 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:01.244 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:01.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:01.254 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/275) 2023-12-08T10:07:01.453 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:01.457 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:01.457 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:01.569 INFO:teuthology.orchestra.run.smithi016.stdout:generic/275 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:01.573 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/275 2023-12-08T10:07:01.573 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/275 2023-12-08T10:07:01.573 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:01.575 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:01.584 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:01.584 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/276) 2023-12-08T10:07:01.759 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:01.763 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:01.763 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:01.883 INFO:teuthology.orchestra.run.smithi016.stdout:generic/276 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:01.887 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/276 2023-12-08T10:07:01.887 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/276 2023-12-08T10:07:01.887 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:01.889 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:01.897 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:01.898 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/277) 2023-12-08T10:07:02.069 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:02.073 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:02.073 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:02.202 INFO:teuthology.orchestra.run.smithi016.stdout:generic/277 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:02.206 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/277 2023-12-08T10:07:02.206 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/277 2023-12-08T10:07:02.206 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:02.208 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:02.216 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:02.217 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/278) 2023-12-08T10:07:02.388 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:02.392 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:02.392 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:02.518 INFO:teuthology.orchestra.run.smithi016.stdout:generic/278 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:02.522 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/278 2023-12-08T10:07:02.522 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/278 2023-12-08T10:07:02.522 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:02.524 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:02.532 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:02.533 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/279) 2023-12-08T10:07:02.707 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:02.711 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:02.712 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:02.845 INFO:teuthology.orchestra.run.smithi016.stdout:generic/279 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:02.849 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/279 2023-12-08T10:07:02.849 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/279 2023-12-08T10:07:02.849 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:02.851 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:02.859 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:02.860 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/280) 2023-12-08T10:07:03.035 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:03.038 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:03.038 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:03.250 INFO:teuthology.orchestra.run.smithi016.stdout:generic/280 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:03.253 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/280 2023-12-08T10:07:03.253 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/280 2023-12-08T10:07:03.254 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:03.256 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:03.273 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:03.273 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/281) 2023-12-08T10:07:03.449 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:03.453 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:03.453 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:03.581 INFO:teuthology.orchestra.run.smithi016.stdout:generic/281 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:03.585 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/281 2023-12-08T10:07:03.585 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/281 2023-12-08T10:07:03.585 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:03.587 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:03.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:03.596 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/282) 2023-12-08T10:07:03.771 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:03.775 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:03.775 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:03.898 INFO:teuthology.orchestra.run.smithi016.stdout:generic/282 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:03.901 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/282 2023-12-08T10:07:03.901 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/282 2023-12-08T10:07:03.901 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:03.904 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:03.912 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:03.912 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/283) 2023-12-08T10:07:04.085 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:04.089 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:04.089 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:04.216 INFO:teuthology.orchestra.run.smithi016.stdout:generic/283 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:04.219 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/283 2023-12-08T10:07:04.219 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/283 2023-12-08T10:07:04.220 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:04.222 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:04.230 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:04.230 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/284) 2023-12-08T10:07:04.403 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:04.407 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:04.407 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:04.517 INFO:teuthology.orchestra.run.smithi016.stdout:generic/284 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:04.521 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/284 2023-12-08T10:07:04.521 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/284 2023-12-08T10:07:04.521 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:04.523 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:04.531 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:04.532 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/285) 2023-12-08T10:07:04.795 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:04.800 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:06.618 INFO:teuthology.orchestra.run.smithi016.stdout:generic/285 2s 2023-12-08T10:07:06.624 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/285 2023-12-08T10:07:06.625 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:06.627 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:06.682 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:06.682 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/286) 2023-12-08T10:07:07.237 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:07.241 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:07.241 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:42.206 INFO:teuthology.orchestra.run.smithi016.stdout:generic/286 34s 2023-12-08T10:07:42.339 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/286 2023-12-08T10:07:42.339 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:42.341 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:42.364 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:42.365 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/287) 2023-12-08T10:07:44.734 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:44.739 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:44.739 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:45.193 INFO:teuthology.orchestra.run.smithi016.stdout:generic/287 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:45.197 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/287 2023-12-08T10:07:45.197 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/287 2023-12-08T10:07:45.197 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:45.208 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:45.208 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/288) 2023-12-08T10:07:45.384 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:45.388 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:45.388 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:45.757 INFO:teuthology.orchestra.run.smithi016.stdout:generic/288 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:45.761 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/288 2023-12-08T10:07:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/288 2023-12-08T10:07:45.762 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:45.764 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:45.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:45.773 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/289) 2023-12-08T10:07:45.947 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:45.951 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:45.951 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:46.070 INFO:teuthology.orchestra.run.smithi016.stdout:generic/289 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:46.074 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/289 2023-12-08T10:07:46.074 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/289 2023-12-08T10:07:46.074 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:46.076 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:46.084 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:46.084 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/290) 2023-12-08T10:07:46.258 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:46.263 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:46.263 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:46.391 INFO:teuthology.orchestra.run.smithi016.stdout:generic/290 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:46.395 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/290 2023-12-08T10:07:46.395 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/290 2023-12-08T10:07:46.395 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:46.398 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:46.406 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:46.406 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/291) 2023-12-08T10:07:46.579 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:46.583 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:46.583 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:46.736 INFO:teuthology.orchestra.run.smithi016.stdout:generic/291 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:46.739 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/291 2023-12-08T10:07:46.740 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/291 2023-12-08T10:07:46.740 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:46.742 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:46.751 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:46.751 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/292) 2023-12-08T10:07:46.924 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:46.928 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:46.929 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:47.140 INFO:teuthology.orchestra.run.smithi016.stdout:generic/292 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:47.144 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/292 2023-12-08T10:07:47.144 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/292 2023-12-08T10:07:47.144 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:47.146 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:47.154 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:47.154 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/293) 2023-12-08T10:07:47.326 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:47.330 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:47.330 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:47.829 INFO:teuthology.orchestra.run.smithi016.stdout:generic/293 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/293 2023-12-08T10:07:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/293 2023-12-08T10:07:47.833 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:47.835 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:47.843 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:47.844 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/294) 2023-12-08T10:07:48.014 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:48.018 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:48.018 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:48.148 INFO:teuthology.orchestra.run.smithi016.stdout:generic/294 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:48.151 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/294 2023-12-08T10:07:48.151 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/294 2023-12-08T10:07:48.151 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:48.153 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:48.162 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:48.162 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/295) 2023-12-08T10:07:48.334 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:48.338 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:48.338 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:48.466 INFO:teuthology.orchestra.run.smithi016.stdout:generic/295 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:48.469 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/295 2023-12-08T10:07:48.469 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/295 2023-12-08T10:07:48.469 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:48.471 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:48.480 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:48.480 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/296) 2023-12-08T10:07:48.651 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:48.655 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:48.655 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:48.799 INFO:teuthology.orchestra.run.smithi016.stdout:generic/296 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:48.803 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/296 2023-12-08T10:07:48.803 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/296 2023-12-08T10:07:48.803 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:48.805 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:48.814 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:48.814 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/297) 2023-12-08T10:07:48.986 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:48.990 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:48.991 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:49.176 INFO:teuthology.orchestra.run.smithi016.stdout:generic/297 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:49.180 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/297 2023-12-08T10:07:49.180 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/297 2023-12-08T10:07:49.180 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:49.182 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:49.191 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:49.191 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/298) 2023-12-08T10:07:49.364 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:49.367 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:49.368 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:49.679 INFO:teuthology.orchestra.run.smithi016.stdout:generic/298 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/298 2023-12-08T10:07:49.682 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/298 2023-12-08T10:07:49.683 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:49.685 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:49.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:49.694 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/299) 2023-12-08T10:07:49.867 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:49.871 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:49.871 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:49.984 INFO:teuthology.orchestra.run.smithi016.stdout:generic/299 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:49.991 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/299 2023-12-08T10:07:49.991 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/299 2023-12-08T10:07:49.991 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:49.991 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:49.998 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:49.998 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/300) 2023-12-08T10:07:50.173 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:50.177 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:50.177 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:50.288 INFO:teuthology.orchestra.run.smithi016.stdout:generic/300 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:50.291 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/300 2023-12-08T10:07:50.291 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/300 2023-12-08T10:07:50.291 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:50.293 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:50.302 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:50.303 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/301) 2023-12-08T10:07:50.490 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:50.494 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:50.494 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:50.605 INFO:teuthology.orchestra.run.smithi016.stdout:generic/301 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:50.609 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/301 2023-12-08T10:07:50.609 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/301 2023-12-08T10:07:50.609 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:50.612 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:50.620 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:50.621 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/302) 2023-12-08T10:07:50.796 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:50.800 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:50.800 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:50.913 INFO:teuthology.orchestra.run.smithi016.stdout:generic/302 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:50.916 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/302 2023-12-08T10:07:50.916 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/302 2023-12-08T10:07:50.916 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:50.919 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:50.936 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:50.936 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/303) 2023-12-08T10:07:51.113 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:51.117 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:51.118 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:51.277 INFO:teuthology.orchestra.run.smithi016.stdout:generic/303 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T10:07:51.280 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/303 2023-12-08T10:07:51.280 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/303 2023-12-08T10:07:51.281 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:51.283 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:51.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:51.292 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/304) 2023-12-08T10:07:51.783 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:51.786 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:51.787 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:51.946 INFO:teuthology.orchestra.run.smithi016.stdout:generic/304 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-12-08T10:07:51.950 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/304 2023-12-08T10:07:51.950 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/304 2023-12-08T10:07:51.950 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:51.952 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:52.008 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:52.008 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/305) 2023-12-08T10:07:52.457 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:52.463 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:52.463 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:52.584 INFO:teuthology.orchestra.run.smithi016.stdout:generic/305 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:52.588 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/305 2023-12-08T10:07:52.588 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/305 2023-12-08T10:07:52.588 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:52.590 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:52.599 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:52.599 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/306) 2023-12-08T10:07:52.774 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:52.778 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:52.778 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:52.892 INFO:teuthology.orchestra.run.smithi016.stdout:generic/306 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:52.895 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/306 2023-12-08T10:07:52.896 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/306 2023-12-08T10:07:52.896 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:52.898 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:52.907 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:52.907 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/307) 2023-12-08T10:07:53.081 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:53.085 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:53.086 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:53.202 INFO:teuthology.orchestra.run.smithi016.stdout:generic/307 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:07:53.205 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/307 2023-12-08T10:07:53.206 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/307 2023-12-08T10:07:53.206 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:53.208 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:53.216 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:53.217 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/308) 2023-12-08T10:07:53.389 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:53.394 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:53.394 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:53.537 INFO:teuthology.orchestra.run.smithi016.stdout:generic/308 0s 2023-12-08T10:07:53.543 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/308 2023-12-08T10:07:53.543 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:53.545 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:53.554 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:53.554 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/309) 2023-12-08T10:07:53.730 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:53.734 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:53.734 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:54.940 INFO:teuthology.orchestra.run.smithi016.stdout:generic/309 1s 2023-12-08T10:07:54.946 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/309 2023-12-08T10:07:54.947 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:07:54.948 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:07:54.957 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:07:54.957 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/310) 2023-12-08T10:07:55.430 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:07:55.435 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:07:55.435 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:13.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:09:13.864 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:09:16.324 INFO:teuthology.orchestra.run.smithi016.stdout:generic/310 81s 2023-12-08T10:09:16.344 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/310 2023-12-08T10:09:16.344 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:09:16.347 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:16.365 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:09:16.365 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/311) 2023-12-08T10:09:17.445 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:09:17.449 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:09:17.449 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:18.226 INFO:teuthology.orchestra.run.smithi016.stdout:generic/311 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:09:18.230 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/311 2023-12-08T10:09:18.230 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/311 2023-12-08T10:09:18.230 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:09:18.233 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:18.243 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:09:18.244 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/312) 2023-12-08T10:09:18.426 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:09:18.430 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:09:18.430 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:18.800 INFO:teuthology.orchestra.run.smithi016.stdout:generic/312 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:09:18.803 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/312 2023-12-08T10:09:18.803 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/312 2023-12-08T10:09:18.803 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:09:18.806 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:18.814 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:09:18.815 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/313) 2023-12-08T10:09:18.986 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:09:18.990 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:09:18.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:23.540 INFO:teuthology.orchestra.run.smithi016.stdout:generic/313 5s 2023-12-08T10:09:23.546 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/313 2023-12-08T10:09:23.547 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:09:23.549 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:23.559 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:09:23.559 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/314) 2023-12-08T10:09:23.741 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:09:23.745 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:09:23.745 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:24.247 INFO:teuthology.orchestra.run.smithi016.stdout:generic/314 1s 2023-12-08T10:09:24.252 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/314 2023-12-08T10:09:24.252 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:09:24.255 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:24.263 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:09:24.263 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/315) 2023-12-08T10:09:24.438 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:09:24.442 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:09:24.442 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:25.141 INFO:teuthology.orchestra.run.smithi016.stdout:generic/315 1s 2023-12-08T10:09:25.147 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/315 2023-12-08T10:09:25.148 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:09:25.150 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:25.159 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:09:25.159 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/316) 2023-12-08T10:09:25.606 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:09:25.610 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:09:25.610 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:26.492 INFO:teuthology.orchestra.run.smithi016.stdout:generic/316 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-12-08T10:09:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/316 2023-12-08T10:09:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/316 2023-12-08T10:09:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:09:26.498 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:26.507 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:09:26.507 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/317) 2023-12-08T10:09:26.961 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:09:26.965 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:09:26.965 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:27.075 INFO:teuthology.orchestra.run.smithi016.stdout:generic/317 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:09:27.078 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/317 2023-12-08T10:09:27.078 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/317 2023-12-08T10:09:27.079 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:09:27.081 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:27.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:09:27.090 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/318) 2023-12-08T10:09:27.261 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:09:27.265 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:09:27.265 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:27.380 INFO:teuthology.orchestra.run.smithi016.stdout:generic/318 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:09:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/318 2023-12-08T10:09:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/318 2023-12-08T10:09:27.384 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:09:27.386 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:27.394 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:09:27.395 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/319) 2023-12-08T10:09:27.566 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:09:27.570 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:09:27.570 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:27.729 INFO:teuthology.orchestra.run.smithi016.stdout:generic/319 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-12-08T10:09:27.732 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/319 2023-12-08T10:09:27.733 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/319 2023-12-08T10:09:27.733 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:09:27.735 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:27.743 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:09:27.744 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/320) 2023-12-08T10:09:27.917 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:09:27.921 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:09:27.921 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:28.036 INFO:teuthology.orchestra.run.smithi016.stdout:generic/320 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:09:28.040 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/320 2023-12-08T10:09:28.040 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/320 2023-12-08T10:09:28.040 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:09:28.042 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:28.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:09:28.051 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/321) 2023-12-08T10:09:28.223 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:09:28.227 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:09:28.227 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:28.351 INFO:teuthology.orchestra.run.smithi016.stdout:generic/321 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:09:28.355 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/321 2023-12-08T10:09:28.355 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/321 2023-12-08T10:09:28.355 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:09:28.357 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:28.365 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:09:28.366 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/322) 2023-12-08T10:09:28.539 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:09:28.544 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:09:28.544 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:28.665 INFO:teuthology.orchestra.run.smithi016.stdout:generic/322 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:09:28.669 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/322 2023-12-08T10:09:28.669 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/322 2023-12-08T10:09:28.669 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:09:28.672 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:28.679 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:09:28.680 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/323) 2023-12-08T10:09:28.852 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:09:28.856 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:09:28.856 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:09:44.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:09:44.428 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:10:14.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:10:14.506 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:10:14.541 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T10:10:14.539+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98679) UID: 0 2023-12-08T10:10:14.541 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T10:10:14.539+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98679) UID: 0 2023-12-08T10:10:14.541 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T10:10:14.539+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98679) UID: 0 2023-12-08T10:10:14.542 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T10:10:14.539+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98679) UID: 0 2023-12-08T10:11:29.532 INFO:teuthology.orchestra.run.smithi016.stdout:generic/323 121s 2023-12-08T10:11:29.538 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/323 2023-12-08T10:11:29.538 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:29.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:29.564 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/324) 2023-12-08T10:11:29.986 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:29.990 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:29.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:30.154 INFO:teuthology.orchestra.run.smithi016.stdout:generic/324 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:30.157 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/324 2023-12-08T10:11:30.158 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/324 2023-12-08T10:11:30.158 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:30.160 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:30.169 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:30.169 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/325) 2023-12-08T10:11:30.344 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:30.348 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:30.348 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:30.480 INFO:teuthology.orchestra.run.smithi016.stdout:generic/325 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:30.484 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/325 2023-12-08T10:11:30.484 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/325 2023-12-08T10:11:30.484 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:30.486 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:30.494 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:30.495 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/326) 2023-12-08T10:11:30.668 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:30.671 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:30.672 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:30.785 INFO:teuthology.orchestra.run.smithi016.stdout:generic/326 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:30.788 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/326 2023-12-08T10:11:30.788 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/326 2023-12-08T10:11:30.789 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:30.791 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:30.799 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:30.799 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/327) 2023-12-08T10:11:30.972 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:30.976 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:30.976 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:31.087 INFO:teuthology.orchestra.run.smithi016.stdout:generic/327 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:31.091 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/327 2023-12-08T10:11:31.091 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/327 2023-12-08T10:11:31.091 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:31.093 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:31.101 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:31.101 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/328) 2023-12-08T10:11:31.273 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:31.277 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:31.277 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:31.388 INFO:teuthology.orchestra.run.smithi016.stdout:generic/328 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:31.391 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/328 2023-12-08T10:11:31.392 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/328 2023-12-08T10:11:31.392 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:31.394 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:31.403 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:31.403 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/329) 2023-12-08T10:11:31.574 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:31.578 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:31.578 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:31.704 INFO:teuthology.orchestra.run.smithi016.stdout:generic/329 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:31.708 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/329 2023-12-08T10:11:31.708 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/329 2023-12-08T10:11:31.709 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:31.710 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:31.718 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:31.719 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/330) 2023-12-08T10:11:31.891 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:31.895 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:31.895 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:32.011 INFO:teuthology.orchestra.run.smithi016.stdout:generic/330 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:32.015 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/330 2023-12-08T10:11:32.015 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/330 2023-12-08T10:11:32.015 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:32.017 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:32.025 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:32.026 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/331) 2023-12-08T10:11:32.197 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:32.201 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:32.201 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:32.322 INFO:teuthology.orchestra.run.smithi016.stdout:generic/331 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:32.326 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/331 2023-12-08T10:11:32.326 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/331 2023-12-08T10:11:32.326 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:32.328 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:32.337 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:32.337 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/332) 2023-12-08T10:11:32.509 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:32.513 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:32.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:32.640 INFO:teuthology.orchestra.run.smithi016.stdout:generic/332 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:32.644 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/332 2023-12-08T10:11:32.644 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/332 2023-12-08T10:11:32.644 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:32.646 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:32.654 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:32.655 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/333) 2023-12-08T10:11:32.827 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:32.831 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:32.831 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:32.940 INFO:teuthology.orchestra.run.smithi016.stdout:generic/333 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:32.944 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/333 2023-12-08T10:11:32.944 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/333 2023-12-08T10:11:32.944 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:32.946 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:32.955 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:32.955 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/334) 2023-12-08T10:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout:generic/334 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/334 2023-12-08T10:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/334 2023-12-08T10:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:33.261 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:33.261 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/335) 2023-12-08T10:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout:generic/335 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/335 2023-12-08T10:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/335 2023-12-08T10:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:33.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:33.575 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/336) 2023-12-08T10:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout:generic/336 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/336 2023-12-08T10:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/336 2023-12-08T10:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:33.896 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:33.896 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/337) 2023-12-08T10:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout:generic/337 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/337 2023-12-08T10:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/337 2023-12-08T10:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:34.199 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:34.200 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/338) 2023-12-08T10:11:34.374 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:34.502 INFO:teuthology.orchestra.run.smithi016.stdout:generic/338 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:34.503 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/338 2023-12-08T10:11:34.503 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/338 2023-12-08T10:11:34.503 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:34.505 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:34.514 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:34.514 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/339) 2023-12-08T10:11:34.686 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:34.690 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:34.690 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:34.798 INFO:teuthology.orchestra.run.smithi016.stdout:generic/339 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:34.802 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/339 2023-12-08T10:11:34.802 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/339 2023-12-08T10:11:34.802 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:34.804 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:34.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:34.813 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/340) 2023-12-08T10:11:34.985 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:34.989 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:34.990 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:35.098 INFO:teuthology.orchestra.run.smithi016.stdout:generic/340 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:35.101 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/340 2023-12-08T10:11:35.101 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/340 2023-12-08T10:11:35.101 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:35.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:35.112 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:35.112 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/341) 2023-12-08T10:11:35.285 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:35.289 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:35.289 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:35.408 INFO:teuthology.orchestra.run.smithi016.stdout:generic/341 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:35.411 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/341 2023-12-08T10:11:35.411 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/341 2023-12-08T10:11:35.411 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:35.414 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:35.422 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:35.422 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/342) 2023-12-08T10:11:35.596 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:35.600 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:35.600 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:35.723 INFO:teuthology.orchestra.run.smithi016.stdout:generic/342 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/342 2023-12-08T10:11:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/342 2023-12-08T10:11:35.727 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:35.730 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:35.751 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:35.752 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/343) 2023-12-08T10:11:35.923 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:35.927 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:35.927 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:36.047 INFO:teuthology.orchestra.run.smithi016.stdout:generic/343 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:36.050 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/343 2023-12-08T10:11:36.050 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/343 2023-12-08T10:11:36.050 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:36.052 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:36.061 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:36.061 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/344) 2023-12-08T10:11:36.232 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:36.236 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:36.236 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:36.344 INFO:teuthology.orchestra.run.smithi016.stdout:generic/344 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:36.347 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/344 2023-12-08T10:11:36.348 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/344 2023-12-08T10:11:36.348 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:36.350 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:36.358 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:36.359 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/345) 2023-12-08T10:11:36.530 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:36.534 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:36.534 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:36.654 INFO:teuthology.orchestra.run.smithi016.stdout:generic/345 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:36.657 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/345 2023-12-08T10:11:36.658 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/345 2023-12-08T10:11:36.658 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:36.660 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:36.686 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:36.687 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/346) 2023-12-08T10:11:36.861 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:36.865 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:36.865 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:36.972 INFO:teuthology.orchestra.run.smithi016.stdout:generic/346 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:36.976 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/346 2023-12-08T10:11:36.976 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/346 2023-12-08T10:11:36.976 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:36.978 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:36.987 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:36.987 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/347) 2023-12-08T10:11:37.159 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:37.163 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:37.164 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:37.313 INFO:teuthology.orchestra.run.smithi016.stdout:generic/347 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:37.316 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/347 2023-12-08T10:11:37.316 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/347 2023-12-08T10:11:37.317 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:37.319 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:37.344 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:37.344 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/348) 2023-12-08T10:11:37.516 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:37.520 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:37.520 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:37.639 INFO:teuthology.orchestra.run.smithi016.stdout:generic/348 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:37.643 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/348 2023-12-08T10:11:37.643 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/348 2023-12-08T10:11:37.643 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:37.645 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:37.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:37.654 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/349) 2023-12-08T10:11:37.829 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:37.833 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:37.833 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:38.740 INFO:teuthology.orchestra.run.smithi016.stdout:generic/349 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-12-08T10:11:38.744 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/349 2023-12-08T10:11:38.744 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/349 2023-12-08T10:11:38.744 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:38.746 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:38.769 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:38.769 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/350) 2023-12-08T10:11:38.944 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:38.948 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:38.948 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:40.176 INFO:teuthology.orchestra.run.smithi016.stdout:generic/350 2s 2023-12-08T10:11:40.181 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/350 2023-12-08T10:11:40.181 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:40.183 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:40.192 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:40.192 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/351) 2023-12-08T10:11:40.365 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:40.369 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:40.369 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:41.262 INFO:teuthology.orchestra.run.smithi016.stdout:generic/351 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2023-12-08T10:11:41.266 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/351 2023-12-08T10:11:41.266 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/351 2023-12-08T10:11:41.266 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:41.268 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:41.321 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:41.321 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/352) 2023-12-08T10:11:41.744 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:41.748 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:41.749 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:41.862 INFO:teuthology.orchestra.run.smithi016.stdout:generic/352 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:41.866 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/352 2023-12-08T10:11:41.866 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/352 2023-12-08T10:11:41.866 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:41.869 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:41.889 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:41.889 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/353) 2023-12-08T10:11:42.060 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:42.065 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:42.065 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:42.175 INFO:teuthology.orchestra.run.smithi016.stdout:generic/353 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:42.179 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/353 2023-12-08T10:11:42.179 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/353 2023-12-08T10:11:42.179 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:42.181 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:42.189 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:42.190 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/354) 2023-12-08T10:11:42.364 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:42.368 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:42.368 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:42.476 INFO:teuthology.orchestra.run.smithi016.stdout:generic/354 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:42.479 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/354 2023-12-08T10:11:42.480 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/354 2023-12-08T10:11:42.480 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:42.482 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:42.490 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:42.491 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/355) 2023-12-08T10:11:42.664 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:42.668 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:42.668 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:43.097 INFO:teuthology.orchestra.run.smithi016.stdout:generic/355 1s 2023-12-08T10:11:43.102 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/355 2023-12-08T10:11:43.102 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:43.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:43.113 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:43.113 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/356) 2023-12-08T10:11:43.624 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:43.628 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:43.628 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:43.748 INFO:teuthology.orchestra.run.smithi016.stdout:generic/356 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:43.752 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/356 2023-12-08T10:11:43.752 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/356 2023-12-08T10:11:43.752 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:43.754 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:43.763 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:43.763 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/357) 2023-12-08T10:11:43.938 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:43.941 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:43.942 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:44.051 INFO:teuthology.orchestra.run.smithi016.stdout:generic/357 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:44.055 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/357 2023-12-08T10:11:44.055 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/357 2023-12-08T10:11:44.055 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:44.057 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:44.066 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:44.066 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/358) 2023-12-08T10:11:44.238 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:44.242 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:44.242 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:44.352 INFO:teuthology.orchestra.run.smithi016.stdout:generic/358 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:44.356 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/358 2023-12-08T10:11:44.356 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/358 2023-12-08T10:11:44.356 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:44.358 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:44.367 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:44.367 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/359) 2023-12-08T10:11:44.539 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:44.543 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:44.543 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:44.653 INFO:teuthology.orchestra.run.smithi016.stdout:generic/359 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:44.657 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/359 2023-12-08T10:11:44.657 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/359 2023-12-08T10:11:44.657 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:44.659 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:44.668 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:44.668 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/360) 2023-12-08T10:11:44.840 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:44.844 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:44.844 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:45.006 INFO:teuthology.orchestra.run.smithi016.stdout:generic/360 1s 2023-12-08T10:11:45.011 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/360 2023-12-08T10:11:45.011 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:45.014 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:45.022 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:45.022 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/361) 2023-12-08T10:11:45.195 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:45.199 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:45.199 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:45.334 INFO:teuthology.orchestra.run.smithi016.stdout:generic/361 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:45.338 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/361 2023-12-08T10:11:45.338 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/361 2023-12-08T10:11:45.338 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:45.341 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:45.348 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:45.348 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/362) 2023-12-08T10:11:45.522 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:45.526 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:45.526 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:45.634 INFO:teuthology.orchestra.run.smithi016.stdout:generic/362 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:45.637 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/362 2023-12-08T10:11:45.637 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/362 2023-12-08T10:11:45.637 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:45.640 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:45.648 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:45.648 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/363) 2023-12-08T10:11:45.824 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:45.828 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:45.828 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:45.940 INFO:teuthology.orchestra.run.smithi016.stdout:generic/363 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:45.940 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/363 2023-12-08T10:11:45.940 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/363 2023-12-08T10:11:45.940 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:45.942 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:45.965 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:45.965 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/364) 2023-12-08T10:11:46.137 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:46.141 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:46.142 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:46.250 INFO:teuthology.orchestra.run.smithi016.stdout:generic/364 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:46.253 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/364 2023-12-08T10:11:46.253 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/364 2023-12-08T10:11:46.253 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:46.256 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:46.264 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:46.265 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/365) 2023-12-08T10:11:46.438 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:46.442 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:46.442 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:46.549 INFO:teuthology.orchestra.run.smithi016.stdout:generic/365 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:46.553 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/365 2023-12-08T10:11:46.553 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/365 2023-12-08T10:11:46.554 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:46.555 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:46.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:46.564 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/366) 2023-12-08T10:11:46.737 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:46.741 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:46.741 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:46.850 INFO:teuthology.orchestra.run.smithi016.stdout:generic/366 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:46.854 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/366 2023-12-08T10:11:46.854 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/366 2023-12-08T10:11:46.854 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:46.856 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:46.865 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:46.865 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/367) 2023-12-08T10:11:47.036 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:47.040 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:47.040 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:47.148 INFO:teuthology.orchestra.run.smithi016.stdout:generic/367 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:47.152 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/367 2023-12-08T10:11:47.152 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/367 2023-12-08T10:11:47.152 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:47.154 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:47.162 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:47.162 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/368) 2023-12-08T10:11:47.337 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:47.341 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:47.341 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:47.448 INFO:teuthology.orchestra.run.smithi016.stdout:generic/368 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:47.452 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/368 2023-12-08T10:11:47.452 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/368 2023-12-08T10:11:47.452 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:47.454 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:47.475 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:47.475 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/369) 2023-12-08T10:11:47.647 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:47.651 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:47.651 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:47.759 INFO:teuthology.orchestra.run.smithi016.stdout:generic/369 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:47.762 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/369 2023-12-08T10:11:47.763 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/369 2023-12-08T10:11:47.763 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:47.765 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:47.773 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:47.773 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/370) 2023-12-08T10:11:47.945 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:47.949 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:47.949 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:48.057 INFO:teuthology.orchestra.run.smithi016.stdout:generic/370 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:48.060 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/370 2023-12-08T10:11:48.060 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/370 2023-12-08T10:11:48.060 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:48.062 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:48.071 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:48.071 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/371) 2023-12-08T10:11:48.242 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:48.246 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:48.246 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:48.359 INFO:teuthology.orchestra.run.smithi016.stdout:generic/371 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:48.363 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/371 2023-12-08T10:11:48.363 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/371 2023-12-08T10:11:48.363 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:48.365 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:48.373 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:48.374 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/372) 2023-12-08T10:11:48.548 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:48.552 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:48.552 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:48.661 INFO:teuthology.orchestra.run.smithi016.stdout:generic/372 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:48.665 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/372 2023-12-08T10:11:48.665 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/372 2023-12-08T10:11:48.665 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:48.667 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:48.675 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:48.676 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/373) 2023-12-08T10:11:48.846 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:48.850 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:48.850 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:48.965 INFO:teuthology.orchestra.run.smithi016.stdout:generic/373 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:48.969 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/373 2023-12-08T10:11:48.969 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/373 2023-12-08T10:11:48.969 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:48.990 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:48.991 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/374) 2023-12-08T10:11:49.163 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:49.167 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:49.167 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:49.289 INFO:teuthology.orchestra.run.smithi016.stdout:generic/374 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:49.292 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/374 2023-12-08T10:11:49.293 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/374 2023-12-08T10:11:49.293 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:49.295 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:49.304 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:49.304 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/375) 2023-12-08T10:11:49.476 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:49.481 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:49.481 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:49.618 INFO:teuthology.orchestra.run.smithi016.stdout:generic/375 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-12-08T10:11:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/375 2023-12-08T10:11:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/375 2023-12-08T10:11:49.622 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:49.624 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:49.632 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:49.632 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/376) 2023-12-08T10:11:49.805 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:49.810 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:49.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:49.934 INFO:teuthology.orchestra.run.smithi016.stdout:generic/376 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:49.938 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/376 2023-12-08T10:11:49.938 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/376 2023-12-08T10:11:49.938 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:49.940 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:49.948 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:49.949 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/377) 2023-12-08T10:11:50.121 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:50.125 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:50.125 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:50.236 INFO:teuthology.orchestra.run.smithi016.stdout:generic/377 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:50.240 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/377 2023-12-08T10:11:50.240 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/377 2023-12-08T10:11:50.240 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:50.242 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:50.251 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:50.251 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/378) 2023-12-08T10:11:50.423 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:50.427 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:50.427 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:50.629 INFO:teuthology.orchestra.run.smithi016.stdout:generic/378 0s 2023-12-08T10:11:50.634 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/378 2023-12-08T10:11:50.634 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:50.636 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:50.645 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:50.645 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/379) 2023-12-08T10:11:51.142 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:51.147 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:51.147 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:51.276 INFO:teuthology.orchestra.run.smithi016.stdout:generic/379 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:51.279 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/379 2023-12-08T10:11:51.279 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/379 2023-12-08T10:11:51.280 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:51.282 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:51.290 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:51.291 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/380) 2023-12-08T10:11:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:51.467 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:51.467 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:51.577 INFO:teuthology.orchestra.run.smithi016.stdout:generic/380 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:51.581 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/380 2023-12-08T10:11:51.581 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/380 2023-12-08T10:11:51.581 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:51.583 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:51.603 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:51.603 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/381) 2023-12-08T10:11:51.775 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:51.779 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:51.779 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:51.889 INFO:teuthology.orchestra.run.smithi016.stdout:generic/381 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:51.893 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/381 2023-12-08T10:11:51.893 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/381 2023-12-08T10:11:51.893 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:51.895 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:51.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:51.904 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/382) 2023-12-08T10:11:52.076 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:52.080 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:52.080 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:52.190 INFO:teuthology.orchestra.run.smithi016.stdout:generic/382 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:52.193 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/382 2023-12-08T10:11:52.194 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/382 2023-12-08T10:11:52.194 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:52.196 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:52.204 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:52.205 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/383) 2023-12-08T10:11:52.379 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:52.382 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:52.383 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:52.493 INFO:teuthology.orchestra.run.smithi016.stdout:generic/383 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:52.496 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/383 2023-12-08T10:11:52.497 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/383 2023-12-08T10:11:52.497 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:52.499 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:52.507 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:52.507 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/384) 2023-12-08T10:11:52.679 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:52.683 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:52.683 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:52.803 INFO:teuthology.orchestra.run.smithi016.stdout:generic/384 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2023-12-08T10:11:52.806 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/384 2023-12-08T10:11:52.806 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/384 2023-12-08T10:11:52.806 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:52.809 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:52.818 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:52.818 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/385) 2023-12-08T10:11:52.990 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:52.994 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:52.995 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:53.106 INFO:teuthology.orchestra.run.smithi016.stdout:generic/385 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:53.109 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/385 2023-12-08T10:11:53.109 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/385 2023-12-08T10:11:53.109 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:53.114 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:53.134 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:53.135 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/386) 2023-12-08T10:11:53.306 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:53.310 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:53.310 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:53.425 INFO:teuthology.orchestra.run.smithi016.stdout:generic/386 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2023-12-08T10:11:53.428 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/386 2023-12-08T10:11:53.429 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/386 2023-12-08T10:11:53.429 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:53.431 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:53.439 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:53.440 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/387) 2023-12-08T10:11:53.612 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:53.616 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:53.616 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:53.726 INFO:teuthology.orchestra.run.smithi016.stdout:generic/387 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:53.729 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/387 2023-12-08T10:11:53.729 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/387 2023-12-08T10:11:53.730 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:53.732 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:53.740 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:53.740 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/388) 2023-12-08T10:11:53.914 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:53.918 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:53.918 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:54.032 INFO:teuthology.orchestra.run.smithi016.stdout:generic/388 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:54.036 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/388 2023-12-08T10:11:54.036 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/388 2023-12-08T10:11:54.036 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:54.038 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:54.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:54.047 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/389) 2023-12-08T10:11:54.220 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:54.224 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:54.224 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:54.343 INFO:teuthology.orchestra.run.smithi016.stdout:generic/389 [not run] O_TMPFILE is not supported 2023-12-08T10:11:54.346 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/389 2023-12-08T10:11:54.346 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/389 2023-12-08T10:11:54.347 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:54.349 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:54.357 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:54.357 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/390) 2023-12-08T10:11:54.532 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:54.535 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:54.536 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:54.660 INFO:teuthology.orchestra.run.smithi016.stdout:generic/390 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:11:54.663 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/390 2023-12-08T10:11:54.663 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/390 2023-12-08T10:11:54.664 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:11:54.666 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:11:54.687 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:11:54.687 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/391) 2023-12-08T10:11:54.860 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:11:54.864 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:11:54.864 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:13.048 INFO:teuthology.orchestra.run.smithi016.stdout:generic/391 18s 2023-12-08T10:12:13.111 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/391 2023-12-08T10:12:13.111 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:13.113 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:13.274 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:13.274 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/392) 2023-12-08T10:12:14.190 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:14.194 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:14.194 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:14.387 INFO:teuthology.orchestra.run.smithi016.stdout:generic/392 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:14.391 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/392 2023-12-08T10:12:14.392 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/392 2023-12-08T10:12:14.392 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:14.394 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:14.406 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:14.407 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/393) 2023-12-08T10:12:14.651 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:14.664 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:14.664 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:14.826 INFO:teuthology.orchestra.run.smithi016.stdout:generic/393 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:14.829 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/393 2023-12-08T10:12:14.830 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/393 2023-12-08T10:12:14.830 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:14.832 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:14.844 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:14.845 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/394) 2023-12-08T10:12:15.087 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:15.093 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:15.093 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:15.311 INFO:teuthology.orchestra.run.smithi016.stdout:generic/394 0s 2023-12-08T10:12:15.317 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/394 2023-12-08T10:12:15.317 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:15.320 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:15.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:15.330 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/395) 2023-12-08T10:12:15.845 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:15.854 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:15.854 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:16.031 INFO:teuthology.orchestra.run.smithi016.stdout:generic/395 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:16.037 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/395 2023-12-08T10:12:16.037 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/395 2023-12-08T10:12:16.037 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:16.039 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:16.048 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:16.049 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/396) 2023-12-08T10:12:16.289 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:16.295 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:16.295 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:16.472 INFO:teuthology.orchestra.run.smithi016.stdout:generic/396 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:16.477 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/396 2023-12-08T10:12:16.477 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/396 2023-12-08T10:12:16.478 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:16.480 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:16.490 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:16.490 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/397) 2023-12-08T10:12:16.744 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:16.753 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:16.753 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:16.956 INFO:teuthology.orchestra.run.smithi016.stdout:generic/397 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:16.960 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/397 2023-12-08T10:12:16.961 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/397 2023-12-08T10:12:16.961 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:16.964 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:16.977 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:16.978 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/398) 2023-12-08T10:12:17.215 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:17.221 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:17.221 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:17.406 INFO:teuthology.orchestra.run.smithi016.stdout:generic/398 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:17.411 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/398 2023-12-08T10:12:17.411 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/398 2023-12-08T10:12:17.411 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:17.415 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:17.427 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:17.428 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/399) 2023-12-08T10:12:17.683 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:17.688 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:17.688 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:17.878 INFO:teuthology.orchestra.run.smithi016.stdout:generic/399 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:17.883 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/399 2023-12-08T10:12:17.883 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/399 2023-12-08T10:12:17.883 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:17.886 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:17.897 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:17.898 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/400) 2023-12-08T10:12:18.134 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:18.139 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:18.139 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:18.296 INFO:teuthology.orchestra.run.smithi016.stdout:generic/400 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2023-12-08T10:12:18.300 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/400 2023-12-08T10:12:18.300 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/400 2023-12-08T10:12:18.301 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:18.304 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:18.313 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:18.313 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/401) 2023-12-08T10:12:18.556 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:18.561 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:18.561 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:18.720 INFO:teuthology.orchestra.run.smithi016.stdout:generic/401 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:18.721 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/401 2023-12-08T10:12:18.721 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/401 2023-12-08T10:12:18.721 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:18.722 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:18.735 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:18.736 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/402) 2023-12-08T10:12:18.939 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:18.943 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:18.944 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:19.053 INFO:teuthology.orchestra.run.smithi016.stdout:generic/402 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:19.057 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/402 2023-12-08T10:12:19.057 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/402 2023-12-08T10:12:19.057 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:19.059 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:19.068 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:19.068 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/403) 2023-12-08T10:12:19.242 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:19.246 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:19.246 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:19.355 INFO:teuthology.orchestra.run.smithi016.stdout:generic/403 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:19.359 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/403 2023-12-08T10:12:19.359 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/403 2023-12-08T10:12:19.359 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:19.361 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:19.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:19.370 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/404) 2023-12-08T10:12:19.542 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:19.547 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:19.547 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:20.419 INFO:teuthology.orchestra.run.smithi016.stdout:generic/404 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2023-12-08T10:12:20.422 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/404 2023-12-08T10:12:20.422 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/404 2023-12-08T10:12:20.423 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:20.425 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:20.484 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:20.485 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/405) 2023-12-08T10:12:20.930 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:20.934 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:20.934 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:21.086 INFO:teuthology.orchestra.run.smithi016.stdout:generic/405 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:21.089 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/405 2023-12-08T10:12:21.089 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/405 2023-12-08T10:12:21.089 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:21.092 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:21.100 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:21.100 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/406) 2023-12-08T10:12:21.274 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:21.278 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:21.278 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:21.386 INFO:teuthology.orchestra.run.smithi016.stdout:generic/406 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:21.390 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/406 2023-12-08T10:12:21.390 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/406 2023-12-08T10:12:21.390 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:21.392 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:21.400 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:21.401 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/407) 2023-12-08T10:12:21.575 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:21.582 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:21.582 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:21.753 INFO:teuthology.orchestra.run.smithi016.stdout:generic/407 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T10:12:21.757 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/407 2023-12-08T10:12:21.757 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/407 2023-12-08T10:12:21.757 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:21.759 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:21.818 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:21.818 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/408) 2023-12-08T10:12:22.218 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:22.222 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:22.222 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:22.382 INFO:teuthology.orchestra.run.smithi016.stdout:generic/408 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-12-08T10:12:22.386 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/408 2023-12-08T10:12:22.386 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/408 2023-12-08T10:12:22.386 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:22.388 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:22.397 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:22.397 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/409) 2023-12-08T10:12:22.874 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:22.878 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:22.878 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:22.992 INFO:teuthology.orchestra.run.smithi016.stdout:generic/409 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:22.996 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/409 2023-12-08T10:12:22.996 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/409 2023-12-08T10:12:22.996 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:22.998 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:23.006 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:23.006 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/410) 2023-12-08T10:12:23.179 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:23.183 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:23.183 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:23.296 INFO:teuthology.orchestra.run.smithi016.stdout:generic/410 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:23.300 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/410 2023-12-08T10:12:23.300 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/410 2023-12-08T10:12:23.300 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:23.302 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:23.323 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:23.323 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/411) 2023-12-08T10:12:23.497 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:23.501 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:23.501 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:23.615 INFO:teuthology.orchestra.run.smithi016.stdout:generic/411 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:23.620 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/411 2023-12-08T10:12:23.620 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/411 2023-12-08T10:12:23.620 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:23.621 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:23.629 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:23.629 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/412) 2023-12-08T10:12:23.809 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:23.813 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:23.813 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:23.922 INFO:teuthology.orchestra.run.smithi016.stdout:generic/412 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/412 2023-12-08T10:12:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/412 2023-12-08T10:12:23.926 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:23.928 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:23.936 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:23.937 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/413) 2023-12-08T10:12:24.109 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:24.112 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:24.113 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:24.228 INFO:teuthology.orchestra.run.smithi016.stdout:generic/413 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:24.232 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/413 2023-12-08T10:12:24.232 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/413 2023-12-08T10:12:24.232 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:24.234 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:24.242 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:24.242 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/414) 2023-12-08T10:12:24.415 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:24.419 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:24.419 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:24.537 INFO:teuthology.orchestra.run.smithi016.stdout:generic/414 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:24.541 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/414 2023-12-08T10:12:24.541 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/414 2023-12-08T10:12:24.541 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:24.543 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:24.552 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:24.552 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/415) 2023-12-08T10:12:24.726 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:24.730 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:24.730 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:24.838 INFO:teuthology.orchestra.run.smithi016.stdout:generic/415 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:24.841 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/415 2023-12-08T10:12:24.841 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/415 2023-12-08T10:12:24.841 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:24.844 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:24.868 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:24.868 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/416) 2023-12-08T10:12:25.041 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:25.045 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:25.045 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:25.155 INFO:teuthology.orchestra.run.smithi016.stdout:generic/416 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:25.158 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/416 2023-12-08T10:12:25.158 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/416 2023-12-08T10:12:25.159 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:25.161 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:25.169 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:25.170 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/417) 2023-12-08T10:12:25.341 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:25.345 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:25.345 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:25.456 INFO:teuthology.orchestra.run.smithi016.stdout:generic/417 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:25.460 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/417 2023-12-08T10:12:25.460 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/417 2023-12-08T10:12:25.460 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:25.462 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:25.471 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:25.471 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/418) 2023-12-08T10:12:25.643 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:25.647 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:25.647 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:25.807 INFO:teuthology.orchestra.run.smithi016.stdout:generic/418 [not run] require ceph-fuse to be valid block disk 2023-12-08T10:12:25.810 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/418 2023-12-08T10:12:25.811 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/418 2023-12-08T10:12:25.811 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:25.813 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:25.821 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:25.822 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/419) 2023-12-08T10:12:25.994 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:25.998 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:25.998 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:26.133 INFO:teuthology.orchestra.run.smithi016.stdout:generic/419 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:26.136 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/419 2023-12-08T10:12:26.137 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/419 2023-12-08T10:12:26.137 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:26.139 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:26.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:26.148 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/420) 2023-12-08T10:12:26.319 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:26.323 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:26.323 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:27.227 INFO:teuthology.orchestra.run.smithi016.stdout:generic/420 1s 2023-12-08T10:12:27.233 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/420 2023-12-08T10:12:27.233 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:27.235 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:27.243 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:27.244 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/421) 2023-12-08T10:12:27.755 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:27.759 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:27.759 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:27.903 INFO:teuthology.orchestra.run.smithi016.stdout:generic/421 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:27.908 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/421 2023-12-08T10:12:27.908 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/421 2023-12-08T10:12:27.908 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:27.909 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:27.918 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:27.918 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/422) 2023-12-08T10:12:28.090 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:28.094 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:28.094 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:28.207 INFO:teuthology.orchestra.run.smithi016.stdout:generic/422 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:28.210 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/422 2023-12-08T10:12:28.210 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/422 2023-12-08T10:12:28.210 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:28.212 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:28.221 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:28.221 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/423) 2023-12-08T10:12:28.395 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:28.399 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:28.400 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:28.642 INFO:teuthology.orchestra.run.smithi016.stdout:generic/423 0s 2023-12-08T10:12:28.647 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/423 2023-12-08T10:12:28.647 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:28.650 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:28.659 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:28.659 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/424) 2023-12-08T10:12:29.122 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:29.126 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:29.126 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:29.277 INFO:teuthology.orchestra.run.smithi016.stdout:generic/424 [not run] file system doesn't support any of /bin/chattr +a/+c/+d/+i 2023-12-08T10:12:29.281 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/424 2023-12-08T10:12:29.281 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/424 2023-12-08T10:12:29.281 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:29.282 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:29.291 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:29.291 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/425) 2023-12-08T10:12:29.466 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:29.470 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:29.470 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:29.580 INFO:teuthology.orchestra.run.smithi016.stdout:generic/425 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:12:29.583 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/425 2023-12-08T10:12:29.584 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/425 2023-12-08T10:12:29.584 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:12:29.586 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:29.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:12:29.595 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/426) 2023-12-08T10:12:29.770 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:12:29.774 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:12:29.774 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:59.358 INFO:teuthology.orchestra.run.smithi016.stdout:generic/426 - output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/426.out.bad) 2023-12-08T10:12:59.380 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/426.out 2023-12-08 08:39:53.779917653 +0000 2023-12-08T10:12:59.380 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/426.out.bad 2023-12-08 10:12:59.348162323 +0000 2023-12-08T10:12:59.380 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -1,5 +1,1029 @@ 2023-12-08T10:12:59.380 INFO:teuthology.orchestra.run.smithi016.stdout: QA output created by 426 2023-12-08T10:12:59.380 INFO:teuthology.orchestra.run.smithi016.stdout: test_file_handles TEST_DIR/426-dir -d 2023-12-08T10:12:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000000) opened an unlinked file! 2023-12-08T10:12:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000001) opened an unlinked file! 2023-12-08T10:12:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000002) opened an unlinked file! 2023-12-08T10:12:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000003) opened an unlinked file! 2023-12-08T10:12:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000004) opened an unlinked file! 2023-12-08T10:12:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000005) opened an unlinked file! 2023-12-08T10:12:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000006) opened an unlinked file! 2023-12-08T10:12:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000007) opened an unlinked file! 2023-12-08T10:12:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000008) opened an unlinked file! 2023-12-08T10:12:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000009) opened an unlinked file! 2023-12-08T10:12:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000010) opened an unlinked file! 2023-12-08T10:12:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000011) opened an unlinked file! 2023-12-08T10:12:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000012) opened an unlinked file! 2023-12-08T10:12:59.381 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000013) opened an unlinked file! 2023-12-08T10:12:59.382 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000014) opened an unlinked file! 2023-12-08T10:12:59.382 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000015) opened an unlinked file! 2023-12-08T10:12:59.382 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000016) opened an unlinked file! 2023-12-08T10:12:59.382 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000017) opened an unlinked file! 2023-12-08T10:12:59.382 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000018) opened an unlinked file! 2023-12-08T10:12:59.382 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000019) opened an unlinked file! 2023-12-08T10:12:59.382 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000020) opened an unlinked file! 2023-12-08T10:12:59.382 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000021) opened an unlinked file! 2023-12-08T10:12:59.382 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000022) opened an unlinked file! 2023-12-08T10:12:59.382 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000023) opened an unlinked file! 2023-12-08T10:12:59.382 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000024) opened an unlinked file! 2023-12-08T10:12:59.382 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000025) opened an unlinked file! 2023-12-08T10:12:59.382 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000026) opened an unlinked file! 2023-12-08T10:12:59.382 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000027) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000028) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000029) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000030) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000031) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000032) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000033) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000034) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000035) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000036) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000037) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000038) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000039) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000040) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000041) opened an unlinked file! 2023-12-08T10:12:59.383 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000042) opened an unlinked file! 2023-12-08T10:12:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000043) opened an unlinked file! 2023-12-08T10:12:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000044) opened an unlinked file! 2023-12-08T10:12:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000045) opened an unlinked file! 2023-12-08T10:12:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000046) opened an unlinked file! 2023-12-08T10:12:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000047) opened an unlinked file! 2023-12-08T10:12:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000048) opened an unlinked file! 2023-12-08T10:12:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000049) opened an unlinked file! 2023-12-08T10:12:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000050) opened an unlinked file! 2023-12-08T10:12:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000051) opened an unlinked file! 2023-12-08T10:12:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000052) opened an unlinked file! 2023-12-08T10:12:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000053) opened an unlinked file! 2023-12-08T10:12:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000054) opened an unlinked file! 2023-12-08T10:12:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000055) opened an unlinked file! 2023-12-08T10:12:59.384 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000056) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000057) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000058) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000059) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000060) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000061) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000062) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000063) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000064) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000065) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000066) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000067) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000068) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000069) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000070) opened an unlinked file! 2023-12-08T10:12:59.385 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000071) opened an unlinked file! 2023-12-08T10:12:59.386 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000072) opened an unlinked file! 2023-12-08T10:12:59.386 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000073) opened an unlinked file! 2023-12-08T10:12:59.386 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000074) opened an unlinked file! 2023-12-08T10:12:59.386 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000075) opened an unlinked file! 2023-12-08T10:12:59.386 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000076) opened an unlinked file! 2023-12-08T10:12:59.386 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000077) opened an unlinked file! 2023-12-08T10:12:59.386 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000078) opened an unlinked file! 2023-12-08T10:12:59.386 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000079) opened an unlinked file! 2023-12-08T10:12:59.386 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000080) opened an unlinked file! 2023-12-08T10:12:59.386 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000081) opened an unlinked file! 2023-12-08T10:12:59.386 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000082) opened an unlinked file! 2023-12-08T10:12:59.386 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000083) opened an unlinked file! 2023-12-08T10:12:59.386 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000084) opened an unlinked file! 2023-12-08T10:12:59.386 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000085) opened an unlinked file! 2023-12-08T10:12:59.387 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000086) opened an unlinked file! 2023-12-08T10:12:59.387 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000087) opened an unlinked file! 2023-12-08T10:12:59.387 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000088) opened an unlinked file! 2023-12-08T10:12:59.387 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000089) opened an unlinked file! 2023-12-08T10:12:59.387 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000090) opened an unlinked file! 2023-12-08T10:12:59.387 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000091) opened an unlinked file! 2023-12-08T10:12:59.387 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000092) opened an unlinked file! 2023-12-08T10:12:59.387 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000093) opened an unlinked file! 2023-12-08T10:12:59.387 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000094) opened an unlinked file! 2023-12-08T10:12:59.387 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000095) opened an unlinked file! 2023-12-08T10:12:59.387 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000096) opened an unlinked file! 2023-12-08T10:12:59.387 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000097) opened an unlinked file! 2023-12-08T10:12:59.387 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000098) opened an unlinked file! 2023-12-08T10:12:59.387 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000099) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000100) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000101) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000102) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000103) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000104) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000105) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000106) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000107) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000108) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000109) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000110) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000111) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000112) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000113) opened an unlinked file! 2023-12-08T10:12:59.388 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000114) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000115) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000116) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000117) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000118) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000119) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000120) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000121) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000122) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000123) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000124) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000125) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000126) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000127) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000128) opened an unlinked file! 2023-12-08T10:12:59.389 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000129) opened an unlinked file! 2023-12-08T10:12:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000130) opened an unlinked file! 2023-12-08T10:12:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000131) opened an unlinked file! 2023-12-08T10:12:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000132) opened an unlinked file! 2023-12-08T10:12:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000133) opened an unlinked file! 2023-12-08T10:12:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000134) opened an unlinked file! 2023-12-08T10:12:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000135) opened an unlinked file! 2023-12-08T10:12:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000136) opened an unlinked file! 2023-12-08T10:12:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000137) opened an unlinked file! 2023-12-08T10:12:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000138) opened an unlinked file! 2023-12-08T10:12:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000139) opened an unlinked file! 2023-12-08T10:12:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000140) opened an unlinked file! 2023-12-08T10:12:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000141) opened an unlinked file! 2023-12-08T10:12:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000142) opened an unlinked file! 2023-12-08T10:12:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000143) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000144) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000145) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000146) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000147) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000148) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000149) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000150) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000151) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000152) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000153) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000154) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000155) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000156) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000157) opened an unlinked file! 2023-12-08T10:12:59.391 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000158) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000159) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000160) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000161) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000162) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000163) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000164) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000165) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000166) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000167) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000168) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000169) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000170) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000171) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000172) opened an unlinked file! 2023-12-08T10:12:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000173) opened an unlinked file! 2023-12-08T10:12:59.393 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000174) opened an unlinked file! 2023-12-08T10:12:59.393 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000175) opened an unlinked file! 2023-12-08T10:12:59.393 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000176) opened an unlinked file! 2023-12-08T10:12:59.393 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000177) opened an unlinked file! 2023-12-08T10:12:59.393 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000178) opened an unlinked file! 2023-12-08T10:12:59.393 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000179) opened an unlinked file! 2023-12-08T10:12:59.393 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000180) opened an unlinked file! 2023-12-08T10:12:59.393 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000181) opened an unlinked file! 2023-12-08T10:12:59.393 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000182) opened an unlinked file! 2023-12-08T10:12:59.393 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000183) opened an unlinked file! 2023-12-08T10:12:59.393 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000184) opened an unlinked file! 2023-12-08T10:12:59.393 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000185) opened an unlinked file! 2023-12-08T10:12:59.393 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000186) opened an unlinked file! 2023-12-08T10:12:59.393 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000187) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000188) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000189) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000190) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000191) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000192) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000193) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000194) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000195) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000196) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000197) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000198) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000199) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000200) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000201) opened an unlinked file! 2023-12-08T10:12:59.394 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000202) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000203) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000204) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000205) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000206) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000207) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000208) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000209) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000210) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000211) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000212) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000213) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000214) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000215) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000216) opened an unlinked file! 2023-12-08T10:12:59.395 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000217) opened an unlinked file! 2023-12-08T10:12:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000218) opened an unlinked file! 2023-12-08T10:12:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000219) opened an unlinked file! 2023-12-08T10:12:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000220) opened an unlinked file! 2023-12-08T10:12:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000221) opened an unlinked file! 2023-12-08T10:12:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000222) opened an unlinked file! 2023-12-08T10:12:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000223) opened an unlinked file! 2023-12-08T10:12:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000224) opened an unlinked file! 2023-12-08T10:12:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000225) opened an unlinked file! 2023-12-08T10:12:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000226) opened an unlinked file! 2023-12-08T10:12:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000227) opened an unlinked file! 2023-12-08T10:12:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000228) opened an unlinked file! 2023-12-08T10:12:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000229) opened an unlinked file! 2023-12-08T10:12:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000230) opened an unlinked file! 2023-12-08T10:12:59.396 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000231) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000232) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000233) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000234) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000235) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000236) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000237) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000238) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000239) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000240) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000241) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000242) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000243) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000244) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000245) opened an unlinked file! 2023-12-08T10:12:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000246) opened an unlinked file! 2023-12-08T10:12:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000247) opened an unlinked file! 2023-12-08T10:12:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000248) opened an unlinked file! 2023-12-08T10:12:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000249) opened an unlinked file! 2023-12-08T10:12:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000250) opened an unlinked file! 2023-12-08T10:12:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000251) opened an unlinked file! 2023-12-08T10:12:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000252) opened an unlinked file! 2023-12-08T10:12:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000253) opened an unlinked file! 2023-12-08T10:12:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000254) opened an unlinked file! 2023-12-08T10:12:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000255) opened an unlinked file! 2023-12-08T10:12:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000256) opened an unlinked file! 2023-12-08T10:12:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000257) opened an unlinked file! 2023-12-08T10:12:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000258) opened an unlinked file! 2023-12-08T10:12:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000259) opened an unlinked file! 2023-12-08T10:12:59.398 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000260) opened an unlinked file! 2023-12-08T10:12:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000261) opened an unlinked file! 2023-12-08T10:12:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000262) opened an unlinked file! 2023-12-08T10:12:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000263) opened an unlinked file! 2023-12-08T10:12:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000264) opened an unlinked file! 2023-12-08T10:12:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000265) opened an unlinked file! 2023-12-08T10:12:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000266) opened an unlinked file! 2023-12-08T10:12:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000267) opened an unlinked file! 2023-12-08T10:12:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000268) opened an unlinked file! 2023-12-08T10:12:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000269) opened an unlinked file! 2023-12-08T10:12:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000270) opened an unlinked file! 2023-12-08T10:12:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000271) opened an unlinked file! 2023-12-08T10:12:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000272) opened an unlinked file! 2023-12-08T10:12:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000273) opened an unlinked file! 2023-12-08T10:12:59.399 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000274) opened an unlinked file! 2023-12-08T10:12:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000275) opened an unlinked file! 2023-12-08T10:12:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000276) opened an unlinked file! 2023-12-08T10:12:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000277) opened an unlinked file! 2023-12-08T10:12:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000278) opened an unlinked file! 2023-12-08T10:12:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000279) opened an unlinked file! 2023-12-08T10:12:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000280) opened an unlinked file! 2023-12-08T10:12:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000281) opened an unlinked file! 2023-12-08T10:12:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000282) opened an unlinked file! 2023-12-08T10:12:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000283) opened an unlinked file! 2023-12-08T10:12:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000284) opened an unlinked file! 2023-12-08T10:12:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000285) opened an unlinked file! 2023-12-08T10:12:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000286) opened an unlinked file! 2023-12-08T10:12:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000287) opened an unlinked file! 2023-12-08T10:12:59.400 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000288) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000289) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000290) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000291) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000292) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000293) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000294) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000295) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000296) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000297) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000298) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000299) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000300) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000301) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000302) opened an unlinked file! 2023-12-08T10:12:59.401 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000303) opened an unlinked file! 2023-12-08T10:12:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000304) opened an unlinked file! 2023-12-08T10:12:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000305) opened an unlinked file! 2023-12-08T10:12:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000306) opened an unlinked file! 2023-12-08T10:12:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000307) opened an unlinked file! 2023-12-08T10:12:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000308) opened an unlinked file! 2023-12-08T10:12:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000309) opened an unlinked file! 2023-12-08T10:12:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000310) opened an unlinked file! 2023-12-08T10:12:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000311) opened an unlinked file! 2023-12-08T10:12:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000312) opened an unlinked file! 2023-12-08T10:12:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000313) opened an unlinked file! 2023-12-08T10:12:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000314) opened an unlinked file! 2023-12-08T10:12:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000315) opened an unlinked file! 2023-12-08T10:12:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000316) opened an unlinked file! 2023-12-08T10:12:59.402 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000317) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000318) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000319) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000320) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000321) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000322) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000323) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000324) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000325) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000326) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000327) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000328) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000329) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000330) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000331) opened an unlinked file! 2023-12-08T10:12:59.403 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000332) opened an unlinked file! 2023-12-08T10:12:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000333) opened an unlinked file! 2023-12-08T10:12:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000334) opened an unlinked file! 2023-12-08T10:12:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000335) opened an unlinked file! 2023-12-08T10:12:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000336) opened an unlinked file! 2023-12-08T10:12:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000337) opened an unlinked file! 2023-12-08T10:12:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000338) opened an unlinked file! 2023-12-08T10:12:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000339) opened an unlinked file! 2023-12-08T10:12:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000340) opened an unlinked file! 2023-12-08T10:12:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000341) opened an unlinked file! 2023-12-08T10:12:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000342) opened an unlinked file! 2023-12-08T10:12:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000343) opened an unlinked file! 2023-12-08T10:12:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000344) opened an unlinked file! 2023-12-08T10:12:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000345) opened an unlinked file! 2023-12-08T10:12:59.404 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000346) opened an unlinked file! 2023-12-08T10:12:59.405 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000347) opened an unlinked file! 2023-12-08T10:12:59.405 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000348) opened an unlinked file! 2023-12-08T10:12:59.405 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000349) opened an unlinked file! 2023-12-08T10:12:59.405 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000350) opened an unlinked file! 2023-12-08T10:12:59.405 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000351) opened an unlinked file! 2023-12-08T10:12:59.405 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000352) opened an unlinked file! 2023-12-08T10:12:59.405 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000353) opened an unlinked file! 2023-12-08T10:12:59.405 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000354) opened an unlinked file! 2023-12-08T10:12:59.405 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000355) opened an unlinked file! 2023-12-08T10:12:59.405 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000356) opened an unlinked file! 2023-12-08T10:12:59.405 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000357) opened an unlinked file! 2023-12-08T10:12:59.405 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000358) opened an unlinked file! 2023-12-08T10:12:59.405 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000359) opened an unlinked file! 2023-12-08T10:12:59.405 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000360) opened an unlinked file! 2023-12-08T10:12:59.406 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000361) opened an unlinked file! 2023-12-08T10:12:59.406 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000362) opened an unlinked file! 2023-12-08T10:12:59.406 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000363) opened an unlinked file! 2023-12-08T10:12:59.406 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000364) opened an unlinked file! 2023-12-08T10:12:59.406 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000365) opened an unlinked file! 2023-12-08T10:12:59.406 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000366) opened an unlinked file! 2023-12-08T10:12:59.406 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000367) opened an unlinked file! 2023-12-08T10:12:59.406 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000368) opened an unlinked file! 2023-12-08T10:12:59.406 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000369) opened an unlinked file! 2023-12-08T10:12:59.406 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000370) opened an unlinked file! 2023-12-08T10:12:59.406 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000371) opened an unlinked file! 2023-12-08T10:12:59.406 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000372) opened an unlinked file! 2023-12-08T10:12:59.406 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000373) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000374) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000375) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000376) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000377) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000378) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000379) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000380) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000381) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000382) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000383) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000384) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000385) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000386) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000387) opened an unlinked file! 2023-12-08T10:12:59.407 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000388) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000389) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000390) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000391) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000392) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000393) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000394) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000395) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000396) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000397) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000398) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000399) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000400) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000401) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000402) opened an unlinked file! 2023-12-08T10:12:59.408 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000403) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000404) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000405) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000406) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000407) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000408) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000409) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000410) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000411) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000412) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000413) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000414) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000415) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000416) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000417) opened an unlinked file! 2023-12-08T10:12:59.409 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000418) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000419) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000420) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000421) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000422) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000423) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000424) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000425) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000426) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000427) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000428) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000429) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000430) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000431) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000432) opened an unlinked file! 2023-12-08T10:12:59.410 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000433) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000434) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000435) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000436) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000437) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000438) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000439) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000440) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000441) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000442) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000443) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000444) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000445) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000446) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000447) opened an unlinked file! 2023-12-08T10:12:59.411 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000448) opened an unlinked file! 2023-12-08T10:12:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000449) opened an unlinked file! 2023-12-08T10:12:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000450) opened an unlinked file! 2023-12-08T10:12:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000451) opened an unlinked file! 2023-12-08T10:12:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000452) opened an unlinked file! 2023-12-08T10:12:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000453) opened an unlinked file! 2023-12-08T10:12:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000454) opened an unlinked file! 2023-12-08T10:12:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000455) opened an unlinked file! 2023-12-08T10:12:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000456) opened an unlinked file! 2023-12-08T10:12:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000457) opened an unlinked file! 2023-12-08T10:12:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000458) opened an unlinked file! 2023-12-08T10:12:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000459) opened an unlinked file! 2023-12-08T10:12:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000460) opened an unlinked file! 2023-12-08T10:12:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000461) opened an unlinked file! 2023-12-08T10:12:59.412 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000462) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000463) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000464) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000465) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000466) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000467) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000468) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000469) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000470) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000471) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000472) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000473) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000474) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000475) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000476) opened an unlinked file! 2023-12-08T10:12:59.413 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000477) opened an unlinked file! 2023-12-08T10:12:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000478) opened an unlinked file! 2023-12-08T10:12:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000479) opened an unlinked file! 2023-12-08T10:12:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000480) opened an unlinked file! 2023-12-08T10:12:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000481) opened an unlinked file! 2023-12-08T10:12:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000482) opened an unlinked file! 2023-12-08T10:12:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000483) opened an unlinked file! 2023-12-08T10:12:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000484) opened an unlinked file! 2023-12-08T10:12:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000485) opened an unlinked file! 2023-12-08T10:12:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000486) opened an unlinked file! 2023-12-08T10:12:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000487) opened an unlinked file! 2023-12-08T10:12:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000488) opened an unlinked file! 2023-12-08T10:12:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000489) opened an unlinked file! 2023-12-08T10:12:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000490) opened an unlinked file! 2023-12-08T10:12:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000491) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000492) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000493) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000494) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000495) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000496) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000497) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000498) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000499) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000500) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000501) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000502) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000503) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000504) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000505) opened an unlinked file! 2023-12-08T10:12:59.415 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000506) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000507) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000508) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000509) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000510) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000511) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000512) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000513) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000514) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000515) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000516) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000517) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000518) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000519) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000520) opened an unlinked file! 2023-12-08T10:12:59.416 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000521) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000522) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000523) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000524) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000525) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000526) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000527) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000528) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000529) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000530) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000531) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000532) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000533) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000534) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000535) opened an unlinked file! 2023-12-08T10:12:59.417 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000536) opened an unlinked file! 2023-12-08T10:12:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000537) opened an unlinked file! 2023-12-08T10:12:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000538) opened an unlinked file! 2023-12-08T10:12:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000539) opened an unlinked file! 2023-12-08T10:12:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000540) opened an unlinked file! 2023-12-08T10:12:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000541) opened an unlinked file! 2023-12-08T10:12:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000542) opened an unlinked file! 2023-12-08T10:12:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000543) opened an unlinked file! 2023-12-08T10:12:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000544) opened an unlinked file! 2023-12-08T10:12:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000545) opened an unlinked file! 2023-12-08T10:12:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000546) opened an unlinked file! 2023-12-08T10:12:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000547) opened an unlinked file! 2023-12-08T10:12:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000548) opened an unlinked file! 2023-12-08T10:12:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000549) opened an unlinked file! 2023-12-08T10:12:59.418 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000550) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000551) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000552) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000553) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000554) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000555) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000556) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000557) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000558) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000559) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000560) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000561) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000562) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000563) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000564) opened an unlinked file! 2023-12-08T10:12:59.419 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000565) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000566) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000567) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000568) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000569) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000570) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000571) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000572) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000573) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000574) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000575) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000576) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000577) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000578) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000579) opened an unlinked file! 2023-12-08T10:12:59.420 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000580) opened an unlinked file! 2023-12-08T10:12:59.421 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000581) opened an unlinked file! 2023-12-08T10:12:59.421 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000582) opened an unlinked file! 2023-12-08T10:12:59.421 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000583) opened an unlinked file! 2023-12-08T10:12:59.421 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000584) opened an unlinked file! 2023-12-08T10:12:59.421 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000585) opened an unlinked file! 2023-12-08T10:12:59.421 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000586) opened an unlinked file! 2023-12-08T10:12:59.421 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000587) opened an unlinked file! 2023-12-08T10:12:59.421 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000588) opened an unlinked file! 2023-12-08T10:12:59.421 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000589) opened an unlinked file! 2023-12-08T10:12:59.421 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000590) opened an unlinked file! 2023-12-08T10:12:59.421 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000591) opened an unlinked file! 2023-12-08T10:12:59.421 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000592) opened an unlinked file! 2023-12-08T10:12:59.421 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000593) opened an unlinked file! 2023-12-08T10:12:59.421 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000594) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000595) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000596) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000597) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000598) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000599) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000600) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000601) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000602) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000603) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000604) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000605) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000606) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000607) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000608) opened an unlinked file! 2023-12-08T10:12:59.422 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000609) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000610) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000611) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000612) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000613) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000614) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000615) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000616) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000617) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000618) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000619) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000620) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000621) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000622) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000623) opened an unlinked file! 2023-12-08T10:12:59.423 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000624) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000625) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000626) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000627) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000628) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000629) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000630) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000631) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000632) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000633) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000634) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000635) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000636) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000637) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000638) opened an unlinked file! 2023-12-08T10:12:59.424 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000639) opened an unlinked file! 2023-12-08T10:12:59.425 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000640) opened an unlinked file! 2023-12-08T10:12:59.425 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000641) opened an unlinked file! 2023-12-08T10:12:59.425 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000642) opened an unlinked file! 2023-12-08T10:12:59.425 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000643) opened an unlinked file! 2023-12-08T10:12:59.425 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000644) opened an unlinked file! 2023-12-08T10:12:59.425 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000645) opened an unlinked file! 2023-12-08T10:12:59.425 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000646) opened an unlinked file! 2023-12-08T10:12:59.425 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000647) opened an unlinked file! 2023-12-08T10:12:59.425 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000648) opened an unlinked file! 2023-12-08T10:12:59.425 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000649) opened an unlinked file! 2023-12-08T10:12:59.425 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000650) opened an unlinked file! 2023-12-08T10:12:59.425 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000651) opened an unlinked file! 2023-12-08T10:12:59.425 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000652) opened an unlinked file! 2023-12-08T10:12:59.425 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000653) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000654) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000655) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000656) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000657) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000658) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000659) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000660) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000661) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000662) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000663) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000664) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000665) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000666) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000667) opened an unlinked file! 2023-12-08T10:12:59.426 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000668) opened an unlinked file! 2023-12-08T10:12:59.427 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000669) opened an unlinked file! 2023-12-08T10:12:59.427 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000670) opened an unlinked file! 2023-12-08T10:12:59.427 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000671) opened an unlinked file! 2023-12-08T10:12:59.427 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000672) opened an unlinked file! 2023-12-08T10:12:59.427 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000673) opened an unlinked file! 2023-12-08T10:12:59.427 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000674) opened an unlinked file! 2023-12-08T10:12:59.427 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000675) opened an unlinked file! 2023-12-08T10:12:59.427 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000676) opened an unlinked file! 2023-12-08T10:12:59.427 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000677) opened an unlinked file! 2023-12-08T10:12:59.427 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000678) opened an unlinked file! 2023-12-08T10:12:59.427 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000679) opened an unlinked file! 2023-12-08T10:12:59.427 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000680) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000681) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000682) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000683) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000684) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000685) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000686) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000687) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000688) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000689) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000690) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000691) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000692) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000693) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000694) opened an unlinked file! 2023-12-08T10:12:59.428 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000695) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000696) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000697) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000698) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000699) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000700) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000701) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000702) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000703) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000704) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000705) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000706) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000707) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000708) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000709) opened an unlinked file! 2023-12-08T10:12:59.429 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000710) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000711) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000712) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000713) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000714) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000715) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000716) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000717) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000718) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000719) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000720) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000721) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000722) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000723) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000724) opened an unlinked file! 2023-12-08T10:12:59.430 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000725) opened an unlinked file! 2023-12-08T10:12:59.431 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000726) opened an unlinked file! 2023-12-08T10:12:59.431 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000727) opened an unlinked file! 2023-12-08T10:12:59.431 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000728) opened an unlinked file! 2023-12-08T10:12:59.431 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000729) opened an unlinked file! 2023-12-08T10:12:59.431 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000730) opened an unlinked file! 2023-12-08T10:12:59.431 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000731) opened an unlinked file! 2023-12-08T10:12:59.431 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000732) opened an unlinked file! 2023-12-08T10:12:59.431 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000733) opened an unlinked file! 2023-12-08T10:12:59.431 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000734) opened an unlinked file! 2023-12-08T10:12:59.431 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000735) opened an unlinked file! 2023-12-08T10:12:59.431 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000736) opened an unlinked file! 2023-12-08T10:12:59.431 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000737) opened an unlinked file! 2023-12-08T10:12:59.431 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000738) opened an unlinked file! 2023-12-08T10:12:59.431 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000739) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000740) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000741) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000742) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000743) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000744) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000745) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000746) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000747) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000748) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000749) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000750) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000751) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000752) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000753) opened an unlinked file! 2023-12-08T10:12:59.432 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000754) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000755) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000756) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000757) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000758) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000759) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000760) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000761) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000762) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000763) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000764) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000765) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000766) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000767) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000768) opened an unlinked file! 2023-12-08T10:12:59.433 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000769) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000770) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000771) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000772) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000773) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000774) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000775) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000776) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000777) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000778) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000779) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000780) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000781) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000782) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000783) opened an unlinked file! 2023-12-08T10:12:59.434 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000784) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000785) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000786) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000787) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000788) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000789) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000790) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000791) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000792) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000793) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000794) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000795) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000796) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000797) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000798) opened an unlinked file! 2023-12-08T10:12:59.435 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000799) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000800) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000801) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000802) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000803) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000804) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000805) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000806) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000807) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000808) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000809) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000810) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000811) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000812) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000813) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000814) opened an unlinked file! 2023-12-08T10:12:59.436 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000815) opened an unlinked file! 2023-12-08T10:12:59.437 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000816) opened an unlinked file! 2023-12-08T10:12:59.437 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000817) opened an unlinked file! 2023-12-08T10:12:59.437 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000818) opened an unlinked file! 2023-12-08T10:12:59.437 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000819) opened an unlinked file! 2023-12-08T10:12:59.437 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000820) opened an unlinked file! 2023-12-08T10:12:59.437 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000821) opened an unlinked file! 2023-12-08T10:12:59.437 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000822) opened an unlinked file! 2023-12-08T10:12:59.437 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000823) opened an unlinked file! 2023-12-08T10:12:59.437 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000824) opened an unlinked file! 2023-12-08T10:12:59.437 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000825) opened an unlinked file! 2023-12-08T10:12:59.437 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000826) opened an unlinked file! 2023-12-08T10:12:59.437 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000827) opened an unlinked file! 2023-12-08T10:12:59.437 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000828) opened an unlinked file! 2023-12-08T10:12:59.437 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000829) opened an unlinked file! 2023-12-08T10:12:59.438 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000830) opened an unlinked file! 2023-12-08T10:12:59.438 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000831) opened an unlinked file! 2023-12-08T10:12:59.438 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000832) opened an unlinked file! 2023-12-08T10:12:59.438 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000833) opened an unlinked file! 2023-12-08T10:12:59.438 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000834) opened an unlinked file! 2023-12-08T10:12:59.438 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000835) opened an unlinked file! 2023-12-08T10:12:59.438 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000836) opened an unlinked file! 2023-12-08T10:12:59.438 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000837) opened an unlinked file! 2023-12-08T10:12:59.438 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000838) opened an unlinked file! 2023-12-08T10:12:59.438 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000839) opened an unlinked file! 2023-12-08T10:12:59.438 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000840) opened an unlinked file! 2023-12-08T10:12:59.438 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000841) opened an unlinked file! 2023-12-08T10:12:59.438 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000842) opened an unlinked file! 2023-12-08T10:12:59.438 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000843) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000844) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000845) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000846) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000847) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000848) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000849) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000850) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000851) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000852) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000853) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000854) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000855) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000856) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000857) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000858) opened an unlinked file! 2023-12-08T10:12:59.439 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000859) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000860) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000861) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000862) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000863) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000864) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000865) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000866) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000867) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000868) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000869) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000870) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000871) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000872) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000873) opened an unlinked file! 2023-12-08T10:12:59.440 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000874) opened an unlinked file! 2023-12-08T10:12:59.441 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000875) opened an unlinked file! 2023-12-08T10:12:59.441 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000876) opened an unlinked file! 2023-12-08T10:12:59.441 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000877) opened an unlinked file! 2023-12-08T10:12:59.441 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000878) opened an unlinked file! 2023-12-08T10:12:59.441 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000879) opened an unlinked file! 2023-12-08T10:12:59.441 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000880) opened an unlinked file! 2023-12-08T10:12:59.441 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000881) opened an unlinked file! 2023-12-08T10:12:59.441 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000882) opened an unlinked file! 2023-12-08T10:12:59.441 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000883) opened an unlinked file! 2023-12-08T10:12:59.441 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000884) opened an unlinked file! 2023-12-08T10:12:59.441 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000885) opened an unlinked file! 2023-12-08T10:12:59.441 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000886) opened an unlinked file! 2023-12-08T10:12:59.441 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000887) opened an unlinked file! 2023-12-08T10:12:59.441 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000888) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000889) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000890) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000891) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000892) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000893) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000894) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000895) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000896) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000897) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000898) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000899) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000900) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000901) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000902) opened an unlinked file! 2023-12-08T10:12:59.442 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000903) opened an unlinked file! 2023-12-08T10:12:59.443 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000904) opened an unlinked file! 2023-12-08T10:12:59.443 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000905) opened an unlinked file! 2023-12-08T10:12:59.443 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000906) opened an unlinked file! 2023-12-08T10:12:59.443 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000907) opened an unlinked file! 2023-12-08T10:12:59.443 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000908) opened an unlinked file! 2023-12-08T10:12:59.443 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000909) opened an unlinked file! 2023-12-08T10:12:59.443 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000910) opened an unlinked file! 2023-12-08T10:12:59.443 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000911) opened an unlinked file! 2023-12-08T10:12:59.443 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000912) opened an unlinked file! 2023-12-08T10:12:59.443 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000913) opened an unlinked file! 2023-12-08T10:12:59.443 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000914) opened an unlinked file! 2023-12-08T10:12:59.443 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000915) opened an unlinked file! 2023-12-08T10:12:59.443 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000916) opened an unlinked file! 2023-12-08T10:12:59.443 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000917) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000918) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000919) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000920) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000921) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000922) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000923) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000924) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000925) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000926) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000927) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000928) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000929) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000930) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000931) opened an unlinked file! 2023-12-08T10:12:59.444 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000932) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000933) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000934) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000935) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000936) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000937) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000938) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000939) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000940) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000941) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000942) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000943) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000944) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000945) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000946) opened an unlinked file! 2023-12-08T10:12:59.445 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000947) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000948) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000949) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000950) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000951) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000952) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000953) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000954) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000955) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000956) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000957) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000958) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000959) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000960) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000961) opened an unlinked file! 2023-12-08T10:12:59.446 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000962) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000963) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000964) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000965) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000966) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000967) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000968) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000969) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000970) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000971) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000972) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000973) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000974) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000975) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000976) opened an unlinked file! 2023-12-08T10:12:59.447 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000977) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000978) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000979) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000980) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000981) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000982) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000983) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000984) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000985) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000986) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000987) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000988) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000989) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000990) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000991) opened an unlinked file! 2023-12-08T10:12:59.448 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000992) opened an unlinked file! 2023-12-08T10:12:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000993) opened an unlinked file! 2023-12-08T10:12:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000994) opened an unlinked file! 2023-12-08T10:12:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000995) opened an unlinked file! 2023-12-08T10:12:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000996) opened an unlinked file! 2023-12-08T10:12:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000997) opened an unlinked file! 2023-12-08T10:12:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000998) opened an unlinked file! 2023-12-08T10:12:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file000999) opened an unlinked file! 2023-12-08T10:12:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001000) opened an unlinked file! 2023-12-08T10:12:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001001) opened an unlinked file! 2023-12-08T10:12:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001002) opened an unlinked file! 2023-12-08T10:12:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001003) opened an unlinked file! 2023-12-08T10:12:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001004) opened an unlinked file! 2023-12-08T10:12:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001005) opened an unlinked file! 2023-12-08T10:12:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001006) opened an unlinked file! 2023-12-08T10:12:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001007) opened an unlinked file! 2023-12-08T10:12:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001008) opened an unlinked file! 2023-12-08T10:12:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001009) opened an unlinked file! 2023-12-08T10:12:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001010) opened an unlinked file! 2023-12-08T10:12:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001011) opened an unlinked file! 2023-12-08T10:12:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001012) opened an unlinked file! 2023-12-08T10:12:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001013) opened an unlinked file! 2023-12-08T10:12:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001014) opened an unlinked file! 2023-12-08T10:12:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001015) opened an unlinked file! 2023-12-08T10:12:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001016) opened an unlinked file! 2023-12-08T10:12:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001017) opened an unlinked file! 2023-12-08T10:12:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001018) opened an unlinked file! 2023-12-08T10:12:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001019) opened an unlinked file! 2023-12-08T10:12:59.450 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001020) opened an unlinked file! 2023-12-08T10:12:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001021) opened an unlinked file! 2023-12-08T10:12:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001022) opened an unlinked file! 2023-12-08T10:12:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/426-dir/file001023) opened an unlinked file! 2023-12-08T10:12:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: test_file_handles TEST_DIR/426-dir 2023-12-08T10:12:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: test_file_handles TEST_DIR/426-dir -l 2023-12-08T10:12:59.451 INFO:teuthology.orchestra.run.smithi016.stdout: test_file_handles TEST_DIR/426-dir -u 2023-12-08T10:12:59.452 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/426 2023-12-08T10:12:59.452 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/426 2023-12-08T10:12:59.452 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T10:12:59.452 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:12:59.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T10:12:59.584 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T10:12:59.584 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T10:12:59.585 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/427) 2023-12-08T10:13:00.535 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:13:00.541 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:13:00.542 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:00.710 INFO:teuthology.orchestra.run.smithi016.stdout:generic/427 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:13:00.713 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/427 2023-12-08T10:13:00.713 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/427 2023-12-08T10:13:00.713 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:13:00.715 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:00.724 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:13:00.725 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/428) 2023-12-08T10:13:00.911 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:13:00.915 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:13:00.915 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:01.107 INFO:teuthology.orchestra.run.smithi016.stdout:generic/428 1s 2023-12-08T10:13:01.113 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/428 2023-12-08T10:13:01.113 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:13:01.115 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:01.124 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:13:01.124 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/429) 2023-12-08T10:13:01.300 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:13:01.304 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:13:01.304 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:01.441 INFO:teuthology.orchestra.run.smithi016.stdout:generic/429 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:13:01.445 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/429 2023-12-08T10:13:01.445 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/429 2023-12-08T10:13:01.445 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:13:01.447 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:01.456 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:13:01.456 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/430) 2023-12-08T10:13:01.629 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:13:01.633 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:13:01.634 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:01.886 INFO:teuthology.orchestra.run.smithi016.stdout:generic/430 0s 2023-12-08T10:13:01.891 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/430 2023-12-08T10:13:01.891 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:13:01.894 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:01.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:13:01.903 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/431) 2023-12-08T10:13:02.318 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:13:02.323 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:13:02.323 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:02.579 INFO:teuthology.orchestra.run.smithi016.stdout:generic/431 0s 2023-12-08T10:13:02.584 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/431 2023-12-08T10:13:02.584 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:13:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:02.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:13:02.595 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/432) 2023-12-08T10:13:03.014 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:13:03.018 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:13:03.018 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:03.267 INFO:teuthology.orchestra.run.smithi016.stdout:generic/432 0s 2023-12-08T10:13:03.273 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/432 2023-12-08T10:13:03.273 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:13:03.275 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:03.284 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:13:03.284 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/433) 2023-12-08T10:13:03.718 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:13:03.722 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:13:03.722 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:04.004 INFO:teuthology.orchestra.run.smithi016.stdout:generic/433 0s 2023-12-08T10:13:04.010 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/433 2023-12-08T10:13:04.010 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:13:04.012 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:04.021 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:13:04.021 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/434) 2023-12-08T10:13:04.450 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:13:04.455 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:13:04.455 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:04.666 INFO:teuthology.orchestra.run.smithi016.stdout:generic/434 - output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/434.out.bad) 2023-12-08T10:13:04.668 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/434.out 2023-12-08 08:39:53.780917638 +0000 2023-12-08T10:13:04.668 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/434.out.bad 2023-12-08 10:13:04.664086959 +0000 2023-12-08T10:13:04.668 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -9,6 +9,6 @@ 2023-12-08T10:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: copy_range: Bad file descriptor 2023-12-08T10:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: d41d8cd98f00b204e9800998ecf8427e TEST_DIR/test-434/copy 2023-12-08T10:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: Try to copy to a device 2023-12-08T10:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: -copy_range: Invalid argument 2023-12-08T10:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: +/tmp/tmp.hY9PAzp4zMtest/test-434/dev1: Permission denied 2023-12-08T10:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: Try to copy to a fifo 2023-12-08T10:13:04.669 INFO:teuthology.orchestra.run.smithi016.stdout: copy_range: Invalid argument 2023-12-08T10:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/434 2023-12-08T10:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/434 2023-12-08T10:13:04.671 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T10:13:04.673 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:04.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T10:13:04.682 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T10:13:04.682 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T10:13:04.682 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/435) 2023-12-08T10:13:05.128 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:13:05.132 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:05.272 INFO:teuthology.orchestra.run.smithi016.stdout:generic/435 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/435 2023-12-08T10:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/435 2023-12-08T10:13:05.276 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:13:05.278 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:05.286 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:13:05.287 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/436) 2023-12-08T10:13:05.469 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:13:05.473 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:06.419 INFO:teuthology.orchestra.run.smithi016.stdout:generic/436 1s 2023-12-08T10:13:06.426 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/436 2023-12-08T10:13:06.426 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:13:06.428 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:06.436 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:13:06.436 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/437) 2023-12-08T10:13:06.863 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:13:06.868 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:13:06.868 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:07.681 INFO:teuthology.orchestra.run.smithi016.stdout:generic/437 1s 2023-12-08T10:13:07.687 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/437 2023-12-08T10:13:07.687 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:13:07.689 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:13:07.698 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:13:07.698 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/438) 2023-12-08T10:13:07.871 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:13:07.875 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:13:07.875 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:07.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:14:07.698 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:14:10.499 INFO:teuthology.orchestra.run.smithi016.stdout:generic/438 63s 2023-12-08T10:14:10.504 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/438 2023-12-08T10:14:10.505 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:10.507 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:10.516 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:10.516 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/439) 2023-12-08T10:14:10.689 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:10.693 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:10.693 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:10.806 INFO:teuthology.orchestra.run.smithi016.stdout:generic/439 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:10.809 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/439 2023-12-08T10:14:10.809 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/439 2023-12-08T10:14:10.809 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:10.811 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:10.822 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:10.822 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/440) 2023-12-08T10:14:10.995 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:10.999 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:10.999 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:11.136 INFO:teuthology.orchestra.run.smithi016.stdout:generic/440 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:11.139 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/440 2023-12-08T10:14:11.139 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/440 2023-12-08T10:14:11.139 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:11.142 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:11.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:11.150 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/441) 2023-12-08T10:14:11.324 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:11.328 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:11.328 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:11.507 INFO:teuthology.orchestra.run.smithi016.stdout:generic/441 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:11.511 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/441 2023-12-08T10:14:11.511 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/441 2023-12-08T10:14:11.511 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:11.513 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:11.522 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:11.523 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/442) 2023-12-08T10:14:11.695 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:11.699 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:11.700 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:11.829 INFO:teuthology.orchestra.run.smithi016.stdout:generic/442 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:11.832 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/442 2023-12-08T10:14:11.832 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/442 2023-12-08T10:14:11.833 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:11.835 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:11.843 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:11.844 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/443) 2023-12-08T10:14:12.015 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:12.019 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:12.019 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:12.164 INFO:teuthology.orchestra.run.smithi016.stdout:generic/443 0s 2023-12-08T10:14:12.169 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/443 2023-12-08T10:14:12.169 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:12.172 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:12.180 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:12.180 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/444) 2023-12-08T10:14:12.353 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:12.357 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:12.358 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:12.498 INFO:teuthology.orchestra.run.smithi016.stdout:generic/444 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-12-08T10:14:12.502 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/444 2023-12-08T10:14:12.503 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/444 2023-12-08T10:14:12.503 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:12.505 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:12.513 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:12.514 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/445) 2023-12-08T10:14:12.688 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:12.692 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:12.692 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:13.557 INFO:teuthology.orchestra.run.smithi016.stdout:generic/445 1s 2023-12-08T10:14:13.562 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/445 2023-12-08T10:14:13.563 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:13.565 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:13.574 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:13.574 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/446) 2023-12-08T10:14:14.014 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:14.018 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:14.018 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:14.128 INFO:teuthology.orchestra.run.smithi016.stdout:generic/446 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:14.132 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/446 2023-12-08T10:14:14.132 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/446 2023-12-08T10:14:14.132 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:14.134 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:14.143 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:14.143 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/447) 2023-12-08T10:14:14.316 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:14.320 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:14.320 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:14.434 INFO:teuthology.orchestra.run.smithi016.stdout:generic/447 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:14.438 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/447 2023-12-08T10:14:14.438 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/447 2023-12-08T10:14:14.438 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:14.440 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:14.448 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:14.448 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/448) 2023-12-08T10:14:14.626 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:14.629 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:14.630 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:15.512 INFO:teuthology.orchestra.run.smithi016.stdout:generic/448 1s 2023-12-08T10:14:15.517 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/448 2023-12-08T10:14:15.517 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:15.520 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:15.528 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:15.529 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/449) 2023-12-08T10:14:15.971 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:15.975 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:15.975 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:16.085 INFO:teuthology.orchestra.run.smithi016.stdout:generic/449 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:16.088 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/449 2023-12-08T10:14:16.088 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/449 2023-12-08T10:14:16.088 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:16.091 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:16.099 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:16.100 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/450) 2023-12-08T10:14:16.272 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:16.276 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:16.276 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:17.289 INFO:teuthology.orchestra.run.smithi016.stdout:generic/450 1s 2023-12-08T10:14:17.295 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/450 2023-12-08T10:14:17.295 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:17.297 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:17.334 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:17.334 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/451) 2023-12-08T10:14:17.758 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:17.762 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:17.762 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:18.070 INFO:teuthology.orchestra.run.smithi016.stdout:generic/451 - output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/451.out.bad) 2023-12-08T10:14:18.071 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/451.out 2023-12-08 08:39:53.781917624 +0000 2023-12-08T10:14:18.071 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/451.out.bad 2023-12-08 10:14:18.067046301 +0000 2023-12-08T10:14:18.071 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -1,2 +1,9 @@ 2023-12-08T10:14:18.071 INFO:teuthology.orchestra.run.smithi016.stdout: QA output created by 451 2023-12-08T10:14:18.071 INFO:teuthology.orchestra.run.smithi016.stdout: +get stale data from buffer read 2023-12-08T10:14:18.072 INFO:teuthology.orchestra.run.smithi016.stdout: +00000000 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 UUUUUUUUUUUUUUUU 2023-12-08T10:14:18.072 INFO:teuthology.orchestra.run.smithi016.stdout: +* 2023-12-08T10:14:18.072 INFO:teuthology.orchestra.run.smithi016.stdout: +00080000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 2023-12-08T10:14:18.072 INFO:teuthology.orchestra.run.smithi016.stdout: +* 2023-12-08T10:14:18.072 INFO:teuthology.orchestra.run.smithi016.stdout: +00092000 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 UUUUUUUUUUUUUUUU 2023-12-08T10:14:18.072 INFO:teuthology.orchestra.run.smithi016.stdout: +000a0000 2023-12-08T10:14:18.072 INFO:teuthology.orchestra.run.smithi016.stdout: Silence is golden 2023-12-08T10:14:18.074 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/451 2023-12-08T10:14:18.074 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/451 2023-12-08T10:14:18.074 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T10:14:18.076 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:18.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T10:14:18.095 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T10:14:18.095 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T10:14:18.096 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/452) 2023-12-08T10:14:18.443 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:18.455 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:18.455 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:18.656 INFO:teuthology.orchestra.run.smithi016.stdout:generic/452 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:18.660 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/452 2023-12-08T10:14:18.660 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/452 2023-12-08T10:14:18.660 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:18.662 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:18.671 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:18.672 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/453) 2023-12-08T10:14:18.843 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:18.847 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:18.847 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:18.969 INFO:teuthology.orchestra.run.smithi016.stdout:generic/453 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:18.973 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/453 2023-12-08T10:14:18.973 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/453 2023-12-08T10:14:18.973 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:18.975 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:18.984 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:18.984 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/454) 2023-12-08T10:14:19.157 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:19.161 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:19.161 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:19.271 INFO:teuthology.orchestra.run.smithi016.stdout:generic/454 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:19.275 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/454 2023-12-08T10:14:19.275 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/454 2023-12-08T10:14:19.275 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:19.277 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:19.286 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:19.286 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/455) 2023-12-08T10:14:19.460 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:19.464 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:19.464 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:19.623 INFO:teuthology.orchestra.run.smithi016.stdout:generic/455 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:19.626 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/455 2023-12-08T10:14:19.626 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/455 2023-12-08T10:14:19.626 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:19.629 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:19.637 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:19.638 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/456) 2023-12-08T10:14:19.810 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:19.814 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:19.814 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:19.933 INFO:teuthology.orchestra.run.smithi016.stdout:generic/456 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:19.937 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/456 2023-12-08T10:14:19.937 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/456 2023-12-08T10:14:19.937 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:19.939 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:19.948 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:19.948 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/457) 2023-12-08T10:14:20.121 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:20.125 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:20.125 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:20.291 INFO:teuthology.orchestra.run.smithi016.stdout:generic/457 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:20.294 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/457 2023-12-08T10:14:20.294 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/457 2023-12-08T10:14:20.295 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:20.297 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:20.306 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:20.306 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/458) 2023-12-08T10:14:20.480 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:20.484 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:20.484 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:20.593 INFO:teuthology.orchestra.run.smithi016.stdout:generic/458 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:20.597 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/458 2023-12-08T10:14:20.597 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/458 2023-12-08T10:14:20.597 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:20.600 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:20.608 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:20.609 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/459) 2023-12-08T10:14:20.782 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:20.786 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:20.787 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:21.021 INFO:teuthology.orchestra.run.smithi016.stdout:generic/459 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:21.025 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/459 2023-12-08T10:14:21.025 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/459 2023-12-08T10:14:21.025 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:21.027 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:21.035 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:21.036 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/460) 2023-12-08T10:14:21.208 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:21.213 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:21.213 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:21.321 INFO:teuthology.orchestra.run.smithi016.stdout:generic/460 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:21.325 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/460 2023-12-08T10:14:21.325 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/460 2023-12-08T10:14:21.325 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:21.327 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:21.336 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:21.336 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/461) 2023-12-08T10:14:21.509 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:21.514 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:21.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:21.623 INFO:teuthology.orchestra.run.smithi016.stdout:generic/461 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:21.627 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/461 2023-12-08T10:14:21.627 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/461 2023-12-08T10:14:21.627 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:21.629 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:21.638 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:21.638 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/462) 2023-12-08T10:14:21.811 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:21.815 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:21.815 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:21.929 INFO:teuthology.orchestra.run.smithi016.stdout:generic/462 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:21.933 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/462 2023-12-08T10:14:21.933 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/462 2023-12-08T10:14:21.933 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:21.936 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:21.944 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:21.944 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/463) 2023-12-08T10:14:22.117 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:22.121 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:22.121 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:22.286 INFO:teuthology.orchestra.run.smithi016.stdout:generic/463 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T10:14:22.290 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/463 2023-12-08T10:14:22.290 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/463 2023-12-08T10:14:22.290 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:22.292 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:22.300 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:22.300 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/464) 2023-12-08T10:14:22.746 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:22.750 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:22.751 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:22.859 INFO:teuthology.orchestra.run.smithi016.stdout:generic/464 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:22.862 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/464 2023-12-08T10:14:22.863 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/464 2023-12-08T10:14:22.863 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:22.865 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:22.874 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:22.874 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/465) 2023-12-08T10:14:23.046 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:23.050 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:23.051 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:35.757 INFO:teuthology.orchestra.run.smithi016.stdout:generic/465 12s 2023-12-08T10:14:35.795 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/465 2023-12-08T10:14:35.795 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:35.798 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:35.818 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:35.819 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/466) 2023-12-08T10:14:36.119 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:36.123 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:36.123 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:36.292 INFO:teuthology.orchestra.run.smithi016.stdout:generic/466 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:36.295 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/466 2023-12-08T10:14:36.295 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/466 2023-12-08T10:14:36.295 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:36.297 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:36.306 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:36.306 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/467) 2023-12-08T10:14:36.477 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:36.483 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:36.484 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:37.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:14:37.842 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:14:41.861 INFO:teuthology.orchestra.run.smithi016.stdout:generic/467 - output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/467.out.bad) 2023-12-08T10:14:41.864 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/467.out 2023-12-08 08:39:53.782917609 +0000 2023-12-08T10:14:41.864 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/467.out.bad 2023-12-08 10:14:41.840709034 +0000 2023-12-08T10:14:41.864 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -1,5 +1,15 @@ 2023-12-08T10:14:41.864 INFO:teuthology.orchestra.run.smithi016.stdout: QA output created by 467 2023-12-08T10:14:41.864 INFO:teuthology.orchestra.run.smithi016.stdout: test_file_handles TEST_DIR/467-dir -dp 2023-12-08T10:14:41.864 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/467-dir/file000000) opened an unlinked file! 2023-12-08T10:14:41.864 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/467-dir/file000001) opened an unlinked file! 2023-12-08T10:14:41.864 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/467-dir/file000002) opened an unlinked file! 2023-12-08T10:14:41.865 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/467-dir/file000003) opened an unlinked file! 2023-12-08T10:14:41.865 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/467-dir/file000004) opened an unlinked file! 2023-12-08T10:14:41.865 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/467-dir/file000005) opened an unlinked file! 2023-12-08T10:14:41.865 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/467-dir/file000006) opened an unlinked file! 2023-12-08T10:14:41.865 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/467-dir/file000007) opened an unlinked file! 2023-12-08T10:14:41.865 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/467-dir/file000008) opened an unlinked file! 2023-12-08T10:14:41.865 INFO:teuthology.orchestra.run.smithi016.stdout: +open_by_handle(/tmp/tmp.hY9PAzp4zMtest/467-dir/file000009) opened an unlinked file! 2023-12-08T10:14:41.865 INFO:teuthology.orchestra.run.smithi016.stdout: test_file_handles TEST_DIR/467-dir -rp 2023-12-08T10:14:41.865 INFO:teuthology.orchestra.run.smithi016.stdout: test_file_handles TEST_DIR/467-dir -dkr 2023-12-08T10:14:41.865 INFO:teuthology.orchestra.run.smithi016.stdout: test_file_handles TEST_DIR/467-dir -lr 2023-12-08T10:14:41.881 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/467 2023-12-08T10:14:41.881 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/467 2023-12-08T10:14:41.881 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T10:14:41.918 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:42.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T10:14:42.104 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T10:14:42.105 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T10:14:42.105 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/468) 2023-12-08T10:14:43.139 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:43.143 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:43.143 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:43.296 INFO:teuthology.orchestra.run.smithi016.stdout:generic/468 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/468 2023-12-08T10:14:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/468 2023-12-08T10:14:43.300 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:43.302 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:43.311 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:43.311 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/469) 2023-12-08T10:14:43.487 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:43.491 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:43.491 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:45.064 INFO:teuthology.orchestra.run.smithi016.stdout:generic/469 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-12-08T10:14:45.068 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/469 2023-12-08T10:14:45.068 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/469 2023-12-08T10:14:45.068 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:45.070 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:45.078 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:45.079 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/470) 2023-12-08T10:14:45.541 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:45.544 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:45.545 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:45.704 INFO:teuthology.orchestra.run.smithi016.stdout:generic/470 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:14:45.707 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/470 2023-12-08T10:14:45.707 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/470 2023-12-08T10:14:45.707 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:14:45.709 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:14:45.718 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:14:45.718 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/471) 2023-12-08T10:14:45.892 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:14:45.896 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:14:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:15:08.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:15:08.066 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:15:38.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:15:38.230 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:15:38.283 INFO:tasks.ceph.mds.b.smithi112.stderr:2023-12-08T10:15:38.281+0000 7f0cd7b54640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 98757) UID: 0 2023-12-08T10:15:38.283 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-12-08T10:15:38.282+0000 7f52ecb1a640 -1 Fail to open '/proc/98757/cmdline' error = (2) No such file or directory 2023-12-08T10:15:38.283 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-12-08T10:15:38.282+0000 7f52ecb1a640 -1 received signal: Hangup from (PID: 98757) UID: 0 2023-12-08T10:15:41.447 INFO:teuthology.orchestra.run.smithi016.stdout:generic/471 56s 2023-12-08T10:15:41.452 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/471 2023-12-08T10:15:41.452 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:15:41.454 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:15:41.463 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:15:41.464 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/472) 2023-12-08T10:15:41.977 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:15:41.984 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:15:41.984 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:15:42.240 INFO:teuthology.orchestra.run.smithi016.stdout:generic/472 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:15:42.245 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/472 2023-12-08T10:15:42.246 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/472 2023-12-08T10:15:42.246 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:15:42.249 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:15:42.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:15:42.267 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/473) 2023-12-08T10:15:42.667 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:15:42.674 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:15:42.674 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:15:42.958 INFO:teuthology.orchestra.run.smithi016.stdout:generic/473 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-12-08T10:15:42.962 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/473 2023-12-08T10:15:42.962 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/473 2023-12-08T10:15:42.962 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:15:42.964 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:15:42.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:15:42.973 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/474) 2023-12-08T10:15:43.149 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:15:43.153 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:15:43.153 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:15:43.263 INFO:teuthology.orchestra.run.smithi016.stdout:generic/474 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:15:43.266 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/474 2023-12-08T10:15:43.267 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/474 2023-12-08T10:15:43.267 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:15:43.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:15:43.277 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:15:43.278 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/475) 2023-12-08T10:15:43.450 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:15:43.454 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:15:43.454 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:15:43.600 INFO:teuthology.orchestra.run.smithi016.stdout:generic/475 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:15:43.604 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/475 2023-12-08T10:15:43.604 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/475 2023-12-08T10:15:43.604 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:15:43.606 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:15:43.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:15:43.615 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/476) 2023-12-08T10:15:43.790 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:15:43.794 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:15:43.794 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:15:43.906 INFO:teuthology.orchestra.run.smithi016.stdout:generic/476 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:15:43.909 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/476 2023-12-08T10:15:43.909 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/476 2023-12-08T10:15:43.909 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:15:43.912 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:15:43.920 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:15:43.920 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/477) 2023-12-08T10:15:44.093 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:15:44.097 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:15:44.097 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:15:52.861 INFO:teuthology.orchestra.run.smithi016.stdout:generic/477 8s 2023-12-08T10:15:52.931 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/477 2023-12-08T10:15:52.931 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:15:52.933 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:15:53.028 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:15:53.028 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/478) 2023-12-08T10:15:54.038 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:15:54.043 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:15:54.043 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:04.101 INFO:teuthology.orchestra.run.smithi016.stdout:generic/478 - output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/478.out.bad) 2023-12-08T10:16:04.101 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/478.out 2023-12-08 08:39:53.783917595 +0000 2023-12-08T10:16:04.101 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/478.out.bad 2023-12-08 10:16:04.095492981 +0000 2023-12-08T10:16:04.101 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -10,7 +10,7 @@ 2023-12-08T10:16:04.101 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.101 INFO:teuthology.orchestra.run.smithi016.stdout: get wrlck 2023-12-08T10:16:04.101 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.101 INFO:teuthology.orchestra.run.smithi016.stdout: -lock could be placed 2023-12-08T10:16:04.101 INFO:teuthology.orchestra.run.smithi016.stdout: +get wrlck 2023-12-08T10:16:04.101 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.101 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.101 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.102 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -25,7 +25,7 @@ 2023-12-08T10:16:04.102 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.102 INFO:teuthology.orchestra.run.smithi016.stdout: get wrlck 2023-12-08T10:16:04.102 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.102 INFO:teuthology.orchestra.run.smithi016.stdout: -lock could be placed 2023-12-08T10:16:04.102 INFO:teuthology.orchestra.run.smithi016.stdout: +get wrlck 2023-12-08T10:16:04.102 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.102 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.102 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.102 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -37,7 +37,7 @@ 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: get rdlck 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: get rdlck 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: -lock could be placed 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: +get rdlck 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: get rdlck 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: get rdlck 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -46,7 +46,7 @@ 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: get rdlck 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: get rdlck 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: -lock could be placed 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: +get rdlck 2023-12-08T10:16:04.103 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.104 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.104 INFO:teuthology.orchestra.run.smithi016.stdout: lock could be placed 2023-12-08T10:16:04.107 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/478 2023-12-08T10:16:04.107 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/478 2023-12-08T10:16:04.107 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T10:16:04.107 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:04.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T10:16:04.116 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T10:16:04.117 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T10:16:04.117 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/479) 2023-12-08T10:16:04.300 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:04.304 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:04.304 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:04.459 INFO:teuthology.orchestra.run.smithi016.stdout:generic/479 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:04.462 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/479 2023-12-08T10:16:04.462 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/479 2023-12-08T10:16:04.463 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:04.465 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:04.486 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:04.487 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/480) 2023-12-08T10:16:04.660 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:04.665 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:04.665 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:04.783 INFO:teuthology.orchestra.run.smithi016.stdout:generic/480 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:04.787 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/480 2023-12-08T10:16:04.787 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/480 2023-12-08T10:16:04.787 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:04.789 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:04.797 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:04.797 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/481) 2023-12-08T10:16:04.969 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:04.973 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:04.973 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:05.092 INFO:teuthology.orchestra.run.smithi016.stdout:generic/481 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:05.096 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/481 2023-12-08T10:16:05.096 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/481 2023-12-08T10:16:05.096 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:05.098 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:05.107 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:05.108 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/482) 2023-12-08T10:16:05.281 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:05.285 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:05.285 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:05.447 INFO:teuthology.orchestra.run.smithi016.stdout:generic/482 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:05.449 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/482 2023-12-08T10:16:05.449 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/482 2023-12-08T10:16:05.449 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:05.451 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:05.460 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:05.460 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/483) 2023-12-08T10:16:05.632 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:05.636 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:05.636 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:05.766 INFO:teuthology.orchestra.run.smithi016.stdout:generic/483 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:05.769 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/483 2023-12-08T10:16:05.769 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/483 2023-12-08T10:16:05.769 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:05.772 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:05.780 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:05.780 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/484) 2023-12-08T10:16:05.960 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:05.965 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:05.965 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:06.097 INFO:teuthology.orchestra.run.smithi016.stdout:generic/484 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:06.101 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/484 2023-12-08T10:16:06.101 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/484 2023-12-08T10:16:06.101 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:06.104 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:06.113 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:06.113 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/485) 2023-12-08T10:16:06.286 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:06.291 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:06.291 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:07.171 INFO:teuthology.orchestra.run.smithi016.stdout:generic/485 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2023-12-08T10:16:07.174 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/485 2023-12-08T10:16:07.175 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/485 2023-12-08T10:16:07.175 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:07.177 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:07.186 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:07.186 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/486) 2023-12-08T10:16:07.652 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:07.656 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:07.656 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:07.798 INFO:teuthology.orchestra.run.smithi016.stdout:generic/486 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:07.802 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/486 2023-12-08T10:16:07.802 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/486 2023-12-08T10:16:07.802 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:07.804 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:07.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:07.813 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/487) 2023-12-08T10:16:07.994 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:07.998 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:07.998 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:08.127 INFO:teuthology.orchestra.run.smithi016.stdout:generic/487 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:08.130 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/487 2023-12-08T10:16:08.131 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/487 2023-12-08T10:16:08.131 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:08.133 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:08.153 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:08.153 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/488) 2023-12-08T10:16:08.329 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:08.334 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:08.334 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:08.448 INFO:teuthology.orchestra.run.smithi016.stdout:generic/488 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:08.452 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/488 2023-12-08T10:16:08.452 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/488 2023-12-08T10:16:08.452 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:08.454 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:08.463 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:08.463 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/489) 2023-12-08T10:16:08.635 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:08.639 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:08.639 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:08.760 INFO:teuthology.orchestra.run.smithi016.stdout:generic/489 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:08.763 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/489 2023-12-08T10:16:08.764 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/489 2023-12-08T10:16:08.764 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:08.765 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:08.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:08.774 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/490) 2023-12-08T10:16:08.948 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:08.952 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:08.952 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:09.884 INFO:teuthology.orchestra.run.smithi016.stdout:generic/490 1s 2023-12-08T10:16:09.890 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/490 2023-12-08T10:16:09.890 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:09.892 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:09.901 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:09.901 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/491) 2023-12-08T10:16:10.349 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:10.358 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:10.358 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:10.486 INFO:teuthology.orchestra.run.smithi016.stdout:generic/491 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:10.491 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/491 2023-12-08T10:16:10.491 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/491 2023-12-08T10:16:10.491 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:10.492 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:10.501 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:10.501 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/492) 2023-12-08T10:16:10.675 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:10.680 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:10.680 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:10.789 INFO:teuthology.orchestra.run.smithi016.stdout:generic/492 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:10.793 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/492 2023-12-08T10:16:10.793 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/492 2023-12-08T10:16:10.793 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:10.796 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:10.806 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:10.806 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/493) 2023-12-08T10:16:10.991 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:10.995 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:10.995 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:11.106 INFO:teuthology.orchestra.run.smithi016.stdout:generic/493 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:11.110 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/493 2023-12-08T10:16:11.110 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/493 2023-12-08T10:16:11.110 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:11.112 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:11.121 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:11.121 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/494) 2023-12-08T10:16:11.296 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:11.300 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:11.300 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:11.409 INFO:teuthology.orchestra.run.smithi016.stdout:generic/494 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:11.413 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/494 2023-12-08T10:16:11.413 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/494 2023-12-08T10:16:11.414 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:11.415 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:11.424 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:11.424 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/495) 2023-12-08T10:16:11.617 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:11.621 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:11.621 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:11.731 INFO:teuthology.orchestra.run.smithi016.stdout:generic/495 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:11.735 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/495 2023-12-08T10:16:11.735 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/495 2023-12-08T10:16:11.735 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:11.737 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:11.746 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:11.746 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/496) 2023-12-08T10:16:11.918 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:11.923 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:11.923 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:12.034 INFO:teuthology.orchestra.run.smithi016.stdout:generic/496 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:12.038 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/496 2023-12-08T10:16:12.039 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/496 2023-12-08T10:16:12.039 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:12.039 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:12.047 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:12.048 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/497) 2023-12-08T10:16:12.220 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:12.224 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:12.224 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:12.340 INFO:teuthology.orchestra.run.smithi016.stdout:generic/497 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:12.343 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/497 2023-12-08T10:16:12.343 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/497 2023-12-08T10:16:12.343 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:12.346 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:12.354 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:12.355 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/498) 2023-12-08T10:16:12.527 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:12.531 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:12.531 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:12.654 INFO:teuthology.orchestra.run.smithi016.stdout:generic/498 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:12.657 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/498 2023-12-08T10:16:12.657 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/498 2023-12-08T10:16:12.657 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:12.661 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:12.668 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:12.668 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/499) 2023-12-08T10:16:12.841 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:12.845 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:12.845 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:12.954 INFO:teuthology.orchestra.run.smithi016.stdout:generic/499 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/499 2023-12-08T10:16:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/499 2023-12-08T10:16:12.957 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:12.959 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:12.968 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:12.969 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/500) 2023-12-08T10:16:13.143 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:13.147 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:13.147 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:13.294 INFO:teuthology.orchestra.run.smithi016.stdout:generic/500 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:13.297 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/500 2023-12-08T10:16:13.298 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/500 2023-12-08T10:16:13.298 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:13.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:13.309 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/501) 2023-12-08T10:16:13.484 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:13.489 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:13.489 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:13.611 INFO:teuthology.orchestra.run.smithi016.stdout:generic/501 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:13.615 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/501 2023-12-08T10:16:13.615 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/501 2023-12-08T10:16:13.615 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:13.617 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:13.629 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:13.629 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/502) 2023-12-08T10:16:13.804 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:13.810 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:13.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:13.927 INFO:teuthology.orchestra.run.smithi016.stdout:generic/502 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:13.931 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/502 2023-12-08T10:16:13.931 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/502 2023-12-08T10:16:13.931 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:13.933 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:13.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:13.942 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/503) 2023-12-08T10:16:14.116 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:14.120 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:14.121 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:14.233 INFO:teuthology.orchestra.run.smithi016.stdout:generic/503 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:14.236 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/503 2023-12-08T10:16:14.236 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/503 2023-12-08T10:16:14.236 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:14.242 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:14.269 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:14.269 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/504) 2023-12-08T10:16:14.446 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:14.451 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:14.451 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:14.602 INFO:teuthology.orchestra.run.smithi016.stdout:generic/504 - output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/504.out.bad) 2023-12-08T10:16:14.604 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/504.out 2023-12-08 08:39:53.785917566 +0000 2023-12-08T10:16:14.605 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/504.out.bad 2023-12-08 10:16:14.600337677 +0000 2023-12-08T10:16:14.605 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -1,2 +1,3 @@ 2023-12-08T10:16:14.605 INFO:teuthology.orchestra.run.smithi016.stdout: QA output created by 504 2023-12-08T10:16:14.605 INFO:teuthology.orchestra.run.smithi016.stdout: +lock info not found 2023-12-08T10:16:14.605 INFO:teuthology.orchestra.run.smithi016.stdout: Silence is golden 2023-12-08T10:16:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/504 2023-12-08T10:16:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/504 2023-12-08T10:16:14.607 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T10:16:14.609 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:14.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T10:16:14.619 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T10:16:14.619 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T10:16:14.619 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/505) 2023-12-08T10:16:14.793 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:14.797 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:14.797 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:14.908 INFO:teuthology.orchestra.run.smithi016.stdout:generic/505 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:14.912 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/505 2023-12-08T10:16:14.912 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/505 2023-12-08T10:16:14.912 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:14.914 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:14.923 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:14.923 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/506) 2023-12-08T10:16:15.099 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:15.102 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:15.103 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:15.214 INFO:teuthology.orchestra.run.smithi016.stdout:generic/506 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:15.217 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/506 2023-12-08T10:16:15.217 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/506 2023-12-08T10:16:15.217 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:15.219 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:15.228 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:15.228 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/507) 2023-12-08T10:16:15.404 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:15.408 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:15.408 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:15.571 INFO:teuthology.orchestra.run.smithi016.stdout:generic/507 [not run] file system doesn't support chattr +ASai 2023-12-08T10:16:15.575 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/507 2023-12-08T10:16:15.575 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/507 2023-12-08T10:16:15.575 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:15.577 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:15.586 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:15.586 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/508) 2023-12-08T10:16:15.761 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:15.766 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:15.766 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:15.877 INFO:teuthology.orchestra.run.smithi016.stdout:generic/508 [not run] lsattr not supported by test filesystem type: ceph-fuse 2023-12-08T10:16:15.881 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/508 2023-12-08T10:16:15.881 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/508 2023-12-08T10:16:15.881 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:15.886 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:15.891 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:15.891 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/509) 2023-12-08T10:16:16.066 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:16.071 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:16.071 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:16.192 INFO:teuthology.orchestra.run.smithi016.stdout:generic/509 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/509 2023-12-08T10:16:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/509 2023-12-08T10:16:16.196 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:16.198 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:16.207 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:16.208 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/510) 2023-12-08T10:16:16.382 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:16.386 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:16.386 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:16.511 INFO:teuthology.orchestra.run.smithi016.stdout:generic/510 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:16.514 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/510 2023-12-08T10:16:16.515 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/510 2023-12-08T10:16:16.515 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:16.515 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:16.523 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:16.524 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/511) 2023-12-08T10:16:16.697 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:16.701 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:16.702 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:16.811 INFO:teuthology.orchestra.run.smithi016.stdout:generic/511 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:16.814 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/511 2023-12-08T10:16:16.814 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/511 2023-12-08T10:16:16.814 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:16.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:16.839 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:16.839 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/512) 2023-12-08T10:16:17.013 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:17.017 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:17.017 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:17.140 INFO:teuthology.orchestra.run.smithi016.stdout:generic/512 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:17.143 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/512 2023-12-08T10:16:17.143 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/512 2023-12-08T10:16:17.143 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:17.146 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:17.154 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:17.155 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/513) 2023-12-08T10:16:17.327 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:17.331 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:17.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:17.442 INFO:teuthology.orchestra.run.smithi016.stdout:generic/513 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:17.446 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/513 2023-12-08T10:16:17.446 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/513 2023-12-08T10:16:17.446 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:17.448 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:17.457 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:17.457 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/514) 2023-12-08T10:16:17.631 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:17.635 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:17.635 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:17.750 INFO:teuthology.orchestra.run.smithi016.stdout:generic/514 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:17.753 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/514 2023-12-08T10:16:17.753 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/514 2023-12-08T10:16:17.754 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:17.756 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:17.777 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:17.777 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/515) 2023-12-08T10:16:17.951 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:17.955 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:17.956 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:18.066 INFO:teuthology.orchestra.run.smithi016.stdout:generic/515 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:18.069 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/515 2023-12-08T10:16:18.069 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/515 2023-12-08T10:16:18.069 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:18.072 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:18.081 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:18.082 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/516) 2023-12-08T10:16:18.254 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:18.258 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:18.258 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:18.415 INFO:teuthology.orchestra.run.smithi016.stdout:generic/516 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-12-08T10:16:18.419 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/516 2023-12-08T10:16:18.419 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/516 2023-12-08T10:16:18.419 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:18.421 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:18.430 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:18.430 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/517) 2023-12-08T10:16:18.881 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:18.887 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:18.887 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:18.994 INFO:teuthology.orchestra.run.smithi016.stdout:generic/517 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:18.998 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/517 2023-12-08T10:16:18.998 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/517 2023-12-08T10:16:18.998 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:19.000 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:19.009 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:19.009 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/518) 2023-12-08T10:16:19.182 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:19.186 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:19.187 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:19.297 INFO:teuthology.orchestra.run.smithi016.stdout:generic/518 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:19.301 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/518 2023-12-08T10:16:19.301 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/518 2023-12-08T10:16:19.301 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:19.303 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:19.312 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:19.312 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/519) 2023-12-08T10:16:19.485 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:19.490 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:19.490 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:19.598 INFO:teuthology.orchestra.run.smithi016.stdout:generic/519 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:19.601 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/519 2023-12-08T10:16:19.601 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/519 2023-12-08T10:16:19.601 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:19.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:19.612 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:19.612 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/520) 2023-12-08T10:16:19.785 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:19.788 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:19.789 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:19.913 INFO:teuthology.orchestra.run.smithi016.stdout:generic/520 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T10:16:19.916 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/520 2023-12-08T10:16:19.916 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/520 2023-12-08T10:16:19.916 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:16:19.919 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:16:19.938 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:16:19.939 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/521) 2023-12-08T10:16:20.111 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:16:20.115 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:16:20.116 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:18:29.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:18:29.788 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:18:59.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:18:59.930 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:19:29.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:19:29.987 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:20:00.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:20:00.079 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:20:30.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:20:30.205 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:21:00.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:21:00.293 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:21:30.430 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:21:30.433 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:22:00.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:22:00.497 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:22:30.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:22:30.603 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:22:30.635 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T10:22:30.633+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98926) UID: 0 2023-12-08T10:22:30.635 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T10:22:30.633+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98926) UID: 0 2023-12-08T10:22:30.635 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T10:22:30.633+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98926) UID: 0 2023-12-08T10:22:30.636 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T10:22:30.633+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 98926) UID: 0 2023-12-08T10:24:43.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:24:43.831 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:25:13.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:25:13.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:25:43.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:25:44.000 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:26:14.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:26:14.115 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:26:44.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:26:44.247 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:27:14.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:27:14.347 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:27:44.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:27:44.694 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:28:14.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:28:14.758 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:28:44.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:28:44.873 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:29:14.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:29:14.981 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:29:45.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:29:45.045 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:30:15.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:30:15.368 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:30:45.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:30:45.432 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:31:15.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:31:15.522 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:31:45.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:31:45.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:32:15.791 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:32:15.794 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:32:46.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:32:46.074 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:33:16.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:33:16.207 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:33:46.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:33:46.273 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:34:16.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:34:16.396 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:34:46.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:34:46.466 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:35:16.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:35:16.565 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:35:46.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:35:46.629 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:36:16.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:36:16.686 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:36:16.770 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T10:36:16.768+0000 7f6d0f17a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 496720) UID: 0 2023-12-08T10:36:16.771 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T10:36:16.768+0000 7eff8ec05640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 496720) UID: 0 2023-12-08T10:36:16.771 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T10:36:16.768+0000 7ff25129f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 496720) UID: 0 2023-12-08T10:36:16.771 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T10:36:16.768+0000 7fdb0d8f4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 496720) UID: 0 2023-12-08T10:39:23.713 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:39:23.717 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:39:53.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:39:53.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:40:24.032 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:40:24.035 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:40:54.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:40:54.165 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:41:24.229 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:41:24.232 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:41:54.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:41:54.637 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:42:24.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:42:24.704 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:42:54.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:42:54.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:43:24.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:43:24.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:43:55.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:43:55.175 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:43:55.205 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T10:43:55.203+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 99559) UID: 0 2023-12-08T10:43:55.206 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T10:43:55.203+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 99559) UID: 0 2023-12-08T10:43:55.206 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T10:43:55.203+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 99559) UID: 0 2023-12-08T10:43:55.206 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T10:43:55.203+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 99559) UID: 0 2023-12-08T10:46:49.225 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:46:49.228 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:47:19.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:47:19.474 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:47:49.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:47:49.541 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:48:19.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:48:19.678 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:48:49.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:48:49.810 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:49:19.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:49:19.950 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:49:50.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:49:50.106 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:49:50.141 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T10:49:50.139+0000 7eff8ec05640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 497038) UID: 0 2023-12-08T10:49:50.141 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T10:49:50.139+0000 7f6d0f17a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 497038) UID: 0 2023-12-08T10:49:50.141 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T10:49:50.139+0000 7ff25129f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 497038) UID: 0 2023-12-08T10:49:50.142 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T10:49:50.139+0000 7fdb0d8f4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 497038) UID: 0 2023-12-08T10:52:34.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:52:34.682 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:53:04.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:53:04.807 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:53:34.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:53:34.879 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:54:04.962 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:54:04.965 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:54:35.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:54:35.242 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:55:05.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:55:05.312 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:55:35.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:55:35.371 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:56:05.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:56:05.479 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:56:35.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:56:35.576 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:57:05.695 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:57:05.698 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:57:35.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:57:35.979 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:58:06.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:58:06.062 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:58:36.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:58:36.491 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:59:06.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:59:06.557 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:59:35.912 INFO:teuthology.orchestra.run.smithi016.stdout:generic/521 2595s 2023-12-08T10:59:35.927 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/521 2023-12-08T10:59:35.927 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T10:59:35.930 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:59:35.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T10:59:35.942 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/522) 2023-12-08T10:59:36.395 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T10:59:36.400 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T10:59:36.400 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T10:59:36.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T10:59:36.671 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:00:06.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:00:06.830 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:00:36.899 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:00:36.902 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:01:06.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:01:06.964 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:01:37.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:01:37.257 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:02:07.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:02:07.321 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:02:37.399 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:02:37.402 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:03:07.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:03:07.500 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:03:37.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:03:37.557 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:04:07.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:04:07.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:04:37.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:04:37.702 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:05:07.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:05:08.001 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:05:38.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:05:38.082 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:05:38.126 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T11:05:38.124+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 100204) UID: 0 2023-12-08T11:05:38.126 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T11:05:38.124+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 100204) UID: 0 2023-12-08T11:05:38.126 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T11:05:38.124+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 100204) UID: 0 2023-12-08T11:05:38.127 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T11:05:38.124+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 100204) UID: 0 2023-12-08T11:08:30.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:08:30.332 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:09:00.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:09:00.465 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:09:30.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:09:30.696 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:10:00.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:10:00.758 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:10:30.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:10:30.823 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:11:00.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:11:00.970 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:11:31.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:11:31.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:12:01.223 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:12:01.226 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:12:31.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:12:31.357 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:13:01.416 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:13:01.419 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:13:31.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:13:31.496 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:14:01.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:14:01.777 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:14:31.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:14:31.840 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:14:31.871 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-12-08T11:14:31.869+0000 7f52ecb1a640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 100448) UID: 0 2023-12-08T11:14:31.871 INFO:tasks.ceph.mds.b.smithi112.stderr:2023-12-08T11:14:31.869+0000 7f0cd7b54640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 100448) UID: 0 2023-12-08T11:17:10.249 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:17:10.252 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:17:40.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:17:40.331 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:18:10.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:18:10.392 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:18:40.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:18:40.478 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:19:10.623 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:19:10.626 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:19:40.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:19:40.789 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:20:10.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:20:10.852 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:20:41.160 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:20:41.163 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:21:11.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:21:11.237 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:21:41.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:21:41.340 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:22:11.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:22:11.439 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:22:41.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:22:41.570 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:23:11.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:23:11.885 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:23:41.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:23:41.942 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:24:12.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:24:12.048 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:24:42.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:24:42.164 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:25:12.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:25:12.528 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:25:42.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:25:42.770 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:26:12.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:26:12.833 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:26:42.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:26:42.930 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:27:12.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:27:12.990 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:27:43.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:27:43.077 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:28:13.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:28:13.136 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:28:43.228 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:28:43.231 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:29:13.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:29:13.346 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:29:43.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:29:43.452 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:29:43.482 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T11:29:43.481+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 100932) UID: 0 2023-12-08T11:29:43.483 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T11:29:43.481+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 100932) UID: 0 2023-12-08T11:29:43.483 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T11:29:43.481+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 100932) UID: 0 2023-12-08T11:29:43.483 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T11:29:43.481+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 100932) UID: 0 2023-12-08T11:30:31.094 INFO:teuthology.orchestra.run.smithi016.stdout:generic/522 1855s 2023-12-08T11:30:31.125 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/522 2023-12-08T11:30:31.125 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:31.127 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:31.143 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:31.144 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/523) 2023-12-08T11:30:31.425 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:31.429 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:31.429 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:31.542 INFO:teuthology.orchestra.run.smithi016.stdout:generic/523 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T11:30:31.545 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/523 2023-12-08T11:30:31.545 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/523 2023-12-08T11:30:31.545 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:31.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:31.557 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:31.557 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/524) 2023-12-08T11:30:31.729 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:31.733 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:31.734 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:31.846 INFO:teuthology.orchestra.run.smithi016.stdout:generic/524 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T11:30:31.850 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/524 2023-12-08T11:30:31.850 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/524 2023-12-08T11:30:31.850 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:31.852 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:31.861 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:31.861 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/525) 2023-12-08T11:30:32.035 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:32.038 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:32.039 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:32.149 INFO:teuthology.orchestra.run.smithi016.stdout:generic/525 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T11:30:32.152 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/525 2023-12-08T11:30:32.153 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/525 2023-12-08T11:30:32.153 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:32.155 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:32.164 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:32.164 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/526) 2023-12-08T11:30:32.337 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:32.341 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:32.341 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:32.483 INFO:teuthology.orchestra.run.smithi016.stdout:generic/526 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T11:30:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/526 2023-12-08T11:30:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/526 2023-12-08T11:30:32.487 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:32.489 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:32.498 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:32.498 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/527) 2023-12-08T11:30:32.671 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:32.675 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:32.675 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:32.805 INFO:teuthology.orchestra.run.smithi016.stdout:generic/527 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T11:30:32.808 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/527 2023-12-08T11:30:32.808 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/527 2023-12-08T11:30:32.808 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:32.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:32.819 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:32.819 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/528) 2023-12-08T11:30:32.992 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:32.996 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:32.996 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:33.135 INFO:teuthology.orchestra.run.smithi016.stdout:generic/528 [not run] inode creation time not supported by this filesystem 2023-12-08T11:30:33.138 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/528 2023-12-08T11:30:33.138 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/528 2023-12-08T11:30:33.139 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:33.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:33.150 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/529) 2023-12-08T11:30:33.324 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:33.327 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:33.474 INFO:teuthology.orchestra.run.smithi016.stdout:generic/529 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-12-08T11:30:33.477 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/529 2023-12-08T11:30:33.477 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/529 2023-12-08T11:30:33.477 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:33.488 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:33.488 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/530) 2023-12-08T11:30:33.662 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:33.666 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:33.775 INFO:teuthology.orchestra.run.smithi016.stdout:generic/530 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T11:30:33.778 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/530 2023-12-08T11:30:33.778 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/530 2023-12-08T11:30:33.779 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:33.789 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:33.790 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/531) 2023-12-08T11:30:33.962 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:33.966 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:34.074 INFO:teuthology.orchestra.run.smithi016.stdout:generic/531 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T11:30:34.078 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/531 2023-12-08T11:30:34.078 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/531 2023-12-08T11:30:34.078 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:34.088 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:34.089 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/532) 2023-12-08T11:30:34.264 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:34.268 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:34.269 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:34.448 INFO:teuthology.orchestra.run.smithi016.stdout:generic/532 0s 2023-12-08T11:30:34.453 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/532 2023-12-08T11:30:34.453 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:34.456 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:34.464 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:34.465 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/533) 2023-12-08T11:30:34.638 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:34.642 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:34.642 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:34.984 INFO:teuthology.orchestra.run.smithi016.stdout:generic/533 0s 2023-12-08T11:30:34.990 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/533 2023-12-08T11:30:34.990 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:34.992 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:35.000 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:35.001 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/534) 2023-12-08T11:30:35.177 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:35.181 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:35.181 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:35.305 INFO:teuthology.orchestra.run.smithi016.stdout:generic/534 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T11:30:35.308 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/534 2023-12-08T11:30:35.308 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/534 2023-12-08T11:30:35.308 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:35.310 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:35.319 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:35.320 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/535) 2023-12-08T11:30:35.493 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:35.497 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:35.497 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:35.617 INFO:teuthology.orchestra.run.smithi016.stdout:generic/535 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T11:30:35.621 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/535 2023-12-08T11:30:35.621 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/535 2023-12-08T11:30:35.621 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:35.624 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:35.632 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:35.632 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/536) 2023-12-08T11:30:35.805 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:35.809 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:35.809 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:35.916 INFO:teuthology.orchestra.run.smithi016.stdout:generic/536 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T11:30:35.920 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/536 2023-12-08T11:30:35.920 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/536 2023-12-08T11:30:35.920 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:35.922 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:35.934 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:35.934 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/537) 2023-12-08T11:30:36.107 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:36.111 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:36.111 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:36.221 INFO:teuthology.orchestra.run.smithi016.stdout:generic/537 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T11:30:36.225 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/537 2023-12-08T11:30:36.225 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/537 2023-12-08T11:30:36.225 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T11:30:36.227 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:30:36.236 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T11:30:36.236 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/538) 2023-12-08T11:30:36.411 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T11:30:36.415 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T11:30:36.415 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T11:32:30.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:32:30.345 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:33:00.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:33:00.584 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:33:30.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:33:30.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:34:00.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:34:00.840 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:34:30.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:34:30.897 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:35:00.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:35:00.978 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:35:31.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:35:31.053 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:36:01.322 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:36:01.325 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:36:31.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:36:31.413 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:36:31.465 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T11:36:31.463+0000 7eff8ec05640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 506916) UID: 0 2023-12-08T11:36:31.465 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T11:36:31.463+0000 7f6d0f17a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 506916) UID: 0 2023-12-08T11:36:31.466 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T11:36:31.463+0000 7fdb0d8f4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 506916) UID: 0 2023-12-08T11:36:31.466 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T11:36:31.463+0000 7ff25129f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 506916) UID: 0 2023-12-08T11:39:19.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:39:19.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:39:49.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:39:49.239 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:40:19.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:40:19.312 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:40:49.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:40:49.394 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:41:19.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:41:19.484 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:41:49.541 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:41:49.544 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:42:19.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:42:19.609 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:42:49.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:42:49.686 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:43:19.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:43:19.759 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:43:50.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:43:50.008 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:44:20.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:44:20.147 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:44:50.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:44:50.212 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:45:20.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:45:20.276 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:45:50.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:45:50.332 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:46:20.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:46:20.422 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:46:50.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:46:50.479 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:47:20.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:47:20.537 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:47:50.599 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:47:50.602 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:48:20.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:48:20.731 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:48:50.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:48:50.798 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:49:20.899 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:49:20.902 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:49:51.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:49:51.221 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:50:21.283 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:50:21.286 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:50:51.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:50:51.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:51:21.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:51:21.511 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:51:51.624 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:51:51.627 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:52:21.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:52:21.706 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:52:51.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:52:51.772 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:53:21.840 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:53:21.844 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:53:51.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:53:51.901 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:54:22.021 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:54:22.024 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:54:52.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:54:52.207 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:55:22.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:55:22.288 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:55:52.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:55:52.427 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:56:22.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:56:22.492 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:56:52.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:56:52.558 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:57:22.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:57:22.615 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:57:52.674 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:57:52.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:58:22.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:58:22.810 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:58:52.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:58:52.900 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:59:23.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:59:23.039 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:59:53.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T11:59:53.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:00:23.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:00:23.268 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:00:53.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:00:53.647 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:01:23.708 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:01:23.711 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:01:53.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:01:53.789 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:02:23.933 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:02:23.936 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:02:54.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:02:54.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:03:24.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:03:24.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:03:54.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:03:54.264 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:04:24.325 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:04:24.328 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:04:54.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:04:54.410 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:05:24.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:05:24.717 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:05:54.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:05:54.782 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:06:24.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:06:24.912 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:06:55.225 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:06:55.228 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:07:25.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:07:25.525 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:07:55.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:07:55.642 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:08:26.018 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:08:26.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:08:39.648 INFO:teuthology.orchestra.run.smithi016.stdout:generic/538 2283s 2023-12-08T12:08:39.653 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/538 2023-12-08T12:08:39.654 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:39.656 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:39.664 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:39.665 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/539) 2023-12-08T12:08:40.146 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:40.150 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:40.150 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:41.929 INFO:teuthology.orchestra.run.smithi016.stdout:generic/539 1s 2023-12-08T12:08:41.934 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/539 2023-12-08T12:08:41.934 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:41.937 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:41.945 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:41.946 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/540) 2023-12-08T12:08:42.401 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:42.405 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:42.405 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:42.524 INFO:teuthology.orchestra.run.smithi016.stdout:generic/540 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:42.528 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/540 2023-12-08T12:08:42.528 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/540 2023-12-08T12:08:42.528 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:42.530 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:42.539 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:42.540 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/541) 2023-12-08T12:08:42.714 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:42.718 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:42.718 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:42.835 INFO:teuthology.orchestra.run.smithi016.stdout:generic/541 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:42.839 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/541 2023-12-08T12:08:42.839 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/541 2023-12-08T12:08:42.839 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:42.841 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:42.850 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:42.850 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/542) 2023-12-08T12:08:43.024 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:43.028 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:43.028 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:43.138 INFO:teuthology.orchestra.run.smithi016.stdout:generic/542 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:43.141 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/542 2023-12-08T12:08:43.141 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/542 2023-12-08T12:08:43.141 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:43.144 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:43.152 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:43.152 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/543) 2023-12-08T12:08:43.324 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:43.329 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:43.329 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:43.439 INFO:teuthology.orchestra.run.smithi016.stdout:generic/543 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:43.442 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/543 2023-12-08T12:08:43.442 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/543 2023-12-08T12:08:43.442 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:43.444 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:43.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:43.453 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/544) 2023-12-08T12:08:43.627 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:43.631 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:43.631 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:43.742 INFO:teuthology.orchestra.run.smithi016.stdout:generic/544 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:43.745 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/544 2023-12-08T12:08:43.745 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/544 2023-12-08T12:08:43.745 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:43.747 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:43.755 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:43.756 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/545) 2023-12-08T12:08:43.930 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:43.934 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:43.934 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:44.128 INFO:teuthology.orchestra.run.smithi016.stdout:generic/545 [not run] file system doesn't support chattr +i 2023-12-08T12:08:44.132 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/545 2023-12-08T12:08:44.132 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/545 2023-12-08T12:08:44.132 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:44.134 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:44.143 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:44.143 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/546) 2023-12-08T12:08:44.319 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:44.323 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:44.323 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:44.511 INFO:teuthology.orchestra.run.smithi016.stdout:generic/546 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:44.514 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/546 2023-12-08T12:08:44.515 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/546 2023-12-08T12:08:44.515 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:44.517 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:44.525 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:44.526 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/547) 2023-12-08T12:08:44.698 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:44.702 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:44.702 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:44.826 INFO:teuthology.orchestra.run.smithi016.stdout:generic/547 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:44.829 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/547 2023-12-08T12:08:44.829 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/547 2023-12-08T12:08:44.829 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:44.832 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:44.840 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:44.840 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/548) 2023-12-08T12:08:45.014 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:45.018 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:45.019 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:45.182 INFO:teuthology.orchestra.run.smithi016.stdout:generic/548 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/548 2023-12-08T12:08:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/548 2023-12-08T12:08:45.186 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:45.188 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:45.196 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:45.197 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/549) 2023-12-08T12:08:45.372 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:45.376 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:45.376 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:45.513 INFO:teuthology.orchestra.run.smithi016.stdout:generic/549 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:45.517 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/549 2023-12-08T12:08:45.517 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/549 2023-12-08T12:08:45.517 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:45.519 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:45.528 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:45.528 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/550) 2023-12-08T12:08:45.702 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:45.706 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:45.707 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:45.843 INFO:teuthology.orchestra.run.smithi016.stdout:generic/550 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:45.846 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/550 2023-12-08T12:08:45.846 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/550 2023-12-08T12:08:45.846 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:45.849 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:45.857 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:45.857 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/551) 2023-12-08T12:08:46.032 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:46.036 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:46.036 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:46.151 INFO:teuthology.orchestra.run.smithi016.stdout:generic/551 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:46.154 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/551 2023-12-08T12:08:46.154 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/551 2023-12-08T12:08:46.155 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:46.157 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:46.165 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:46.165 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/552) 2023-12-08T12:08:46.341 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:46.345 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:46.345 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:46.468 INFO:teuthology.orchestra.run.smithi016.stdout:generic/552 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:46.471 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/552 2023-12-08T12:08:46.471 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/552 2023-12-08T12:08:46.471 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:46.474 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:46.482 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:46.483 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/553) 2023-12-08T12:08:46.657 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:46.661 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:46.661 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:46.840 INFO:teuthology.orchestra.run.smithi016.stdout:generic/553 [not run] xfs_io chattr +i failed (old kernel/wrong fs/bad args?) 2023-12-08T12:08:46.844 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/553 2023-12-08T12:08:46.844 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/553 2023-12-08T12:08:46.844 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:46.846 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:46.854 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:46.855 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/554) 2023-12-08T12:08:47.321 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:47.325 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:47.325 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:47.440 INFO:teuthology.orchestra.run.smithi016.stdout:generic/554 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:47.443 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/554 2023-12-08T12:08:47.444 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/554 2023-12-08T12:08:47.444 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:47.446 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:47.465 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:47.465 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/555) 2023-12-08T12:08:47.639 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:47.643 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:47.643 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:47.793 INFO:teuthology.orchestra.run.smithi016.stdout:generic/555 [not run] xfs_io chattr +ia failed (old kernel/wrong fs/bad args?) 2023-12-08T12:08:47.797 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/555 2023-12-08T12:08:47.797 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/555 2023-12-08T12:08:47.797 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:47.799 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:47.807 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:47.808 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/556) 2023-12-08T12:08:48.261 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:48.266 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:48.266 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:48.395 INFO:teuthology.orchestra.run.smithi016.stdout:generic/556 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:48.399 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/556 2023-12-08T12:08:48.399 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/556 2023-12-08T12:08:48.399 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:48.401 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:48.410 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:48.410 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/557) 2023-12-08T12:08:48.583 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:48.587 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:48.587 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:48.710 INFO:teuthology.orchestra.run.smithi016.stdout:generic/557 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:48.713 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/557 2023-12-08T12:08:48.713 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/557 2023-12-08T12:08:48.713 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:48.715 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:48.724 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:48.724 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/558) 2023-12-08T12:08:48.898 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:48.902 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:48.902 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:49.043 INFO:teuthology.orchestra.run.smithi016.stdout:generic/558 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:49.047 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/558 2023-12-08T12:08:49.047 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/558 2023-12-08T12:08:49.047 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:49.049 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:49.066 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:49.066 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/559) 2023-12-08T12:08:49.241 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:49.245 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:49.245 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:49.355 INFO:teuthology.orchestra.run.smithi016.stdout:generic/559 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:49.359 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/559 2023-12-08T12:08:49.359 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/559 2023-12-08T12:08:49.359 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:49.361 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:49.370 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:49.370 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/560) 2023-12-08T12:08:49.544 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:49.548 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:49.548 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:49.658 INFO:teuthology.orchestra.run.smithi016.stdout:generic/560 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:49.661 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/560 2023-12-08T12:08:49.661 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/560 2023-12-08T12:08:49.661 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:49.666 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:49.672 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:49.672 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/561) 2023-12-08T12:08:49.846 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:49.850 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:49.850 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:49.961 INFO:teuthology.orchestra.run.smithi016.stdout:generic/561 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/561 2023-12-08T12:08:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/561 2023-12-08T12:08:49.965 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:49.967 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:49.976 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:49.976 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/562) 2023-12-08T12:08:50.152 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:50.156 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:50.156 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:50.266 INFO:teuthology.orchestra.run.smithi016.stdout:generic/562 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:50.269 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/562 2023-12-08T12:08:50.269 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/562 2023-12-08T12:08:50.269 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:50.272 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:50.280 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:50.280 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/563) 2023-12-08T12:08:50.455 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:50.459 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:50.460 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:50.602 INFO:teuthology.orchestra.run.smithi016.stdout:generic/563 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:50.606 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/563 2023-12-08T12:08:50.606 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/563 2023-12-08T12:08:50.606 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:50.608 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:50.627 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:50.627 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/564) 2023-12-08T12:08:50.802 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:50.806 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:50.806 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:51.214 INFO:teuthology.orchestra.run.smithi016.stdout:generic/564 1s 2023-12-08T12:08:51.220 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/564 2023-12-08T12:08:51.220 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:51.222 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:51.231 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:51.231 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/565) 2023-12-08T12:08:51.687 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:51.690 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:51.691 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:51.806 INFO:teuthology.orchestra.run.smithi016.stdout:generic/565 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:51.810 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/565 2023-12-08T12:08:51.810 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/565 2023-12-08T12:08:51.810 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:51.812 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:51.820 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:51.821 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/566) 2023-12-08T12:08:51.995 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:51.999 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:52.000 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:52.110 INFO:teuthology.orchestra.run.smithi016.stdout:generic/566 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:52.113 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/566 2023-12-08T12:08:52.113 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/566 2023-12-08T12:08:52.113 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:52.116 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:52.136 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:52.137 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/567) 2023-12-08T12:08:52.311 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:52.316 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:52.316 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:52.426 INFO:teuthology.orchestra.run.smithi016.stdout:generic/567 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:52.430 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/567 2023-12-08T12:08:52.430 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/567 2023-12-08T12:08:52.430 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:52.432 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:52.440 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:52.441 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/568) 2023-12-08T12:08:52.615 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:52.619 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:52.619 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:52.789 INFO:teuthology.orchestra.run.smithi016.stdout:generic/568 0s 2023-12-08T12:08:52.819 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/568 2023-12-08T12:08:52.820 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:52.823 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:52.839 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:52.839 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/569) 2023-12-08T12:08:53.245 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:53.249 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:53.249 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:53.399 INFO:teuthology.orchestra.run.smithi016.stdout:generic/569 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:53.403 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/569 2023-12-08T12:08:53.403 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/569 2023-12-08T12:08:53.403 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:53.405 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:53.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:53.414 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/570) 2023-12-08T12:08:53.588 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:53.592 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:53.592 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:53.703 INFO:teuthology.orchestra.run.smithi016.stdout:generic/570 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:08:53.707 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/570 2023-12-08T12:08:53.707 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/570 2023-12-08T12:08:53.707 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:08:53.709 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:53.718 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:08:53.718 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/571) 2023-12-08T12:08:53.893 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:08:53.897 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:08:53.897 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:08:56.091 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:08:56.885 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:09:00.144 INFO:teuthology.orchestra.run.smithi016.stdout:generic/571 7s 2023-12-08T12:09:00.150 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/571 2023-12-08T12:09:00.150 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:00.152 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:00.162 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:00.162 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/572) 2023-12-08T12:09:00.631 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:00.635 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:00.635 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:00.744 INFO:teuthology.orchestra.run.smithi016.stdout:generic/572 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:00.748 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/572 2023-12-08T12:09:00.748 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/572 2023-12-08T12:09:00.748 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:00.750 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:00.759 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:00.759 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/573) 2023-12-08T12:09:00.932 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:00.936 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:00.936 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:01.046 INFO:teuthology.orchestra.run.smithi016.stdout:generic/573 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:01.048 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/573 2023-12-08T12:09:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/573 2023-12-08T12:09:01.049 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:01.051 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:01.059 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:01.059 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/574) 2023-12-08T12:09:01.234 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:01.238 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:01.238 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:01.348 INFO:teuthology.orchestra.run.smithi016.stdout:generic/574 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:01.352 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/574 2023-12-08T12:09:01.352 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/574 2023-12-08T12:09:01.352 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:01.354 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:01.363 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:01.363 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/575) 2023-12-08T12:09:01.538 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:01.542 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:01.542 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:01.652 INFO:teuthology.orchestra.run.smithi016.stdout:generic/575 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:01.656 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/575 2023-12-08T12:09:01.656 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/575 2023-12-08T12:09:01.656 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:01.658 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:01.666 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:01.667 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/576) 2023-12-08T12:09:01.843 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:01.847 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:01.847 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:01.983 INFO:teuthology.orchestra.run.smithi016.stdout:generic/576 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:01.987 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/576 2023-12-08T12:09:01.987 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/576 2023-12-08T12:09:01.987 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:01.989 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:01.998 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:01.998 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/577) 2023-12-08T12:09:02.172 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:02.176 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:02.176 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:02.286 INFO:teuthology.orchestra.run.smithi016.stdout:generic/577 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:02.290 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/577 2023-12-08T12:09:02.290 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/577 2023-12-08T12:09:02.290 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:02.293 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:02.301 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:02.302 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/578) 2023-12-08T12:09:02.477 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:02.481 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:02.632 INFO:teuthology.orchestra.run.smithi016.stdout:generic/578 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-12-08T12:09:02.636 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/578 2023-12-08T12:09:02.636 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/578 2023-12-08T12:09:02.636 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:02.638 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:02.646 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:02.647 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/579) 2023-12-08T12:09:02.827 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:02.831 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:02.831 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:02.951 INFO:teuthology.orchestra.run.smithi016.stdout:generic/579 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:02.954 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/579 2023-12-08T12:09:02.954 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/579 2023-12-08T12:09:02.954 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:02.957 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:02.965 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:02.965 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/580) 2023-12-08T12:09:03.140 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:03.144 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:03.144 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:03.286 INFO:teuthology.orchestra.run.smithi016.stdout:generic/580 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:03.289 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/580 2023-12-08T12:09:03.289 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/580 2023-12-08T12:09:03.289 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:03.291 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:03.300 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:03.300 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/581) 2023-12-08T12:09:03.475 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:03.480 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:03.480 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:04.292 INFO:teuthology.orchestra.run.smithi016.stdout:generic/581 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:04.296 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/581 2023-12-08T12:09:04.296 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/581 2023-12-08T12:09:04.296 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:04.298 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:04.307 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:04.307 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/582) 2023-12-08T12:09:04.484 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:04.488 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:04.488 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:04.626 INFO:teuthology.orchestra.run.smithi016.stdout:generic/582 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:04.629 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/582 2023-12-08T12:09:04.630 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/582 2023-12-08T12:09:04.630 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:04.632 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:04.640 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:04.640 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/583) 2023-12-08T12:09:04.816 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:04.820 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:04.820 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:04.957 INFO:teuthology.orchestra.run.smithi016.stdout:generic/583 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:04.961 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/583 2023-12-08T12:09:04.961 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/583 2023-12-08T12:09:04.961 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:04.963 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:04.971 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:04.971 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/584) 2023-12-08T12:09:05.146 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:05.150 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:05.150 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:05.301 INFO:teuthology.orchestra.run.smithi016.stdout:generic/584 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:05.305 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/584 2023-12-08T12:09:05.305 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/584 2023-12-08T12:09:05.305 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:05.307 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:05.315 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:05.316 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/585) 2023-12-08T12:09:05.490 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:05.494 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:05.494 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:05.622 INFO:teuthology.orchestra.run.smithi016.stdout:generic/585 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:05.623 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/585 2023-12-08T12:09:05.623 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/585 2023-12-08T12:09:05.623 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:05.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:05.634 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/586) 2023-12-08T12:09:05.809 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:05.813 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:05.814 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:26.438 INFO:teuthology.orchestra.run.smithi016.stdout:generic/586 21s 2023-12-08T12:09:26.444 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/586 2023-12-08T12:09:26.444 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:26.446 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:26.455 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:26.455 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/587) 2023-12-08T12:09:26.915 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:26.935 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:26.935 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:26.957 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:09:26.966 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:09:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:generic/587 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2023-12-08T12:09:27.324 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/587 2023-12-08T12:09:27.324 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/587 2023-12-08T12:09:27.324 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:27.326 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:27.338 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:27.338 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/588) 2023-12-08T12:09:27.517 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:27.521 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:27.521 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:27.675 INFO:teuthology.orchestra.run.smithi016.stdout:generic/588 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:27.678 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/588 2023-12-08T12:09:27.679 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/588 2023-12-08T12:09:27.679 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:27.681 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:27.689 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:27.689 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/589) 2023-12-08T12:09:27.863 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:27.867 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:27.868 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:27.982 INFO:teuthology.orchestra.run.smithi016.stdout:generic/589 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:27.985 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/589 2023-12-08T12:09:27.985 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/589 2023-12-08T12:09:27.985 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:27.987 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:27.996 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:27.996 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/590) 2023-12-08T12:09:28.172 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:28.176 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:28.176 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:28.292 INFO:teuthology.orchestra.run.smithi016.stdout:generic/590 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:28.295 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/590 2023-12-08T12:09:28.295 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/590 2023-12-08T12:09:28.295 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:28.298 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:28.306 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:28.306 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/591) 2023-12-08T12:09:28.482 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:28.487 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:28.487 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:28.828 INFO:teuthology.orchestra.run.smithi016.stdout:generic/591 0s 2023-12-08T12:09:28.833 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/591 2023-12-08T12:09:28.834 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:28.836 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:28.844 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:28.845 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/592) 2023-12-08T12:09:29.343 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:29.347 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:29.347 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:29.486 INFO:teuthology.orchestra.run.smithi016.stdout:generic/592 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:29.490 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/592 2023-12-08T12:09:29.490 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/592 2023-12-08T12:09:29.490 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:29.492 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:29.502 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:29.502 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/593) 2023-12-08T12:09:29.683 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:29.687 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:29.687 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:29.833 INFO:teuthology.orchestra.run.smithi016.stdout:generic/593 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:29.837 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/593 2023-12-08T12:09:29.837 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/593 2023-12-08T12:09:29.837 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:29.839 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:29.848 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:29.848 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/594) 2023-12-08T12:09:30.024 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:30.028 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:30.028 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:30.141 INFO:teuthology.orchestra.run.smithi016.stdout:generic/594 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:30.144 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/594 2023-12-08T12:09:30.144 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/594 2023-12-08T12:09:30.144 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:30.146 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:30.155 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:30.155 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/595) 2023-12-08T12:09:30.328 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:30.332 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:30.333 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:30.472 INFO:teuthology.orchestra.run.smithi016.stdout:generic/595 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:30.476 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/595 2023-12-08T12:09:30.476 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/595 2023-12-08T12:09:30.476 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:30.478 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:30.487 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:30.487 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/596) 2023-12-08T12:09:30.662 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:30.666 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:30.666 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:30.869 INFO:teuthology.orchestra.run.smithi016.stdout:generic/596 [not run] file system doesn't support chattr +S 2023-12-08T12:09:30.873 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/596 2023-12-08T12:09:30.873 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/596 2023-12-08T12:09:30.873 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:30.875 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:30.884 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:30.884 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/597) 2023-12-08T12:09:31.058 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:31.062 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:31.063 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:31.702 INFO:teuthology.orchestra.run.smithi016.stdout:generic/597 - output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/597.out.bad) 2023-12-08T12:09:31.704 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/597.out 2023-12-08 08:39:53.792917465 +0000 2023-12-08T12:09:31.704 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/597.out.bad 2023-12-08 12:09:31.699704133 +0000 2023-12-08T12:09:31.704 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -4,11 +4,11 @@ 2023-12-08T12:09:31.705 INFO:teuthology.orchestra.run.smithi016.stdout: fs.protected_symlinks = 0 2023-12-08T12:09:31.705 INFO:teuthology.orchestra.run.smithi016.stdout: successfully followed symlink 2023-12-08T12:09:31.705 INFO:teuthology.orchestra.run.smithi016.stdout: fs.protected_symlinks = 1 2023-12-08T12:09:31.705 INFO:teuthology.orchestra.run.smithi016.stdout: -Permission denied 2023-12-08T12:09:31.705 INFO:teuthology.orchestra.run.smithi016.stdout: +successfully followed symlink 2023-12-08T12:09:31.705 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:31.705 INFO:teuthology.orchestra.run.smithi016.stdout: == Test hardlink create protection when 2023-12-08T12:09:31.705 INFO:teuthology.orchestra.run.smithi016.stdout: == process != target owner and process cannot read target 2023-12-08T12:09:31.705 INFO:teuthology.orchestra.run.smithi016.stdout: fs.protected_hardlinks = 0 2023-12-08T12:09:31.705 INFO:teuthology.orchestra.run.smithi016.stdout: -successfully created hardlink 2023-12-08T12:09:31.705 INFO:teuthology.orchestra.run.smithi016.stdout: +Permission denied 2023-12-08T12:09:31.705 INFO:teuthology.orchestra.run.smithi016.stdout: fs.protected_hardlinks = 1 2023-12-08T12:09:31.706 INFO:teuthology.orchestra.run.smithi016.stdout: -ln: failed to create hard link 'TEST_DIR/597/sticky_dir/hardlink' => 'TEST_DIR/597/target': Operation not permitted 2023-12-08T12:09:31.706 INFO:teuthology.orchestra.run.smithi016.stdout: +Permission denied 2023-12-08T12:09:31.707 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/597 2023-12-08T12:09:31.707 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/597 2023-12-08T12:09:31.707 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T12:09:31.709 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:31.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T12:09:31.718 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T12:09:31.718 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T12:09:31.719 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/598) 2023-12-08T12:09:32.181 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:32.185 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:32.185 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:32.652 INFO:teuthology.orchestra.run.smithi016.stdout:generic/598 - output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/598.out.bad) 2023-12-08T12:09:32.654 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/598.out 2023-12-08 08:39:53.792917465 +0000 2023-12-08T12:09:32.654 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/598.out.bad 2023-12-08 12:09:32.648690319 +0000 2023-12-08T12:09:32.654 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -5,13 +5,10 @@ 2023-12-08T12:09:32.654 INFO:teuthology.orchestra.run.smithi016.stdout: = only group writable dir 2023-12-08T12:09:32.654 INFO:teuthology.orchestra.run.smithi016.stdout: fs.protected_regular = 1 2023-12-08T12:09:32.654 INFO:teuthology.orchestra.run.smithi016.stdout: = group & world writable dir 2023-12-08T12:09:32.654 INFO:teuthology.orchestra.run.smithi016.stdout: -Permission denied 2023-12-08T12:09:32.654 INFO:teuthology.orchestra.run.smithi016.stdout: = only group writable dir 2023-12-08T12:09:32.654 INFO:teuthology.orchestra.run.smithi016.stdout: fs.protected_regular = 2 2023-12-08T12:09:32.654 INFO:teuthology.orchestra.run.smithi016.stdout: = group & world writable dir 2023-12-08T12:09:32.654 INFO:teuthology.orchestra.run.smithi016.stdout: -Permission denied 2023-12-08T12:09:32.655 INFO:teuthology.orchestra.run.smithi016.stdout: = only group writable dir 2023-12-08T12:09:32.655 INFO:teuthology.orchestra.run.smithi016.stdout: -Permission denied 2023-12-08T12:09:32.655 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:32.655 INFO:teuthology.orchestra.run.smithi016.stdout: == Test fifo open when owned by another and fifo owner != dir owner 2023-12-08T12:09:32.657 INFO:teuthology.orchestra.run.smithi016.stdout: fs.protected_fifos = 0 2023-12-08T12:09:32.657 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -19,10 +16,7 @@ 2023-12-08T12:09:32.657 INFO:teuthology.orchestra.run.smithi016.stdout: = only group writable dir 2023-12-08T12:09:32.657 INFO:teuthology.orchestra.run.smithi016.stdout: fs.protected_fifos = 1 2023-12-08T12:09:32.657 INFO:teuthology.orchestra.run.smithi016.stdout: = group & world writable dir 2023-12-08T12:09:32.658 INFO:teuthology.orchestra.run.smithi016.stdout: -Permission denied 2023-12-08T12:09:32.658 INFO:teuthology.orchestra.run.smithi016.stdout: = only group writable dir 2023-12-08T12:09:32.658 INFO:teuthology.orchestra.run.smithi016.stdout: fs.protected_fifos = 2 2023-12-08T12:09:32.658 INFO:teuthology.orchestra.run.smithi016.stdout: = group & world writable dir 2023-12-08T12:09:32.658 INFO:teuthology.orchestra.run.smithi016.stdout: -Permission denied 2023-12-08T12:09:32.658 INFO:teuthology.orchestra.run.smithi016.stdout: = only group writable dir 2023-12-08T12:09:32.658 INFO:teuthology.orchestra.run.smithi016.stdout: -Permission denied 2023-12-08T12:09:32.659 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/598 2023-12-08T12:09:32.659 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/598 2023-12-08T12:09:32.659 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T12:09:32.659 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:32.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T12:09:32.668 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T12:09:32.668 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T12:09:32.668 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/599) 2023-12-08T12:09:33.130 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:33.134 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:33.255 INFO:teuthology.orchestra.run.smithi016.stdout:generic/599 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:33.258 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/599 2023-12-08T12:09:33.258 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/599 2023-12-08T12:09:33.258 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:33.261 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:33.270 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:33.270 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/600) 2023-12-08T12:09:33.444 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:33.448 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:33.559 INFO:teuthology.orchestra.run.smithi016.stdout:generic/600 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:33.562 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/600 2023-12-08T12:09:33.563 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/600 2023-12-08T12:09:33.563 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:33.588 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:33.588 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/601) 2023-12-08T12:09:33.762 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:33.766 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:33.895 INFO:teuthology.orchestra.run.smithi016.stdout:generic/601 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:33.899 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/601 2023-12-08T12:09:33.899 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/601 2023-12-08T12:09:33.899 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:33.901 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:33.910 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:33.910 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/602) 2023-12-08T12:09:34.086 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:34.093 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:34.227 INFO:teuthology.orchestra.run.smithi016.stdout:generic/602 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:34.231 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/602 2023-12-08T12:09:34.231 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/602 2023-12-08T12:09:34.231 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:34.241 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:34.242 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/603) 2023-12-08T12:09:34.414 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:34.418 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:34.550 INFO:teuthology.orchestra.run.smithi016.stdout:generic/603 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:34.553 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/603 2023-12-08T12:09:34.554 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/603 2023-12-08T12:09:34.554 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:34.556 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:34.576 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:34.576 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/604) 2023-12-08T12:09:34.751 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:34.755 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:34.755 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:34.866 INFO:teuthology.orchestra.run.smithi016.stdout:generic/604 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:34.870 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/604 2023-12-08T12:09:34.870 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/604 2023-12-08T12:09:34.870 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:34.872 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:34.880 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:34.881 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/605) 2023-12-08T12:09:35.055 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:35.059 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:35.059 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:35.171 INFO:teuthology.orchestra.run.smithi016.stdout:generic/605 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/605 2023-12-08T12:09:35.174 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/605 2023-12-08T12:09:35.175 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:35.177 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:35.185 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:35.185 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/606) 2023-12-08T12:09:35.361 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:35.365 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:35.365 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:35.475 INFO:teuthology.orchestra.run.smithi016.stdout:generic/606 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:35.478 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/606 2023-12-08T12:09:35.478 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/606 2023-12-08T12:09:35.478 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:35.481 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:35.489 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:35.489 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/607) 2023-12-08T12:09:35.664 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:35.668 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:35.668 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:35.779 INFO:teuthology.orchestra.run.smithi016.stdout:generic/607 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:35.782 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/607 2023-12-08T12:09:35.782 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/607 2023-12-08T12:09:35.782 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:35.785 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:35.793 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:35.793 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/608) 2023-12-08T12:09:35.969 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:35.973 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:35.973 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:36.082 INFO:teuthology.orchestra.run.smithi016.stdout:generic/608 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:36.086 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/608 2023-12-08T12:09:36.086 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/608 2023-12-08T12:09:36.086 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:36.088 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:36.118 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:36.118 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/609) 2023-12-08T12:09:36.294 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:36.297 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:36.298 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:36.845 INFO:teuthology.orchestra.run.smithi016.stdout:generic/609 0s 2023-12-08T12:09:36.856 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/609 2023-12-08T12:09:36.856 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:36.858 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:36.867 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:36.867 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/610) 2023-12-08T12:09:37.319 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:37.324 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:37.324 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:37.433 INFO:teuthology.orchestra.run.smithi016.stdout:generic/610 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:37.436 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/610 2023-12-08T12:09:37.436 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/610 2023-12-08T12:09:37.436 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:37.439 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:37.447 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:37.448 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/611) 2023-12-08T12:09:37.621 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:37.625 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:37.625 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:37.747 INFO:teuthology.orchestra.run.smithi016.stdout:generic/611 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:37.751 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/611 2023-12-08T12:09:37.751 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/611 2023-12-08T12:09:37.751 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:37.753 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:37.761 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:37.762 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/612) 2023-12-08T12:09:37.937 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:37.941 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:37.941 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:38.113 INFO:teuthology.orchestra.run.smithi016.stdout:generic/612 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T12:09:38.116 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/612 2023-12-08T12:09:38.117 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/612 2023-12-08T12:09:38.117 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:38.119 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:38.127 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:38.127 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/613) 2023-12-08T12:09:38.582 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:38.586 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:38.586 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:38.738 INFO:teuthology.orchestra.run.smithi016.stdout:generic/613 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:38.742 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/613 2023-12-08T12:09:38.742 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/613 2023-12-08T12:09:38.742 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:38.744 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:38.762 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:38.763 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/614) 2023-12-08T12:09:38.939 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:38.943 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:38.943 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:39.052 INFO:teuthology.orchestra.run.smithi016.stdout:generic/614 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:39.056 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/614 2023-12-08T12:09:39.056 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/614 2023-12-08T12:09:39.056 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:39.058 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:39.067 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:39.067 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/615) 2023-12-08T12:09:39.242 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:39.246 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:39.246 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:39.357 INFO:teuthology.orchestra.run.smithi016.stdout:generic/615 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/615 2023-12-08T12:09:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/615 2023-12-08T12:09:39.361 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:39.363 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:39.371 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:39.372 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/616) 2023-12-08T12:09:39.548 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:39.552 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:39.552 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:39.709 INFO:teuthology.orchestra.run.smithi016.stdout:generic/616 [not run] kernel does not support IO_URING 2023-12-08T12:09:39.713 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/616 2023-12-08T12:09:39.713 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/616 2023-12-08T12:09:39.713 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:39.715 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:39.742 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:39.742 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/617) 2023-12-08T12:09:39.917 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:39.921 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:39.921 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:40.087 INFO:teuthology.orchestra.run.smithi016.stdout:generic/617 [not run] kernel does not support IO_URING 2023-12-08T12:09:40.090 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/617 2023-12-08T12:09:40.090 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/617 2023-12-08T12:09:40.090 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:40.093 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:40.101 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:40.101 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/618) 2023-12-08T12:09:40.277 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:40.281 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:40.281 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:40.392 INFO:teuthology.orchestra.run.smithi016.stdout:generic/618 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:40.396 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/618 2023-12-08T12:09:40.396 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/618 2023-12-08T12:09:40.396 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:40.399 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:40.407 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:40.408 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/619) 2023-12-08T12:09:40.584 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:40.588 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:40.588 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:40.698 INFO:teuthology.orchestra.run.smithi016.stdout:generic/619 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:40.701 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/619 2023-12-08T12:09:40.702 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/619 2023-12-08T12:09:40.702 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:40.704 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:40.712 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:40.712 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/620) 2023-12-08T12:09:40.888 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:40.892 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:40.893 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:41.034 INFO:teuthology.orchestra.run.smithi016.stdout:generic/620 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:41.037 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/620 2023-12-08T12:09:41.037 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/620 2023-12-08T12:09:41.037 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:41.040 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:41.048 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:41.048 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/621) 2023-12-08T12:09:41.223 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:41.227 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:41.227 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:41.374 INFO:teuthology.orchestra.run.smithi016.stdout:generic/621 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:41.378 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/621 2023-12-08T12:09:41.378 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/621 2023-12-08T12:09:41.378 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:41.380 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:41.388 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:41.389 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/622) 2023-12-08T12:09:41.564 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:41.568 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:41.568 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:41.680 INFO:teuthology.orchestra.run.smithi016.stdout:generic/622 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:41.683 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/622 2023-12-08T12:09:41.683 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/622 2023-12-08T12:09:41.683 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:41.686 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:41.694 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:41.695 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/623) 2023-12-08T12:09:41.871 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:41.875 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:41.875 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:41.985 INFO:teuthology.orchestra.run.smithi016.stdout:generic/623 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:41.989 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/623 2023-12-08T12:09:41.989 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/623 2023-12-08T12:09:41.989 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:41.991 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:42.000 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:42.000 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/624) 2023-12-08T12:09:42.175 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:42.180 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:42.180 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:42.290 INFO:teuthology.orchestra.run.smithi016.stdout:generic/624 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:42.294 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/624 2023-12-08T12:09:42.294 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/624 2023-12-08T12:09:42.294 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:42.296 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:42.304 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:42.305 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/625) 2023-12-08T12:09:42.479 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:42.484 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:42.484 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:42.594 INFO:teuthology.orchestra.run.smithi016.stdout:generic/625 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:42.597 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/625 2023-12-08T12:09:42.598 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/625 2023-12-08T12:09:42.598 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:42.600 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:42.609 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:42.609 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/626) 2023-12-08T12:09:42.785 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:42.789 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:42.789 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:42.915 INFO:teuthology.orchestra.run.smithi016.stdout:generic/626 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:42.918 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/626 2023-12-08T12:09:42.918 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/626 2023-12-08T12:09:42.919 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:42.921 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:42.929 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:42.929 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/627) 2023-12-08T12:09:43.104 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:43.108 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:43.108 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:43.267 INFO:teuthology.orchestra.run.smithi016.stdout:generic/627 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:43.270 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/627 2023-12-08T12:09:43.270 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/627 2023-12-08T12:09:43.271 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:43.273 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:43.299 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:43.299 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/628) 2023-12-08T12:09:43.474 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:43.478 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:43.478 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:43.638 INFO:teuthology.orchestra.run.smithi016.stdout:generic/628 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:43.642 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/628 2023-12-08T12:09:43.642 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/628 2023-12-08T12:09:43.642 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:43.644 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:43.653 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:43.653 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/629) 2023-12-08T12:09:43.829 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:43.833 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:43.833 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:43.957 INFO:teuthology.orchestra.run.smithi016.stdout:generic/629 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:43.961 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/629 2023-12-08T12:09:43.961 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/629 2023-12-08T12:09:43.961 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:43.963 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:43.971 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:43.972 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/630) 2023-12-08T12:09:44.147 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:44.151 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:44.151 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:44.260 INFO:teuthology.orchestra.run.smithi016.stdout:generic/630 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:44.263 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/630 2023-12-08T12:09:44.263 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/630 2023-12-08T12:09:44.263 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:44.266 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:44.274 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:44.274 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/631) 2023-12-08T12:09:44.448 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:44.452 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:44.452 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:44.562 INFO:teuthology.orchestra.run.smithi016.stdout:generic/631 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:44.565 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/631 2023-12-08T12:09:44.565 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/631 2023-12-08T12:09:44.565 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:44.567 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:44.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:44.576 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/632) 2023-12-08T12:09:44.751 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:44.755 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:44.755 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:47.475 INFO:teuthology.orchestra.run.smithi016.stdout:generic/632 3s 2023-12-08T12:09:47.481 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/632 2023-12-08T12:09:47.481 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:47.483 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:47.491 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:47.491 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/633) 2023-12-08T12:09:47.666 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:47.670 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:47.670 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:47.974 INFO:teuthology.orchestra.run.smithi016.stdout:generic/633 [failed, exit status 1]- output mismatch (see /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/633.out.bad) 2023-12-08T12:09:47.977 INFO:teuthology.orchestra.run.smithi016.stdout: --- tests/generic/633.out 2023-12-08 08:39:53.795917422 +0000 2023-12-08T12:09:47.977 INFO:teuthology.orchestra.run.smithi016.stdout: +++ /tmp/tmp.MkHH5XcxMMxfstests-dev/results//generic/633.out.bad 2023-12-08 12:09:47.971467270 +0000 2023-12-08T12:09:47.977 INFO:teuthology.orchestra.run.smithi016.stdout: @@ -1,2 +1,5 @@ 2023-12-08T12:09:47.977 INFO:teuthology.orchestra.run.smithi016.stdout: QA output created by 633 2023-12-08T12:09:47.977 INFO:teuthology.orchestra.run.smithi016.stdout: Silence is golden 2023-12-08T12:09:47.977 INFO:teuthology.orchestra.run.smithi016.stdout: +vfstest.c: 1164: protected_symlinks - Success - failure: openat 2023-12-08T12:09:47.977 INFO:teuthology.orchestra.run.smithi016.stdout: +vfstest.c: 1171: protected_symlinks - Success - failure: wait_for_pid 2023-12-08T12:09:47.977 INFO:teuthology.orchestra.run.smithi016.stdout: +vfstest.c: 2418: run_test - Success - failure: following protected symlinks on regular mounts 2023-12-08T12:09:47.980 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/633 2023-12-08T12:09:47.980 INFO:teuthology.orchestra.run.smithi016.stdout:Failures: generic/633 2023-12-08T12:09:47.980 INFO:teuthology.orchestra.run.smithi016.stdout:Failed 1 of 1 tests 2023-12-08T12:09:47.984 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:47.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T12:09:47.997 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-12-08T12:09:47.997 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-12-08T12:09:47.998 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/634) 2023-12-08T12:09:48.422 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:48.426 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:48.426 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:48.533 INFO:teuthology.orchestra.run.smithi016.stdout:generic/634 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:48.537 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/634 2023-12-08T12:09:48.537 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/634 2023-12-08T12:09:48.537 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:48.539 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:48.547 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:48.548 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/635) 2023-12-08T12:09:48.720 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:48.725 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:48.725 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:48.837 INFO:teuthology.orchestra.run.smithi016.stdout:generic/635 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:48.840 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/635 2023-12-08T12:09:48.840 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/635 2023-12-08T12:09:48.840 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:48.843 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:48.851 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:48.851 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/636) 2023-12-08T12:09:49.024 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:49.028 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:49.028 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:49.138 INFO:teuthology.orchestra.run.smithi016.stdout:generic/636 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:49.141 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/636 2023-12-08T12:09:49.141 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/636 2023-12-08T12:09:49.142 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:49.144 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:49.152 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:49.152 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/637) 2023-12-08T12:09:49.328 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:49.332 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:49.332 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:50.474 INFO:teuthology.orchestra.run.smithi016.stdout:generic/637 1s 2023-12-08T12:09:50.479 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/637 2023-12-08T12:09:50.480 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:50.482 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:50.494 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:50.494 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/638) 2023-12-08T12:09:50.994 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:50.998 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:50.998 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:51.155 INFO:teuthology.orchestra.run.smithi016.stdout:generic/638 0s 2023-12-08T12:09:51.160 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/638 2023-12-08T12:09:51.160 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:51.162 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:51.171 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:51.171 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/639) 2023-12-08T12:09:51.345 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:51.349 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:51.349 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:54.971 INFO:teuthology.orchestra.run.smithi016.stdout:generic/639 3s 2023-12-08T12:09:54.978 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/639 2023-12-08T12:09:54.978 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:54.981 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:54.991 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:54.991 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/640) 2023-12-08T12:09:55.176 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:55.180 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:55.180 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:55.300 INFO:teuthology.orchestra.run.smithi016.stdout:generic/640 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:55.303 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/640 2023-12-08T12:09:55.303 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/640 2023-12-08T12:09:55.303 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:55.314 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:55.314 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/641) 2023-12-08T12:09:55.489 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:55.493 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:55.493 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:55.602 INFO:teuthology.orchestra.run.smithi016.stdout:generic/641 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:55.606 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/641 2023-12-08T12:09:55.606 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/641 2023-12-08T12:09:55.606 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:55.608 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:55.616 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:55.616 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/642) 2023-12-08T12:09:55.791 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:55.795 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:55.796 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:55.909 INFO:teuthology.orchestra.run.smithi016.stdout:generic/642 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:55.913 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/642 2023-12-08T12:09:55.913 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/642 2023-12-08T12:09:55.913 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:55.915 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:55.923 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:55.924 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/643) 2023-12-08T12:09:56.099 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:56.103 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:56.103 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:56.213 INFO:teuthology.orchestra.run.smithi016.stdout:generic/643 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:56.217 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/643 2023-12-08T12:09:56.217 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/643 2023-12-08T12:09:56.217 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:56.219 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:56.228 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:56.228 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/644) 2023-12-08T12:09:56.403 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:56.407 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:56.407 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:56.519 INFO:teuthology.orchestra.run.smithi016.stdout:generic/644 [not run] vfstest not support by ceph-fuse 2023-12-08T12:09:56.522 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/644 2023-12-08T12:09:56.522 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/644 2023-12-08T12:09:56.522 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:56.525 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:56.533 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:56.533 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/645) 2023-12-08T12:09:56.710 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:56.714 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:56.714 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:56.830 INFO:teuthology.orchestra.run.smithi016.stdout:generic/645 [not run] vfstest not support by ceph-fuse 2023-12-08T12:09:56.833 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/645 2023-12-08T12:09:56.834 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/645 2023-12-08T12:09:56.834 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:56.836 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:56.844 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:56.845 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/646) 2023-12-08T12:09:57.022 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:57.026 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:57.026 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:57.036 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:09:57.038 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:09:57.136 INFO:teuthology.orchestra.run.smithi016.stdout:generic/646 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:57.140 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/646 2023-12-08T12:09:57.140 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/646 2023-12-08T12:09:57.140 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:57.142 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:57.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:57.151 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/647) 2023-12-08T12:09:57.371 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:57.375 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:57.376 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:57.720 INFO:teuthology.orchestra.run.smithi016.stdout:generic/647 0s 2023-12-08T12:09:57.725 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/647 2023-12-08T12:09:57.725 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:57.728 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:57.736 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:57.736 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/648) 2023-12-08T12:09:58.213 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:58.217 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:58.217 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:58.385 INFO:teuthology.orchestra.run.smithi016.stdout:generic/648 [not run] this test requires a valid $SCRATCH_DEV 2023-12-08T12:09:58.388 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/648 2023-12-08T12:09:58.389 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/648 2023-12-08T12:09:58.389 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:58.391 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:58.400 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:58.400 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/649) 2023-12-08T12:09:58.575 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:58.579 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:58.579 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:58.757 INFO:teuthology.orchestra.run.smithi016.stdout:generic/649 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-12-08T12:09:58.760 INFO:teuthology.orchestra.run.smithi016.stdout:Ran: generic/649 2023-12-08T12:09:58.760 INFO:teuthology.orchestra.run.smithi016.stdout:Not run: generic/649 2023-12-08T12:09:58.760 INFO:teuthology.orchestra.run.smithi016.stdout:Passed all 1 tests 2023-12-08T12:09:58.763 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:09:58.771 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-12-08T12:09:58.771 DEBUG:teuthology.orchestra.run.smithi016:> (cd /tmp/tmp.MkHH5XcxMMxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/650) 2023-12-08T12:09:59.250 INFO:teuthology.orchestra.run.smithi016.stdout:FSTYP -- ceph-fuse 2023-12-08T12:09:59.256 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM -- Linux/x86_64 smithi016 5.14.0-390.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Nov 24 10:44:56 UTC 2023 2023-12-08T12:09:59.257 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-12-08T12:10:27.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:10:27.126 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:10:57.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:10:57.285 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:11:27.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:11:27.463 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:11:57.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:11:57.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:11:57.712 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T12:11:57.710+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102368) UID: 0 2023-12-08T12:11:57.712 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T12:11:57.710+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102368) UID: 0 2023-12-08T12:11:57.712 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T12:11:57.710+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102368) UID: 0 2023-12-08T12:11:57.713 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T12:11:57.710+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102368) UID: 0 2023-12-08T12:14:59.708 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:14:59.717 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:14:59.964 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T12:14:59.962+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102397) UID: 0 2023-12-08T12:14:59.965 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T12:14:59.962+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102397) UID: 0 2023-12-08T12:14:59.965 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T12:14:59.962+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102397) UID: 0 2023-12-08T12:14:59.966 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T12:14:59.962+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102397) UID: 0 2023-12-08T12:18:06.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:18:06.183 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:18:06.213 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T12:18:06.211+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102422) UID: 0 2023-12-08T12:18:06.214 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T12:18:06.211+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102422) UID: 0 2023-12-08T12:18:06.214 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T12:18:06.211+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102422) UID: 0 2023-12-08T12:18:06.215 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T12:18:06.211+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102422) UID: 0 2023-12-08T12:18:06.433 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T12:18:06.432+0000 7ff25129f640 -1 received signal: Hangup from (PID: 563935) UID: 0 2023-12-08T12:18:06.436 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T12:18:06.434+0000 7fdb0d8f4640 -1 received signal: Hangup from (PID: 563935) UID: 0 2023-12-08T12:18:06.438 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T12:18:06.431+0000 7f6d0f17a640 -1 received signal: Hangup from (PID: 563935) UID: 0 2023-12-08T12:18:06.447 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T12:18:06.440+0000 7eff8ec05640 -1 Fail to open '/proc/563935/cmdline' error = (2) No such file or directory 2023-12-08T12:18:06.447 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T12:18:06.440+0000 7eff8ec05640 -1 received signal: Hangup from (PID: 563935) UID: 0 2023-12-08T12:28:01.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:28:01.842 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:28:32.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:28:32.026 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:29:02.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:29:02.202 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:29:32.368 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:29:32.376 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:30:02.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:30:02.907 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:30:33.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:30:33.166 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:30:33.203 INFO:tasks.ceph.mds.b.smithi112.stderr:2023-12-08T12:30:33.201+0000 7f0cd7b54640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 102555) UID: 0 2023-12-08T12:30:33.203 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-12-08T12:30:33.201+0000 7f52ecb1a640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 102555) UID: 0 2023-12-08T12:33:19.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:33:19.328 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:33:19.529 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T12:33:19.525+0000 7fdb0d8f4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 566364) UID: 0 2023-12-08T12:33:19.529 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T12:33:19.525+0000 7ff25129f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 566364) UID: 0 2023-12-08T12:33:19.530 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T12:33:19.525+0000 7eff8ec05640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 566364) UID: 0 2023-12-08T12:33:19.539 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T12:33:19.525+0000 7f6d0f17a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 566364) UID: 0 2023-12-08T12:53:57.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:53:57.562 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:53:57.756 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T12:53:57.754+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102638) UID: 0 2023-12-08T12:53:57.757 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T12:53:57.754+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102638) UID: 0 2023-12-08T12:53:57.757 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T12:53:57.754+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102638) UID: 0 2023-12-08T12:53:57.757 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T12:53:57.754+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 102638) UID: 0 2023-12-08T12:57:48.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:57:48.230 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:58:18.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:58:18.382 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:58:48.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:58:48.560 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:59:18.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:59:18.820 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:59:48.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T12:59:49.000 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:00:19.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:00:19.273 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:00:49.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:00:49.497 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:01:19.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:01:19.687 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:01:49.855 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:01:49.867 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:02:20.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:02:20.342 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:02:50.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:02:50.529 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:03:20.872 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:03:20.880 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:03:51.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:03:51.067 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:04:21.291 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:04:21.296 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:04:51.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:04:51.516 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:05:21.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:05:21.844 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:05:52.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:05:52.089 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:06:22.325 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:06:22.329 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:06:52.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:06:52.494 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:07:22.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:07:22.708 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:07:22.741 INFO:tasks.ceph.mds.b.smithi112.stderr:2023-12-08T13:07:22.740+0000 7f0cd7b54640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 103033) UID: 0 2023-12-08T13:07:22.742 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-12-08T13:07:22.740+0000 7f52ecb1a640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 103033) UID: 0 2023-12-08T13:10:14.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:10:14.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:10:44.881 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:10:44.890 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:11:15.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:11:15.177 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:11:45.361 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:11:45.367 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:12:15.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:12:15.569 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:12:45.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:12:45.772 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:13:15.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:13:15.984 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:13:46.129 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:13:46.139 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:14:16.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:14:16.454 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:14:46.629 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:14:46.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:15:16.806 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:15:16.811 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:15:47.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:15:47.097 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:16:17.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:16:17.297 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:16:47.504 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:16:47.515 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:17:17.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:17:17.716 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:17:47.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:17:47.927 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:18:18.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:18:18.194 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:18:48.341 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:18:48.346 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:19:18.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:19:18.488 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:19:48.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:19:48.701 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:20:18.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:20:18.940 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:20:49.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:20:49.139 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:21:19.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:21:19.283 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:21:49.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:21:49.415 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:22:19.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:22:19.610 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:22:49.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:22:49.771 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:23:19.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:23:19.971 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:23:50.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:23:50.154 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:24:20.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:24:20.420 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:24:50.599 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:24:50.606 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:25:20.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:25:20.829 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:25:51.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:25:51.031 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:26:21.255 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:26:21.260 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:26:51.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:26:51.735 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:27:21.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:27:21.932 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:27:52.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:27:52.336 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:28:22.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:28:22.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:28:52.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:28:52.643 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:29:23.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:29:23.111 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:29:53.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:29:53.266 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:30:23.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:30:23.553 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:30:53.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:30:53.703 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:31:23.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:31:23.847 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:31:24.077 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T13:31:24.073+0000 7eff8ec05640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 576072) UID: 0 2023-12-08T13:31:24.080 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T13:31:24.079+0000 7f6d0f17a640 -1 received signal: Hangup from (PID: 576072) UID: 0 2023-12-08T13:31:24.099 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T13:31:24.079+0000 7ff25129f640 -1 received signal: Hangup from (PID: 576072) UID: 0 2023-12-08T13:31:24.120 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T13:31:24.087+0000 7fdb0d8f4640 -1 Fail to open '/proc/576072/cmdline' error = (2) No such file or directory 2023-12-08T13:31:24.120 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T13:31:24.087+0000 7fdb0d8f4640 -1 received signal: Hangup from (PID: 576072) UID: 0 2023-12-08T13:42:33.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:42:33.039 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:43:03.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:43:03.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:43:33.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:43:33.650 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:44:03.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:44:03.888 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:44:34.164 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:44:34.168 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:45:04.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:45:04.324 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:45:34.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:45:34.634 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:45:34.674 INFO:tasks.ceph.mds.b.smithi112.stderr:2023-12-08T13:45:34.674+0000 7f0cd7b54640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 103999) UID: 0 2023-12-08T13:45:34.674 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-12-08T13:45:34.674+0000 7f52ecb1a640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 103999) UID: 0 2023-12-08T13:48:26.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:48:26.666 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:48:56.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:48:56.831 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:48:56.862 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T13:48:56.862+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 104049) UID: 0 2023-12-08T13:48:56.862 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T13:48:56.862+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 104049) UID: 0 2023-12-08T13:48:56.862 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T13:48:56.862+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 104049) UID: 0 2023-12-08T13:48:56.862 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T13:48:56.862+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 104049) UID: 0 2023-12-08T13:52:00.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:52:00.732 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:52:00.934 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T13:52:00.934+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 104076) UID: 0 2023-12-08T13:52:00.934 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T13:52:00.934+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 104076) UID: 0 2023-12-08T13:52:00.935 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T13:52:00.934+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 104076) UID: 0 2023-12-08T13:52:00.935 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T13:52:00.934+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 104076) UID: 0 2023-12-08T13:55:17.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:55:17.680 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:55:48.010 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:55:48.018 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:56:18.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:56:18.245 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:56:48.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:56:48.409 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:57:18.559 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:57:18.569 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:57:48.729 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:57:48.733 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:58:18.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:58:18.905 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:58:49.075 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:58:49.079 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:59:19.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:59:19.342 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:59:49.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T13:59:49.797 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:00:20.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:00:20.123 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:00:50.341 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:00:50.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:01:20.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:01:20.577 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:01:20.757 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T14:01:20.756+0000 7fdb0d8f4640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 580787) UID: 0 2023-12-08T14:01:20.757 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T14:01:20.755+0000 7f6d0f17a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 580787) UID: 0 2023-12-08T14:01:20.759 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T14:01:20.756+0000 7ff25129f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 580787) UID: 0 2023-12-08T14:01:20.762 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T14:01:20.760+0000 7eff8ec05640 -1 Fail to open '/proc/580787/cmdline' error = (2) No such file or directory 2023-12-08T14:01:20.762 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T14:01:20.760+0000 7eff8ec05640 -1 received signal: Hangup from (PID: 580787) UID: 0 2023-12-08T14:12:00.265 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:12:00.271 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:12:00.431 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T14:12:00.429+0000 7fb32d740640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 104371) UID: 0 2023-12-08T14:12:00.431 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T14:12:00.429+0000 7f0ae298c640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 104371) UID: 0 2023-12-08T14:12:00.431 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T14:12:00.429+0000 7f7f9cd1e640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 104371) UID: 0 2023-12-08T14:12:00.432 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T14:12:00.429+0000 7f147a3f3640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 104371) UID: 0 2023-12-08T14:12:00.751 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T14:12:00.748+0000 7f6d0f17a640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 582419) UID: 0 2023-12-08T14:12:00.754 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T14:12:00.750+0000 7eff8ec05640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 582419) UID: 0 2023-12-08T14:12:00.754 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T14:12:00.753+0000 7fdb0d8f4640 -1 received signal: Hangup from (PID: 582419) UID: 0 2023-12-08T14:12:00.755 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T14:12:00.748+0000 7ff25129f640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 582419) UID: 0 2023-12-08T14:12:20.681 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T14:12:20.679+0000 7fdb03f05640 -1 bdev(0x56286b390e00 /var/lib/ceph/osd/ceph-3/block) _aio_thread got r=-5 ((5) Input/output error) 2023-12-08T14:12:20.706 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T14:12:20.700+0000 7fdafcedd640 -1 bdev(0x56286b391c00 /var/lib/ceph/osd/ceph-3/block) _sync_write pwritev error: (5) Input/output error 2023-12-08T14:12:20.711 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T14:12:20.704+0000 7fdb036f3640 -1 bdev(0x56286b391c00 /var/lib/ceph/osd/ceph-3/block) _aio_thread got r=-5 ((5) Input/output error) 2023-12-08T14:12:20.729 INFO:tasks.ceph.osd.3.smithi016.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-46-ga2cd52c0/rpm/el9/BUILD/ceph-19.0.0-46-ga2cd52c0/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7fdb036f3640 time 2023-12-08T14:12:20.706330+0000 2023-12-08T14:12:20.729 INFO:tasks.ceph.osd.3.smithi016.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-46-ga2cd52c0/rpm/el9/BUILD/ceph-19.0.0-46-ga2cd52c0/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2023-12-08T14:12:20.731 INFO:tasks.ceph.osd.3.smithi016.stderr: ceph version 19.0.0-46-ga2cd52c0 (a2cd52c00149b5caff897ea7f79d0529b62620da) reef (dev) 2023-12-08T14:12:20.731 INFO:tasks.ceph.osd.3.smithi016.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x562868b34fc9] 2023-12-08T14:12:20.731 INFO:tasks.ceph.osd.3.smithi016.stderr: 2: (KernelDevice::_aio_thread()+0x10f6) [0x562869485de6] 2023-12-08T14:12:20.731 INFO:tasks.ceph.osd.3.smithi016.stderr: 3: ceph-osd(+0xda2fd1) [0x562869466fd1] 2023-12-08T14:12:20.731 INFO:tasks.ceph.osd.3.smithi016.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7fdb10e9f802] 2023-12-08T14:12:20.731 INFO:tasks.ceph.osd.3.smithi016.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7fdb10e3f450] 2023-12-08T14:12:20.734 INFO:tasks.ceph.osd.3.smithi016.stderr:*** Caught signal (Aborted) ** 2023-12-08T14:12:20.734 INFO:tasks.ceph.osd.3.smithi016.stderr: in thread 7fdb036f3640 thread_name:bstore_aio 2023-12-08T14:12:20.736 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-12-08T14:12:20.733+0000 7fdb036f3640 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-46-ga2cd52c0/rpm/el9/BUILD/ceph-19.0.0-46-ga2cd52c0/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7fdb03f05640 time 2023-12-08T14:12:20.692751+0000 2023-12-08T14:12:20.736 INFO:tasks.ceph.osd.3.smithi016.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-46-ga2cd52c0/rpm/el9/BUILD/ceph-19.0.0-46-ga2cd52c0/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2023-12-08T14:12:20.736 INFO:tasks.ceph.osd.3.smithi016.stderr: 2023-12-08T14:12:20.736 INFO:tasks.ceph.osd.3.smithi016.stderr: ceph version 19.0.0-46-ga2cd52c0 (a2cd52c00149b5caff897ea7f79d0529b62620da) reef (dev) 2023-12-08T14:12:20.736 INFO:tasks.ceph.osd.3.smithi016.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x562868b34fc9] 2023-12-08T14:12:20.736 INFO:tasks.ceph.osd.3.smithi016.stderr: 2: (KernelDevice::_aio_thread()+0x10f6) [0x562869485de6] 2023-12-08T14:12:20.736 INFO:tasks.ceph.osd.3.smithi016.stderr: 3: ceph-osd(+0xda2fd1) [0x562869466fd1] 2023-12-08T14:12:20.736 INFO:tasks.ceph.osd.3.smithi016.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7fdb10e9f802] 2023-12-08T14:12:20.736 INFO:tasks.ceph.osd.3.smithi016.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7fdb10e3f450] 2023-12-08T14:12:20.736 INFO:tasks.ceph.osd.3.smithi016.stderr: 2023-12-08T14:12:20.736 INFO:tasks.ceph.osd.3.smithi016.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-46-ga2cd52c0/rpm/el9/BUILD/ceph-19.0.0-46-ga2cd52c0/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7fdb03f05640 time 2023-12-08T14:12:20.692751+0000 2023-12-08T14:12:20.737 INFO:tasks.ceph.osd.3.smithi016.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-46-ga2cd52c0/rpm/el9/BUILD/ceph-19.0.0-46-ga2cd52c0/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2023-12-08T14:12:20.737 INFO:tasks.ceph.osd.3.smithi016.stderr: ceph version 19.0.0-46-ga2cd52c0 (a2cd52c00149b5caff897ea7f79d0529b62620da) reef (dev) 2023-12-08T14:12:20.737 INFO:tasks.ceph.osd.3.smithi016.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xc6) [0x562868b34fc9] 2023-12-08T14:12:20.737 INFO:tasks.ceph.osd.3.smithi016.stderr: 2: (KernelDevice::_aio_thread()+0x10f6) [0x562869485de6] 2023-12-08T14:12:20.737 INFO:tasks.ceph.osd.3.smithi016.stderr: 3: ceph-osd(+0xda2fd1) [0x562869466fd1] 2023-12-08T14:12:20.737 INFO:tasks.ceph.osd.3.smithi016.stderr: 4: /lib64/libc.so.6(+0x9f802) [0x7fdb10e9f802] 2023-12-08T14:12:20.737 INFO:tasks.ceph.osd.3.smithi016.stderr: 5: /lib64/libc.so.6(+0x3f450) [0x7fdb10e3f450] 2023-12-08T14:12:32.653 INFO:tasks.ceph.osd.3.smithi016.stderr:daemon-helper: command crashed with signal 6 2023-12-08T14:12:35.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-12-08T14:12:42.561 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~7s 2023-12-08T14:12:49.174 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~13s 2023-12-08T14:12:55.785 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~20s 2023-12-08T14:13:02.399 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~26s 2023-12-08T14:13:09.010 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~33s 2023-12-08T14:13:15.625 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~40s 2023-12-08T14:13:22.240 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~46s 2023-12-08T14:13:28.854 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~53s 2023-12-08T14:13:35.468 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~60s 2023-12-08T14:13:42.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~66s 2023-12-08T14:13:48.693 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~73s 2023-12-08T14:13:55.306 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~79s 2023-12-08T14:14:01.921 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~86s 2023-12-08T14:14:08.535 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~93s 2023-12-08T14:14:15.148 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~99s 2023-12-08T14:14:21.762 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~106s 2023-12-08T14:14:28.373 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~112s 2023-12-08T14:14:34.986 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~119s 2023-12-08T14:14:41.598 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~126s 2023-12-08T14:14:48.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~132s 2023-12-08T14:14:54.823 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~139s 2023-12-08T14:15:01.436 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~145s 2023-12-08T14:15:08.049 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~152s 2023-12-08T14:15:14.664 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~159s 2023-12-08T14:15:21.276 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~165s 2023-12-08T14:15:27.889 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~172s 2023-12-08T14:15:34.502 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~179s 2023-12-08T14:15:41.114 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~185s 2023-12-08T14:15:47.728 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~192s 2023-12-08T14:15:54.340 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~198s 2023-12-08T14:16:00.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~205s 2023-12-08T14:16:07.567 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~212s 2023-12-08T14:16:14.180 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~218s 2023-12-08T14:16:20.793 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~225s 2023-12-08T14:16:27.406 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~231s 2023-12-08T14:16:34.019 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~238s 2023-12-08T14:16:40.631 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~245s 2023-12-08T14:16:47.244 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~251s 2023-12-08T14:16:53.857 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~258s 2023-12-08T14:17:00.468 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~265s 2023-12-08T14:17:07.081 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~271s 2023-12-08T14:17:13.694 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~278s 2023-12-08T14:17:20.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~284s 2023-12-08T14:17:26.919 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~291s 2023-12-08T14:17:33.532 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~298s 2023-12-08T14:17:40.146 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~304s 2023-12-08T14:17:40.146 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-12-08T14:17:40.147 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T14:17:40.147 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T14:17:40.238 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057148,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498964k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19728 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249480k,nr_inodes=812370,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498964k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 ceph-fuse /tmp/tmp.hY9PAzp4zMtest fuse.ceph-fuse rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0 2023-12-08T14:17:40.239 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi016.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-12-08T14:17:40.239 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-12-08T14:17:40.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T14:17:40.239 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-08T14:17:40.334 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-12-08T14:17:40.334 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T14:17:40.392 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4057148,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498964k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19728 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249480k,nr_inodes=812370,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498964k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 ceph-fuse /tmp/tmp.hY9PAzp4zMtest fuse.ceph-fuse rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0 2023-12-08T14:17:40.392 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.2 is not mounted at ubuntu@smithi016.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2 2023-12-08T14:17:40.393 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2023-12-08T14:17:40.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T14:17:40.393 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-08T14:17:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-08T14:17:40.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T14:17:40.451 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T14:17:40.451 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T14:17:40.573 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056363,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2426 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-08T14:17:40.573 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.1 is not mounted at ubuntu@smithi112.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1 2023-12-08T14:17:40.573 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-12-08T14:17:40.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T14:17:40.573 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-08T14:17:40.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T14:17:40.589 INFO:teuthology.orchestra.run.smithi112.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-12-08T14:17:40.589 DEBUG:teuthology.orchestra.run.smithi112:> set -ex 2023-12-08T14:17:40.590 DEBUG:teuthology.orchestra.run.smithi112:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-08T14:17:40.646 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056363,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2426 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1301,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-08T14:17:40.646 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.3 is not mounted at ubuntu@smithi112.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.3 2023-12-08T14:17:40.646 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi112.front.sepia.ceph.com 2023-12-08T14:17:40.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-08T14:17:40.647 DEBUG:teuthology.orchestra.run.smithi112:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-08T14:17:40.702 INFO:teuthology.orchestra.run.smithi112.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-08T14:17:40.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-08T14:17:42.314 INFO:tasks.ceph.osd.0:Sent signal 15 2023-12-08T14:17:42.314 INFO:tasks.ceph.osd.1:Sent signal 15 2023-12-08T14:17:42.315 INFO:tasks.ceph.osd.2:Sent signal 15 2023-12-08T14:17:42.315 INFO:tasks.ceph.osd.4:Sent signal 15 2023-12-08T14:17:42.315 INFO:tasks.ceph.osd.5:Sent signal 15 2023-12-08T14:17:42.315 INFO:tasks.ceph.osd.6:Sent signal 15 2023-12-08T14:17:42.315 INFO:tasks.ceph.osd.7:Sent signal 15 2023-12-08T14:17:42.316 INFO:tasks.ceph.mds.a:Sent signal 15 2023-12-08T14:17:42.316 INFO:tasks.ceph.mds.c:Sent signal 15 2023-12-08T14:17:42.316 INFO:tasks.ceph.mds.b:Sent signal 15 2023-12-08T14:17:42.316 INFO:tasks.ceph.mds.d:Sent signal 15 2023-12-08T14:17:42.316 INFO:tasks.ceph.mon.a:Sent signal 15 2023-12-08T14:17:42.317 INFO:tasks.ceph.mon.b:Sent signal 15 2023-12-08T14:17:42.317 INFO:tasks.ceph.mon.c:Sent signal 15 2023-12-08T14:17:42.317 INFO:tasks.ceph.mgr.y:Sent signal 15 2023-12-08T14:17:42.317 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-12-08T14:17:42.318 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T14:17:42.314+0000 7f0ae298c640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 89484) UID: 0 2023-12-08T14:17:42.318 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T14:17:42.314+0000 7f0ae298c640 -1 osd.4 43 *** Got signal Terminated *** 2023-12-08T14:17:42.318 INFO:tasks.ceph.osd.4.smithi112.stderr:2023-12-08T14:17:42.314+0000 7f0ae298c640 -1 osd.4 43 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-08T14:17:42.318 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T14:17:42.314+0000 7f147a3f3640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 89488) UID: 0 2023-12-08T14:17:42.318 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T14:17:42.314+0000 7f147a3f3640 -1 osd.5 43 *** Got signal Terminated *** 2023-12-08T14:17:42.318 INFO:tasks.ceph.osd.5.smithi112.stderr:2023-12-08T14:17:42.314+0000 7f147a3f3640 -1 osd.5 43 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-08T14:17:42.319 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T14:17:42.314+0000 7fb32d740640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 89486) UID: 0 2023-12-08T14:17:42.319 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T14:17:42.314+0000 7fb32d740640 -1 osd.6 43 *** Got signal Terminated *** 2023-12-08T14:17:42.319 INFO:tasks.ceph.osd.6.smithi112.stderr:2023-12-08T14:17:42.314+0000 7fb32d740640 -1 osd.6 43 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-08T14:17:42.319 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T14:17:42.314+0000 7f7f9cd1e640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 89489) UID: 0 2023-12-08T14:17:42.320 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T14:17:42.314+0000 7f7f9cd1e640 -1 osd.7 43 *** Got signal Terminated *** 2023-12-08T14:17:42.320 INFO:tasks.ceph.osd.7.smithi112.stderr:2023-12-08T14:17:42.314+0000 7f7f9cd1e640 -1 osd.7 43 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-08T14:17:42.320 INFO:tasks.ceph.mds.b.smithi112.stderr:2023-12-08T14:17:42.315+0000 7f0cd7b54640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i b (PID: 96679) UID: 0 2023-12-08T14:17:42.320 INFO:tasks.ceph.mds.b.smithi112.stderr:2023-12-08T14:17:42.315+0000 7f0cd7b54640 -1 mds.b *** got signal Terminated *** 2023-12-08T14:17:42.320 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-12-08T14:17:42.315+0000 7f52ecb1a640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 96680) UID: 0 2023-12-08T14:17:42.321 INFO:tasks.ceph.mds.d.smithi112.stderr:2023-12-08T14:17:42.315+0000 7f52ecb1a640 -1 mds.d *** got signal Terminated *** 2023-12-08T14:17:42.321 INFO:tasks.ceph.mon.b.smithi112.stderr:2023-12-08T14:17:42.315+0000 7f36c1023640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 88913) UID: 0 2023-12-08T14:17:42.321 INFO:tasks.ceph.mon.b.smithi112.stderr:2023-12-08T14:17:42.315+0000 7f36c1023640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-12-08T14:17:42.321 INFO:tasks.ceph.mon.c.smithi112.stderr:2023-12-08T14:17:42.316+0000 7f0307bf2640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 88916) UID: 0 2023-12-08T14:17:42.321 INFO:tasks.ceph.mon.c.smithi112.stderr:2023-12-08T14:17:42.316+0000 7f0307bf2640 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-12-08T14:17:42.322 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T14:17:42.313+0000 7f6d0f17a640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 89452) UID: 0 2023-12-08T14:17:42.322 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T14:17:42.313+0000 7f6d0f17a640 -1 osd.0 43 *** Got signal Terminated *** 2023-12-08T14:17:42.322 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-12-08T14:17:42.313+0000 7f6d0f17a640 -1 osd.0 43 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-08T14:17:42.322 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T14:17:42.319+0000 7eff8ec05640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 89458) UID: 0 2023-12-08T14:17:42.322 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T14:17:42.319+0000 7eff8ec05640 -1 osd.2 43 *** Got signal Terminated *** 2023-12-08T14:17:42.322 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-12-08T14:17:42.319+0000 7eff8ec05640 -1 osd.2 43 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-08T14:17:42.323 INFO:tasks.ceph.mds.a.smithi016.stderr:2023-12-08T14:17:42.321+0000 7f4179de7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i a (PID: 103884) UID: 0 2023-12-08T14:17:42.323 INFO:tasks.ceph.mds.a.smithi016.stderr:2023-12-08T14:17:42.321+0000 7f4179de7640 -1 mds.a *** got signal Terminated *** 2023-12-08T14:17:42.328 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-12-08T14:17:42.326+0000 7f504518c640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i c (PID: 103883) UID: 0 2023-12-08T14:17:42.328 INFO:tasks.ceph.mds.c.smithi016.stderr:2023-12-08T14:17:42.326+0000 7f504518c640 -1 mds.c *** got signal Terminated *** 2023-12-08T14:17:42.335 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-12-08T14:17:42.326+0000 7f5425fe7640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 89172) UID: 0 2023-12-08T14:17:42.336 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T14:17:42.314+0000 7ff25129f640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 89453) UID: 0 2023-12-08T14:17:42.336 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T14:17:42.314+0000 7ff25129f640 -1 osd.1 43 *** Got signal Terminated *** 2023-12-08T14:17:42.336 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-12-08T14:17:42.314+0000 7ff25129f640 -1 osd.1 43 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-08T14:17:42.336 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-12-08T14:17:42.326+0000 7f5425fe7640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-12-08T14:17:42.729 INFO:tasks.ceph.mgr.y.smithi016.stderr:daemon-helper: command crashed with signal 15 2023-12-08T14:21:44.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:21:44.670 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:21:44.992 INFO:teuthology.orchestra.run.smithi016.stderr:ceph-osd: no process found 2023-12-08T14:29:41.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:29:41.170 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:30:11.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:30:11.342 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:30:41.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:30:41.481 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:31:11.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:31:11.627 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:31:41.743 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:31:41.747 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:32:11.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:32:11.871 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:32:41.989 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:32:41.994 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:33:12.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:33:12.207 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:33:42.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:33:42.347 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:34:12.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:34:12.493 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:34:42.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:34:42.614 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:35:12.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:35:12.756 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:35:42.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:35:42.881 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:36:13.001 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:36:13.005 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:36:43.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:36:43.136 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:37:13.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:37:13.259 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:37:43.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:37:43.416 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:38:13.535 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:38:13.539 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:38:43.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:38:43.691 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:39:13.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:39:13.838 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:39:44.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:39:44.033 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:40:14.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:40:14.148 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:40:44.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:40:44.263 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:41:14.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:41:14.512 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:41:44.688 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:41:44.693 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:42:14.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:42:14.822 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:42:44.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:42:44.963 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:43:15.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:43:15.086 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:43:45.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:43:45.208 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:44:15.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:44:15.324 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:44:45.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:44:45.657 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:45:15.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:45:15.777 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:45:45.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:45:45.911 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:46:16.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:46:16.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:46:46.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:46:46.164 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:47:16.282 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:47:16.289 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:47:46.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:47:46.411 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:48:16.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:48:16.531 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:48:46.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:48:46.656 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:49:16.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:49:16.808 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:49:46.933 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:49:46.937 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:50:17.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:50:17.093 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:50:47.217 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:50:47.222 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:51:17.349 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:51:17.353 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:51:47.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:51:47.501 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:52:17.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:52:17.657 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:52:47.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:52:47.812 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:53:17.951 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:53:17.956 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:53:48.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:53:48.143 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:54:18.253 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:54:18.257 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:54:48.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:54:48.391 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:55:18.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:55:18.540 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:55:48.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:55:48.667 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:56:18.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:56:18.799 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:56:48.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:56:48.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:57:19.032 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:57:19.036 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:57:49.164 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:57:49.168 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:58:19.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:58:19.308 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:58:49.455 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:58:49.459 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:59:19.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:59:19.597 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:59:49.752 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T14:59:49.756 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:00:19.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:00:19.884 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:00:50.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:00:50.017 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:01:20.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:01:20.146 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:01:50.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:01:50.274 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:02:20.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:02:20.429 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:02:50.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:02:50.560 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:03:20.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:03:20.687 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:03:50.858 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:03:50.864 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:04:20.979 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:04:20.983 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:04:51.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:04:51.107 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:05:21.350 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:05:21.355 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:05:51.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:05:51.486 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:06:21.613 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:06:21.618 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:06:51.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:06:51.760 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:07:21.884 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:07:21.889 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:07:51.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:07:52.003 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:08:22.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:08:22.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:08:52.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:08:52.341 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:09:22.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:09:22.508 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:09:52.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:09:52.643 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:10:22.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:10:22.768 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:10:52.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:10:52.899 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:11:23.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:11:23.095 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:11:53.227 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:11:53.232 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:12:23.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:12:23.362 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:12:53.496 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:12:53.500 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:13:23.628 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:13:23.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:13:53.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:13:53.777 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:14:23.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:14:23.903 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:14:54.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:14:54.064 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:15:24.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:15:24.182 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:15:54.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:15:54.307 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:16:24.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:16:24.446 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:16:54.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:16:54.592 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:17:24.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:17:24.747 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:17:54.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:17:54.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:18:25.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:18:25.043 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:18:55.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:18:55.173 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:19:25.291 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:19:25.296 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:19:55.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:19:55.427 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:20:25.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:20:25.558 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:20:55.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:20:55.689 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:21:25.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:21:25.822 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:21:55.946 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:21:55.951 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:22:26.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:22:26.075 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:22:56.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:22:56.205 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:23:26.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:23:26.320 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:23:56.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:23:56.458 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:24:26.585 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:24:26.590 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:24:56.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:24:56.732 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:25:26.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:25:26.863 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:25:57.003 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:25:57.008 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:26:27.134 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:26:27.139 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:26:57.257 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:26:57.261 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:27:27.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:27:27.398 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:27:57.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:27:57.546 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:28:27.672 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:28:27.676 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:28:57.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:28:57.812 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:29:27.946 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:29:27.951 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:29:58.078 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:29:58.082 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:30:28.192 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:30:28.196 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:30:58.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:30:58.321 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:31:28.447 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:31:28.451 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:31:58.570 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:31:58.574 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:32:28.709 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:32:28.714 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:32:58.848 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:32:58.854 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:33:28.971 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:33:28.976 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:33:59.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:33:59.107 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:34:29.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:34:29.238 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:34:59.364 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:34:59.369 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:35:29.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:35:29.514 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:35:59.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:35:59.645 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:36:29.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:36:29.792 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:36:59.933 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:36:59.938 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:37:30.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:37:30.066 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:38:00.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:38:00.206 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:38:30.440 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:38:30.445 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:39:00.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:39:00.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:39:30.709 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:39:30.714 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:40:00.842 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:40:00.848 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:40:30.973 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:40:30.978 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:41:01.125 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:41:01.130 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:41:31.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:41:31.249 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:42:01.384 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:42:01.391 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:42:31.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:42:31.512 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:43:01.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:43:01.666 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:43:31.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:43:31.796 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:44:01.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:44:01.944 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:44:32.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:44:32.090 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:45:02.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:45:02.237 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:45:32.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:45:32.404 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:46:02.551 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:46:02.555 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:46:32.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:46:32.688 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:47:02.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:47:02.826 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:47:32.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:47:32.980 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:48:03.130 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:48:03.135 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:48:33.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:48:33.282 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:49:03.413 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:49:03.417 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:49:33.570 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:49:33.574 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:50:03.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:50:03.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:50:33.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:50:33.883 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:51:04.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:51:04.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:51:34.180 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:51:34.184 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:52:04.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:52:04.313 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:52:34.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:52:34.437 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:53:04.571 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:53:04.576 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:53:34.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:53:34.715 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:54:04.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:54:04.881 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:54:35.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:54:35.017 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:55:05.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:55:05.172 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:55:35.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:55:35.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:56:05.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:56:05.438 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:56:35.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:56:35.570 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:57:05.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:57:05.700 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:57:35.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:57:35.814 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:58:06.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:58:06.072 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:58:36.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:58:36.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:59:06.329 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:59:06.334 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:59:36.452 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T15:59:36.456 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:00:06.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:00:06.606 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:00:36.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:00:36.753 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:01:06.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:01:06.909 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:01:37.029 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:01:37.034 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:02:07.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:02:07.163 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:02:37.290 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:02:37.294 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:03:07.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:03:07.426 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:03:37.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:03:37.559 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:04:07.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:04:07.698 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:04:37.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:04:37.813 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:05:07.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:05:07.944 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:05:38.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:05:38.088 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:06:08.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:06:08.245 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:06:38.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:06:38.381 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:07:08.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:07:08.547 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:07:38.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:07:38.685 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:08:08.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:08:08.841 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:08:38.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:08:38.973 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:09:09.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:09:09.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:09:39.246 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:09:39.250 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:10:09.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:10:09.381 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:10:39.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:10:39.513 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:11:09.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:11:09.677 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:11:39.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:11:39.808 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:12:09.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:12:09.940 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:12:40.091 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:12:40.096 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:13:10.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:13:10.228 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:13:40.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:13:40.358 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:14:10.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:14:10.482 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:14:40.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:14:40.637 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:15:10.757 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:15:10.762 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:15:40.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:15:40.906 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:16:11.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:16:11.032 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:16:41.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:16:41.163 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:17:11.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:17:11.322 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:17:41.456 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:17:41.461 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:18:11.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:18:11.590 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:18:41.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:18:41.730 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:19:11.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:19:11.885 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:19:42.018 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:19:42.022 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:20:12.144 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:20:12.148 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:20:42.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:20:42.284 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:21:12.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:21:12.430 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:21:42.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:21:42.587 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:22:12.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:22:12.733 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:22:42.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:22:42.897 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:23:13.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:23:13.028 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:23:43.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:23:43.143 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:24:13.275 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:24:13.280 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:24:43.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:24:43.427 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:25:13.570 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:25:13.574 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:25:43.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:25:43.705 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:26:13.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:26:13.836 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:26:43.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:26:43.984 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:27:14.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:27:14.140 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:27:44.241 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:27:44.246 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:28:14.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:28:14.379 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:28:44.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:28:44.501 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:29:14.629 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:29:14.634 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:29:44.781 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:29:44.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:30:14.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:30:14.933 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:30:45.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:30:45.054 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:31:15.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:31:15.320 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:31:45.461 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:31:45.465 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:32:15.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:32:15.598 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:32:45.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:32:45.755 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:33:15.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:33:15.910 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:33:46.035 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:33:46.040 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:34:16.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:34:16.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:34:46.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:34:46.298 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:35:16.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:35:16.449 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:35:46.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:35:46.570 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:36:16.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:36:16.711 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:36:46.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:36:46.842 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:37:16.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:37:16.996 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:37:47.134 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:37:47.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:38:17.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:38:17.320 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:38:47.437 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:38:47.442 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:39:17.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:39:17.572 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:39:47.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:39:47.703 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:40:17.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:40:17.835 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:40:47.985 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:40:47.991 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:41:18.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:41:18.140 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:41:48.287 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:41:48.291 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:42:18.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:42:18.419 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:42:48.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:42:48.542 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:43:18.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:43:18.665 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:43:48.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:43:48.806 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:44:18.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:44:18.936 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:44:49.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:44:49.067 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:45:19.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:45:19.208 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:45:49.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:45:49.338 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:46:19.484 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:46:19.489 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:46:49.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:46:49.625 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:47:19.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:47:19.757 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:47:49.924 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:47:49.928 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:48:20.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:48:20.067 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:48:50.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:48:50.194 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:49:20.549 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:49:20.555 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:49:50.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:49:50.681 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:50:20.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:50:20.825 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:50:50.957 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:50:50.962 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:51:21.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:51:21.109 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:51:51.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:51:51.227 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:52:21.366 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:52:21.371 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:52:51.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:52:51.490 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:53:21.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:53:21.635 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:53:51.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:53:51.753 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:54:21.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:54:21.883 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:54:51.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:54:52.002 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:55:22.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:55:22.145 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:55:52.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:55:52.283 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:56:22.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:56:22.411 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:56:52.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:56:52.544 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:57:22.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:57:22.674 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:57:52.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:57:52.814 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:58:22.948 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:58:22.953 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:58:53.096 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:58:53.101 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:59:23.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:59:23.289 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:59:53.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T16:59:53.435 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:00:23.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:00:23.582 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:00:53.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:00:53.738 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:01:23.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:01:23.864 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:01:54.003 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:01:54.008 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:02:24.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:02:24.160 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:02:54.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:02:54.301 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:03:24.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:03:24.440 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:03:54.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:03:54.578 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:04:24.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:04:24.693 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:04:54.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:04:54.821 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:05:24.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:05:24.965 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:05:55.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:05:55.090 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:06:25.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:06:25.228 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:06:55.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:06:55.350 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:07:25.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:07:25.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:07:55.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:07:55.607 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:08:25.888 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:08:25.892 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:08:56.034 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:08:56.038 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:09:26.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:09:26.202 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:09:56.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:09:56.341 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:10:26.467 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:10:26.472 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:10:56.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:10:56.603 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:11:26.737 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:11:26.741 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:11:56.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:11:56.880 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:12:27.006 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:12:27.010 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:12:57.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:12:57.249 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:13:27.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:13:27.405 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:13:57.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:13:57.560 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:14:27.711 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:14:27.716 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:14:57.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:14:57.855 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:15:27.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:15:27.984 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:15:58.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:15:58.100 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:16:28.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:16:28.255 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:16:58.388 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:16:58.392 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:17:28.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:17:28.519 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:17:58.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:17:58.658 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:18:28.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:18:28.795 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:18:58.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:18:58.939 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:19:29.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:19:29.071 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:19:59.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:19:59.226 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:20:29.353 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:20:29.358 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:20:59.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:20:59.562 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:21:29.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:21:29.719 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:21:59.870 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:21:59.874 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:22:30.016 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:22:30.020 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:23:00.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:23:00.134 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:23:30.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:23:30.278 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:24:00.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:24:00.429 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:24:30.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:24:30.559 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:25:00.685 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:25:00.689 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:25:30.820 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:25:30.825 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:26:00.948 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:26:00.953 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:26:31.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:26:31.084 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:27:01.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:27:01.215 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:27:31.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:27:31.347 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:28:01.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:28:01.472 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:28:31.615 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:28:31.620 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:29:01.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:29:01.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:29:31.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:29:31.931 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:30:02.089 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:30:02.097 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:30:32.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:30:32.226 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:31:02.384 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:31:02.389 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:31:32.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:31:32.527 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:32:02.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:32:02.674 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:32:32.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:32:32.813 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:33:02.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:33:02.935 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:33:33.059 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:33:33.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:34:03.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:34:03.205 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:34:33.364 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:34:33.369 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:35:03.495 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:35:03.499 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:35:33.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:35:33.636 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:36:03.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:36:03.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:36:33.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:36:33.914 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:37:04.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:37:04.052 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:37:34.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:37:34.188 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:38:04.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:38:04.311 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:38:34.440 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:38:34.444 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:39:04.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:39:04.558 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:39:34.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:39:34.706 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:40:04.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:40:04.829 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:40:34.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:40:34.968 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:41:05.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:41:05.092 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:41:35.237 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:41:35.241 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:42:05.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:42:05.397 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:42:35.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:42:35.545 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:43:05.671 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:43:05.675 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:43:35.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:43:35.809 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:44:05.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:44:05.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:44:36.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:44:36.063 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:45:06.206 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:45:06.211 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:45:36.348 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:45:36.352 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:46:06.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:46:06.472 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:46:36.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:46:36.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:47:06.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:47:06.760 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:47:36.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:47:36.909 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:48:07.034 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:48:07.039 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:48:37.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:48:37.203 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:49:07.345 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:49:07.349 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:49:37.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:49:37.483 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:50:07.623 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:50:07.628 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:50:37.804 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:50:37.808 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:51:07.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:51:07.948 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:51:38.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:51:38.113 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:52:08.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:52:08.241 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:52:38.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:52:38.421 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:53:08.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:53:08.777 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:53:38.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:53:38.923 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:54:09.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:54:09.048 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:54:39.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:54:39.201 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:55:09.343 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:55:09.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:55:39.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:55:39.486 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:56:09.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:56:09.605 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:56:39.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:56:39.737 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:57:09.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:57:09.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:57:40.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:57:40.050 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:58:10.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:58:10.206 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:58:40.349 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:58:40.354 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:59:10.493 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:59:10.498 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:59:40.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T17:59:40.653 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:00:10.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:00:10.781 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:00:40.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:00:40.932 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:01:11.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:01:11.086 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:01:41.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:01:41.226 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:02:11.351 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:02:11.355 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:02:41.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:02:41.496 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:03:11.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:03:11.641 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:03:41.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:03:41.775 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:04:11.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:04:11.897 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:04:42.035 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:04:42.040 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:05:12.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:05:12.178 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:05:42.321 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:05:42.326 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:06:12.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:06:12.473 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:06:42.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:06:42.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:07:12.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:07:12.758 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:07:42.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:07:42.914 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:08:13.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:08:13.069 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:08:43.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:08:43.208 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:09:13.358 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:09:13.363 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:09:43.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:09:43.492 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:10:13.635 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:10:13.640 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:10:43.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:10:43.781 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:11:13.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:11:13.911 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:11:44.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:11:44.055 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:12:14.181 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:12:14.186 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:12:44.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:12:44.320 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:13:14.430 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:13:14.434 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:13:44.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:13:44.585 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:14:14.719 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:14:14.723 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:14:44.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:14:44.870 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:15:14.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:15:15.000 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:15:45.134 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:15:45.139 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:16:15.273 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:16:15.277 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:16:45.478 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:16:45.483 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:17:15.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:17:15.602 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:17:45.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:17:45.752 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:18:15.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:18:15.907 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:18:46.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:18:46.046 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:19:16.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:19:16.176 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:19:46.321 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:19:46.325 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:20:16.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:20:16.461 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:20:46.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:20:46.602 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:21:16.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:21:16.728 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:21:46.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:21:46.867 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:22:16.985 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:22:16.990 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:22:47.117 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:22:47.121 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:23:17.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:23:17.236 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:23:47.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:23:47.401 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:24:17.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:24:17.524 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:24:47.665 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:24:47.670 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:25:17.812 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:25:17.817 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:25:47.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:25:47.941 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:26:18.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:26:18.103 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:26:48.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:26:48.248 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:27:18.383 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:27:18.388 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:27:48.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:27:48.526 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:28:18.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:28:18.657 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:28:48.799 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:28:48.803 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:29:18.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:29:18.967 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:29:49.127 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:29:49.131 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:30:19.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:30:19.249 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:30:49.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:30:49.380 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:31:19.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:31:19.505 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:31:49.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:31:49.653 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:32:19.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:32:19.790 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:32:49.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:32:49.929 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:33:20.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:33:20.069 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:33:50.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:33:50.201 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:34:20.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:34:20.339 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:34:50.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:34:50.520 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:35:20.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:35:20.675 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:35:50.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:35:50.825 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:36:20.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:36:20.970 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:36:51.112 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:36:51.116 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:37:21.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:37:21.247 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:37:51.398 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:37:51.402 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:38:21.518 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:38:21.523 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:38:51.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:38:51.675 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:39:21.806 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:39:21.811 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:39:51.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:39:51.934 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:40:22.075 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:40:22.080 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:40:52.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:40:52.219 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:41:22.340 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:41:22.347 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:41:52.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:41:52.485 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:42:22.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:42:22.607 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:42:52.734 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:42:52.739 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:43:22.858 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:43:22.862 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:43:52.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:43:53.002 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:44:23.119 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:44:23.124 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:44:53.259 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:44:53.264 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:45:23.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:45:23.395 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:45:53.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:45:53.526 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:46:23.645 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:46:23.650 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:46:53.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:46:53.788 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:47:23.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:47:23.912 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:47:54.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:47:54.060 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:48:24.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:48:24.174 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:48:54.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:48:54.306 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:49:24.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:49:24.419 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:49:54.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:49:54.551 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:50:24.671 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:50:24.676 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:50:54.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:50:54.822 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:51:24.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:51:24.965 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:51:55.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:51:55.119 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:52:25.246 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:52:25.251 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:52:55.369 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:52:55.373 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:53:25.499 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:53:25.503 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:53:55.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:53:55.808 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:54:25.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:54:25.942 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:54:56.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:54:56.094 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:55:26.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:55:26.238 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:55:56.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:55:56.604 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:56:26.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:56:26.744 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:56:56.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:56:56.881 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:57:26.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:57:26.997 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:57:57.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:57:57.128 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:58:27.247 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:58:27.252 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:58:57.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:58:57.382 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:59:27.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:59:27.506 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:59:57.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T18:59:57.642 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:00:27.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:00:27.761 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:00:57.896 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:00:57.900 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:01:28.018 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:01:28.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:01:58.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:01:58.168 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:02:28.286 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:02:28.290 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:02:58.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:02:58.428 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:03:28.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:03:28.592 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:03:58.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:03:58.721 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:04:28.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:04:28.880 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:04:59.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:04:59.011 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:05:29.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:05:29.156 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:05:59.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:05:59.294 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:06:29.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:06:29.458 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:06:59.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:06:59.593 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:07:29.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:07:29.753 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:07:59.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:07:59.907 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:08:30.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:08:30.246 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:09:00.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:09:00.402 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:09:30.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:09:30.542 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:10:00.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:10:00.673 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:10:30.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:10:30.797 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:11:00.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:11:00.940 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:11:31.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:11:31.059 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:12:01.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:12:01.194 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:12:31.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:12:31.332 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:13:01.457 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:13:01.462 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:13:31.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:13:31.602 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:14:01.745 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:14:01.749 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:14:31.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:14:31.881 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:15:02.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:15:02.035 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:15:32.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:15:32.161 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:16:02.341 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:16:02.346 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:16:32.472 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:16:32.476 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:17:02.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:17:02.616 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:17:32.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:17:32.745 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:18:02.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:18:02.884 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:18:33.003 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:18:33.007 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:19:03.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:19:03.147 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:19:33.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:19:33.271 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:20:03.398 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:20:03.402 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:20:33.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:20:33.526 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:21:03.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:21:03.665 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:21:33.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:21:33.787 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:22:03.915 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:22:03.919 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:22:34.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:22:34.051 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:23:04.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:23:04.184 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:23:34.300 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:23:34.305 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:24:04.440 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:24:04.445 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:24:34.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:24:34.568 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:25:04.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:25:04.725 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:25:34.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:25:34.856 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:26:04.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:26:04.989 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:26:35.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:26:35.121 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:27:05.258 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:27:05.262 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:27:35.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:27:35.390 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:28:05.541 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:28:05.546 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:28:35.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:28:35.696 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:29:05.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:29:05.835 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:29:35.986 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:29:35.990 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:30:06.133 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:30:06.138 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:30:36.264 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:30:36.269 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:31:06.413 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:31:06.418 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:31:36.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:31:36.531 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:32:06.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:32:06.668 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:32:36.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:32:36.832 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:33:06.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:33:06.970 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:33:37.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:33:37.109 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:34:07.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:34:07.246 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:34:37.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:34:37.378 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:35:07.496 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:35:07.500 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:35:37.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:35:37.623 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:36:07.750 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:36:07.755 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:36:37.881 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:36:37.885 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:37:08.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:37:08.016 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:37:38.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:37:38.194 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:38:08.316 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:38:08.321 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:38:38.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:38:38.630 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:39:08.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:39:08.776 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:39:38.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:39:38.924 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:40:09.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:40:09.053 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:40:39.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:40:39.209 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:41:09.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:41:09.348 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:41:39.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:41:39.486 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:42:09.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:42:09.617 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:42:39.758 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:42:39.763 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:43:09.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:43:09.902 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:43:40.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:43:40.066 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:44:10.217 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:44:10.221 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:44:40.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:44:40.350 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:45:10.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:45:10.483 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:45:40.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:45:40.637 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:46:10.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:46:10.774 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:46:40.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:46:40.922 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:47:11.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:47:11.070 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:47:41.212 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:47:41.218 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:48:11.350 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:48:11.355 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:48:41.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:48:41.478 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:49:11.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:49:11.624 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:49:41.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:49:41.771 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:50:11.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:50:11.926 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:50:42.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:50:42.041 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:51:12.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:51:12.182 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:51:42.300 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:51:42.304 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:52:12.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:52:12.436 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:52:42.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:52:42.567 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:53:12.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:53:12.698 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:53:42.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:53:42.822 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:54:12.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:54:12.977 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:54:43.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:54:43.109 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:55:13.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:55:13.240 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:55:43.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:55:43.362 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:56:13.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:56:13.493 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:56:43.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:56:43.616 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:57:13.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:57:13.768 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:57:43.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:57:43.912 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:58:14.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:58:14.043 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:58:44.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:58:44.174 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:59:14.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:59:14.314 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:59:44.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T19:59:44.429 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:00:14.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:00:14.551 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:00:44.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:00:44.674 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:01:14.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:01:14.814 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:01:44.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:01:44.928 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:02:15.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:02:15.073 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:02:45.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:02:45.213 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:03:15.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:03:15.353 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:03:45.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:03:45.499 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:04:15.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:04:15.622 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:04:45.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:04:45.776 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:05:15.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:05:15.930 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:05:46.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:05:46.055 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:06:16.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:06:16.192 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:06:46.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:06:46.343 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:07:16.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:07:16.480 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:07:46.630 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:07:46.634 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:08:16.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:08:17.888 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:08:48.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:08:48.017 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:09:18.154 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:09:18.159 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:09:48.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:09:48.299 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:10:18.423 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:10:18.427 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:10:48.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:10:48.566 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:11:18.709 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:11:18.714 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:11:48.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:11:48.872 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:12:19.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:12:19.010 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:12:49.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:12:49.155 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:13:19.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:13:19.312 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:13:49.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:13:49.459 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:14:19.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:14:19.597 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:14:49.719 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:14:49.724 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:15:19.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:15:19.866 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:15:49.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:15:50.004 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:16:20.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:16:20.145 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:16:50.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:16:50.282 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:17:20.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:17:20.451 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:17:50.629 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:17:50.633 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:18:20.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:18:20.768 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:18:50.885 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:18:50.890 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:19:21.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:19:21.023 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:19:51.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:19:51.145 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:20:21.298 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:20:21.303 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:20:51.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:20:51.495 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:21:21.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:21:21.656 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:21:51.773 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:21:51.780 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:22:21.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:22:21.926 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:22:52.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:22:52.047 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:23:22.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:23:22.180 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:23:52.322 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:23:52.327 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:24:22.467 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:24:22.471 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:24:52.607 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:24:52.611 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:25:22.753 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:25:22.759 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:25:52.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:25:52.913 DEBUG:teuthology.orchestra.run.smithi112:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-08T20:26:12.030 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2023-12-08T20:26:12.031 DEBUG:teuthology.task.console_log:Killing console logger for smithi016 2023-12-08T20:26:12.031 DEBUG:teuthology.task.console_log:Killing console logger for smithi112 2023-12-08T20:26:12.031 DEBUG:teuthology.exit:Finished running handlers